3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli

3D Processing and metadata

inges1on at POLIMI

Gabriele Guidi*

Sara Gonizzi Barsan1

Laura Loredana Micoli

Politecnico di Milano ‐ Mechanical Engineering Dept.

Project

•  3 years EU-ICT pilot project

•  Aim: supply Europeana with 3D items such as:

– Archaeological sites

– Architectures

– Monuments

– Artifacts

Including UNESCO World Heritage assets

http://3dicons-project.eu

Project numbers

•  16 Partners

•  11 Countries

•  3000 3D models + metadata

•  36 months (30 months acquisition phase)

•  Project: 100 models/month (average)

•  POLIMI unit: 537 3D models + metadata

•  POLIMI unit: 18 items/month (average)

Massive digitization project, each production step �

has to be greatly optimized! �

Massive Digitization (Libraries)Book

(physical object)

Bibliographic record (descriptive metadata)

create/convert

2D scan

Digital object

storing

Repository

(technical metadata)

Metadata Record

Digital object url

Massive Digitization (3DICONS)

storing

Repository

(technical metadata)

CH Asset(physical object)

3D capture

3D model(digital object)

(descriptive metadata)

create/convert

Digital object url

Metadata Record

Source of Heritage Assets for POLIMI �The Archaeological Museum in Milan

The architectural structure Settled upon a complex stratification of archeological ruins, tangible sign of the ancient role of Milan as Capital of the Western Roman Empire

The content

1000+ archaeological items including: •  epigraphs •  statues

•  mosaics •  furniture

•  potteries related to Greek, Etruscan, Roman and Medieval periods

•  Specific skill required

The complex ar1cula1on of data requires a high level of

exper1se in the archaeological/historical field

•  Time consuming process Collec1ng the informa1on required for arranging

suitable descrip1ve metadata might require more 1me

than allowed by the project dura1on

Descriptive metadata (85%)

Technical metadata definition (1%)

3D acquisition and modeling (14%)

Metadata creation

✗ Only Heritage Assets with pre-existing metadata

have been chosen: conversion instead of creation�

•  POLIMI source of metadata is SIRBeC (Information System of Cultural Heritage of the Lombardia Region)

•  All records can be exported in xml format

•  The SIRBeC data structure is compliant with the CARARE metadata schema, used by 3DICONS as reference for structuring their metadata

•  Only metadata mapping has to be designed

Metadata conversion

Data collec1on workflow

3D data collection

Image‐based

modelling

triangula1on‐

based systems TOF system

Small

texturized

objects

Small un‐

texturized

objects

Buildings

(77%)

(14%)

(9%)

SFM•  SFM is therefore the most used technology in

this project

•  but SFM is nearly a "black box" giving an output with little of no way of intervention on the final output

•  the only controllable inputs are good quality images

We need an optimized image acquisition protocol

in order to maximize the quality of 3D output �

Possible imaging problems

•  Image blurring due to: –  Movement on shooting

–  Wrong focusing

–  Limited Depth of Field

•  Lighting/dynamic range –  Backlights/mixed color temp.

–  Light spots

–  Highlights

•  Confusing scene elements –  Painted walls/mosaics

–  High contrast elements around the subject

DEPTH OF FIELD

Aperture tests on a small artifact

•  Camera: Canon 5D mkII

•  Sensor: Full frame CMOS 21.1 Mpixel

•  Lens: 50mm macro

•  Manual focusing on the

left eye @ x10

•  Camera-target distance: 22.5 cm

•  Avg. GSD: 28µm

15 cm

7.5 cm

Aperture F 2.5

DOF @22.5 cm

2.4 mm

Focal Plane (FP)

FP + 23mm

Aperture F 5.6

DOF @22.5 cm

5.1 mm

FP

FP + 23mm

Aperture F 11

DOF @22.5 cm

10.2 mm

FP

FP + 23mm

Aperture F 22

DOF @22.5 cm

20.4 mm

FP

FP + 23mm

Aperture F 32

DOF @22.5 cm

28.8 mm

FP

FP + 23mm

SFM/matching at different aperturesFront Le^ Right

•  Automatic identification of tie points•  Image orientation•  Dense color cloud image matching (high)

All processing was made with AGISOFT Photoscan

Matchable points vs. aperture

F Tie

points

Dense cloud

size

2.5 200 420637

Focal zone: 22.4 cm – 22.6 cm


F Tie

points

Dense cloud

size

2.5 200 420637

5.6 408 1286552



F Tie

points

Dense cloud

size

2.5 200 420637

5.6 408 1286552

11 694 2827132

Focal zone: 22 cm – 23 cm


F Tie

points

Dense cloud

size

2.5 200 420637

5.6 408 1286552

11 694 2827132

22 848 3854555



F Tie

points

Dense cloud

size

2.5 200 420637

5.6 408 1286552

11 694 2827132

22 848 3854555

32 1033 4117599


LIGHTING/DYNAMIC RANGE

Light filtering on shiny surfaces

No filter

Circular

polarizer

Light filtering

No filter Circular

polarizer

Matchable points vs. filtering

Filtering Tie

points

Dense

cloud size

None 978 1502661

Circular

polarizer

1051 1572751

+5% matched points

HDR processing

•  Enhances details in images containing both overexposed and underexposed areas

•  Allows therefore to increase the number of points in image matching

•  Since the SW manages jpegs only the full HDR is tone mapped and converted in 24 bit RGB

•  This allows to increase by the number of matchable points in the darker areas

HDR Processing example

0.60s, f/32Exposure OK

2.50s, f/32+2 stop

0.15s, f/32-2 stop

Tone mappedHDR

•  4 groups of 3 shots have been taken from different orientations

•  SFM with:

–  Properly exposed shots

–  Corresponding one mapped shots

Matchable points vs. processing

Processing Tie

points

Dense

cloud size

None 5467 5637516

HDR 5646 5858700

+4% matched points

CONFUSING SCENE ELEMENTS�

Masking images

Without mask

Withmask

Photo shooting with backgrounds

Corresponding landing page

A few examples - Archaeological Museum, Milan

Texturized mesh models from SFM

Good practices adopted

Issues

•  Image blurring due to –  Movement on shooting

–  Wrong focusing

–  Limited Depth of Field

•  Lighting/dynamic range –  Backlights/mixed color temp.

–  Light spots

–  Highlights

•  Confusing scene elements –  Painted walls/mosaics

–  High contrast elements around the subject

Shooting/pre-processing solutions

–  Tripod

–  Manual focusing @ 10x

–  Small apertures (16-32)

–  Light shielding panels

–  Mask post processing

–  HDR/Polarizer filter

–  Black/white background •  hides confusing elements

•  speeds up masking

Conclusions•  The best practices of a massive

3D digitization project has been shown

•  Metadata conversion from preexisting sources was needed for quickly generating searchable material

•  SFM is a key technology for shortening 3D digitization to a sustainable level

•  No many intervention is possible on SFM, the only actual action is improving image quality

•  Proper imaging protocols may increase the 3D model quality and the success rate, in possible bad environmental conditions.

Thank you for your attention

[email protected]

Internet

3D processing and metadata ingestion at POLIMI, Gabriele Guidi, Sara Gonizzi Barsanti and Laura Loredana Micoli