11

3DX Innovation3DX Innovation

XcannerXcanner

the Plastic-Clay 3D Surface Modeler

ENSC 440, Group #1 Monday April 16ENSC 440, Group #1 Monday April 16 thth, 2007, 2007

22

AgendaAgenda

Introduction Xcanner System Future Work Conclusion, Q&A Demonstration

33

IntroductionIntroduction

Motivation

Chief Executive Officer: Gina Chang

Chief Hardware Officer: Kevin Hu

Chief Operation Officer: Jeff Liu

Chief Software Officer: Sean Sun

3DX Team

• Expertise • Passion • 440

Company Mascot: Squeezy

44

Market ResearchMarket Research

Emphasis on Quality expectation Accuracy

Emphasis on Color & Texture Low cost

Small or new movie / gaming studios

Medical & manufacturing

Movie & gaming industry

Target

55

• Inexpensive

• Portable

• Fast3DX Innovation

Xcanner Price

Xcanner Goals

• True Color Info

• High Resolution

• Exportable Model3DX Innovation

Xcanner Quality

Existing ScannersExisting Scanners

Higher Price & Performance Quality

Lower Price & Performance Quality

Stephan Pleines’s 3D Scanner

No color

No export

DAVID Hi-end Planar Surface Scanner

No color

Face / planar presentation

No 360 3D info available

NextEngine: Hi-end Desktop Scanner

Texture

US $2,495

CyberWare Model Shop Color 3D Scanner

Color

Large Objects

US $97,000

CyberWare Desktop 3D Scanner (Model 15)

No color

10” x 6” x 3”

US $28,600

66

Timeline

System Integration

Control Block

Hardware Assembly System Prototyping

Technical Refinement

Jan 27 Feb 12 Mar 4 Apr 4Mar 12

Jan 27

Feb 11 Mar 1

Apr 8Feb 22

Time

Time

Est

imat

edA

ctu

al

77

Xcanner OverviewXcanner Overview

Shape

Color

Reconstruct the Shape & Map

the Color

88

System Block DiagramSystem Block Diagram

Image Capturing Platform

ImageProcessing

Module

Action Handler

GUI

Hardware Software

CAD & 3DX File

USB Model Exporting Module

Control Unit

Internal Data Structure

Raw Image Data

99

Image Processing Module (IPM)Image Processing Module (IPM)

Outline

1. Design Concept

2. Design Overview

3. Design Analysis

4. Requirements- COO Jeff Liu -

1010

Curvature Detection: ConceptCurvature Detection: Concept

Front View @ a Specific Curvature

View angle = 0 No information on curve variation

1111

Curvature Detection: ConceptCurvature Detection: Concept

Side View @ a Specific Curvature

View angle > 0 Information on curve variation available

1212

System Block DiagramSystem Block Diagram

Image Capturing Platform

ImageProcessing

Module

Action Handler

GUI

Hardware Software

CAD & 3DX File

USB Model Exporting Module

Control Unit

Internal Data Structure

Raw Image Data

1313

Module Block DiagramModule Block Diagram

Raw Image Data

Pre-Processing

Curvature Detection

Color Mapping

Coordinate Transform & Adjust

Internal Data Structure

Data Acquisition

Data Processing

Data Output

1414

Image Pre-ProcessingImage Pre-Processing

Center of Rotation

Color Grayscale

Noise Reduction

1515

Curvature DetectionCurvature Detection

• Binary Condition

Noiseless Environment

Filtering

Trimming

Thresholding

Line Scan

1616

Coordinate TransformCoordinate Transform

theta = angle of rotation of the target objecttheta = angle of rotation of the target object x’ = xcos(theta)x’ = xcos(theta) y’ = xsin(theta)y’ = xsin(theta) z’ = yz’ = y

1717

Surface ReconstructionSurface Reconstruction

Combine curvatures

Map color information

1818

System RequirementsSystem Requirements

Scan Conditions Noise-free Concentrated lighting White light

Positioning Fixed location Rotation control Centering position

1919

Image Capturing Platform (ICP) Image Capturing Platform (ICP) & Exporting Module& Exporting Module

- CHO Kevin Hu -

Outline

1. ICP Overview

2. Curvature Extraction

3. Required Devices

4. Exporting Module

5. Summary

2020

System Block Diagram ReviewSystem Block Diagram Review

Action Handler

GUI

Hardware Software

USB

Control Unit

Image Capturing Platform

ImageProcessing

Module

CAD & 3DX File

Model Exporting Module

Internal Data Structure

Raw Image Data

2121

Image Capturing Platform OverviewImage Capturing Platform Overview

Objectives

Projecting method

Required devices

1. Exposes Curvature

2. Creates Suitable Scanning Environment

2222

Extracting Curvature IExtracting Curvature I

Project a line on the object Capture Shape

Laser

2323

Extracting Curvature IIExtracting Curvature II

Light

Capture Color

2424

Required DevicesRequired Devices

Motor 360-degree

rotation

Light Source LED Light Line Laser

Color & Shape info

Camera Collect Data

2525

System Block Diagram ReviewSystem Block Diagram Review

Action Handler

GUI

Hardware Software

USB

Control Unit

Image Capturing Platform

ImageProcessing

Module

CAD & 3DX File

Model Exporting Module

Internal Data Structure

Raw Image Data

2626

Exporting ModuleExporting Module

.3DX File

.OBJ File For user application

For Xcanner model viewer

Internal Data Structure

2727

SummarySummary

Image Capturing Platform Curvature Projection &

Capture Scanning environment

Exporting Module Export to CAD model

Device Controls Rotation CCD Image Captures User Interface

2828

Control UnitControl Unit

- CSO Sean Sun -

Outline

1. Control Unit Overview

2. GUI Display

3. Sample Results

4. System Performance

5. Summary

2929

Control Unit OverviewControl Unit Overview

Role: Bringing all modules together

Action Handler

GUI

USB

Control Unit

Model Generating & Displaying Modules

Camera & Motor Control Utilities

Image Capturing Platform

ImageProcessing

Module

CAD & 3DX File

Model Exporting Module

Internal Data Structure

Raw Image Data

3030

Control UnitControl Unit

Functional Call

3131

GUIGUI

3232

Results IResults I

3333

Results IIResults II

3434

Results IIIResults III

3535

System PerformanceSystem Performance

For an object with dimension 9cm x 9cm x 9cm

Resolution: 0.5mm X, Z plane Accuracy: < 4% Y plane Accuracy: < 1% Accuracy increases for smaller objects

3636

SummarySummary

User input handling User control in scanning sequence System feedback Task parallelism Solution to hardware limitation

3737

Overall SummaryOverall Summary

-CEO Gina Chang-

Outline

1. Finance

2. Future Work

3. We’ve Learned

4. Demonstration

3838

FinanceFinance

Production Items Actual ($)

Webcam 100

Servo motor kit 70

Light source 30

Enclosure *0

Miscellaneous apparatus 30

Product Total Cost $ 230* Could vary upon the source of material chosen

R & D - $510

R & D - $270

Production - $250

Production - $230

$ 760

$ 500

Estimated

Actual Actual

3939

Future WorkFuture Work

Multiple-Angle Camera Capturing Positioning & shapes Correction & verification

Automatic lighting control Manufacturing standardization Processing time improvement

Publication of paper Startup company

Technical

Others

4040

What We’ve LearnedWhat We’ve Learned

• Project Management

• Team Work

Any lastly…..

• Technical Aspect

4141

the “ Fantastic 4 ”the “ Fantastic 4 ”Group DynamicsGroup Dynamics

3DX “Angels”The Life-long Friendships

4242

AcknowledgementAcknowledgement

Dr. Lakshman One

Mr. Steve Whitmore

TA: Brad Oldham Vinay Kotamraju Amir Nioumand

4343

Questions?

4444

Supplement - Index

Gina Chang

Kevin Hu

Jeff Liu

Sean Sun

4545

Supplement – OBJ Format

mtllib master.mtlmtllib master.mtl

vv 0 2 20 2 2v v 0 0 20 0 2v v 2 0 22 0 2v v 2 2 22 2 2……..……..

g face1g face1usemtl redusemtl redf 1 2 3 4f 1 2 3 4

g face2g face2……..……..

Coding Format Display Problem

Can’t Display Can’t Display ColorColor Too many facesToo many faces

ReduceReduce #:#: VerticesVertices FacesFaces

Compressing Compressing DistortionDistortion

newmtl redKd 1 0 0illum 0

4646

Supplement – Stl Format

solid cube_corner solid cube_corner facet normal 0.0 0.0 1.0 facet normal 0.0 0.0 1.0 outer loop outer loop

vertex 1.0 1.0 0.0 vertex 1.0 1.0 0.0 vertex -1.0 1.0 vertex -1.0 1.0

0.0 0.0 vertex 0.0 -1.0 vertex 0.0 -1.0

0.0 0.0 endloop endloop endfacetendfacet... …... …... …... …

endsolid endsolid

Coding Format Drawbacks

Larger in file size 3 times larger

Harder in color display

4747

Supplement – Rotating Base

Friction Size Weight Capacity Stability

Prototype 1 Prototype 2

Contact area reduction Larger plate Gear stability

4848

Supplement – Hardware I

360 degrees rotation required

Servo Motor: up to 180 Add-on gear

Rotation Motor Calibration

Problem: Servo motor doesn’t report its position accurately

Array of adjustments for every region of rotation

4949

Supplement – Hardware II

Light intensity consistency Battery drains

Modify with AC power adaptor

Intensity and uniform direction Slit

Lighting

White light Poses the least effect on

object’s color White light borrowed from

Brad:High Intensity Low Intensity

5050

Supplement – Background I

Dark (Black)

Non-reflective

Requirements Possible Implementations

Paint with charcoal smeared on top Paint with mixture of charcoal & paint Pure paint only

5151

Supplement – Background II

Image Comparison

Before After

5252

Supplement – Extracting Curvature

Three potential mechanisms

Shadow

Laser

Shade

String

Laser Only

Laser & Light

Structure Light

5353

Supplement – Scan Topology I: Structure Light, No Shade

Too much noise

Matlab Model

Image w/ Edge Detection

Image w/ Shade

No clean edges

5454

Supplement – Scan Topology II: Shade

Corner case

5555

Supplement – Scan Topology III: String

Too much noise

Shadow is not strong enough

Not as disguisable

String

Matlab Model

5656

Supplement – Scan Topology IV: Laser

No color information

5757

Supplement – Scanning Topology Comparison

Shadow Laser

MethodsPure Light

Shade String LaserLaser

& Light

S/N Ratio

Low High Low High High

Clean Edge

No Yes Merely Yes Yes

Color Info

Yes Yes Yes No Yes

5858

Supplement – Perspective Distortion I

Key: Larger when closer to camera

Distortion is proportional to: Size of the object Distance between object &

Camera

5959

Supplement – Perspective Distortion II

6060

Supplement – Perspective Distortion III

6161

Supplement – Perspective Distortion IV

Sampled surface points to camera lens distance

Superposition: separate dimensions Horizontal Vertical

CCD Camera issue

Distortion Correction

Rotation Axis =

Reference Point

6262

Supplement – Distortion Correction (Side view)

uncorrecteduncorrected corrected

6363

Supplement – Distortion Correction (Top view)

uncorrecteduncorrected corrected

6464

Supplement – Multi-angle Camera Capturing

1. Reduce blind spots

2. Correct distortion

3. Speed up processing time

6565

Supplement – Training

Line laser Surface highest point Focal center of camera lens

Positioning: must align with axis of rotation

6666

Supplement – Problems w/ Misalignment

Misalignment Scenario (wrt axis of rotation)

Resulting Distortion

Surface Highest Point

Geometric Center

Line Laser

Perspective Distortion

3D Model Distortion

View Angle Distortion

6767

Target AnalysisTarget Analysis

• Sale of 3D laser scanning in year 2006

= $ 253 million

Potential

• Emerging market in video games in year 2006

= $12.5 billion

Incentive

• $ 95 per finished second

• $ 1,000 per scene

Target Small & new movie or gaming studio

3D animation

6868

Supplement – Existing 3D Scan I

Crude Modeling

Stephan Pleines’s 3D scanner

No Color No Export

6969

Supplement – Existing 3D Scan II

Accurate modeling No color

DAVID: Hi-end Planar Surface Scan

7070

Supplement – Existing 3D Scan III

Superb modeling

Texture

US$2,495

NextEngine: Least expensive Hi-End Desktop Scanner

7171

Supplement – Source of ErrorSupplement – Source of Error

Limitation of single cameraLimitation of single camera Perspective Distortion Perspective Distortion Camera DistortionCamera Distortion Thickness of laser projectionThickness of laser projection Lighting Requirement Lighting Requirement Discontinuity in motor rotationDiscontinuity in motor rotation Flatness of supporting baseFlatness of supporting base Displacement of rotational axis Displacement of rotational axis

7272

Supplement – 3D Projection on 2D Plane Original point, [x, y, z, 1]T

World Transform Camera Transform Perspective Transform Final coordinates, [x'/ω', y'/ω']T

'

'

'

'

][][][

1 z

y

x

transformePerspectivtransformCameratransformWorldz

y

x

7373

Supplement – Rotation Vector Representation

X` = RX, where X is the original coordinates, X` is the new coordinates R is the Rotation Vector

7474

Supplement – Rodrigues Rotation Formula

7575

Supplement – Applying Rodrigues Formula

]00[ y

7676

Supplement – Perspective Projection

F, focal length

x Is the projection coordinates

7777

Supplement – Distortion Analysis

Barrel Distortion

Pincushion Distortion

Combined Effect

7878

Supplement – Test With Target Sheet

Target SheetPhotographed Target

Sheet

7979

Supplement – Considering Distortion

8080

Supplement – Applying Theories

Correct Camera distortionCorrect Camera distortion Reverse Perspective Transform Reverse Perspective Transform Verify results with Rodrigues FormulaVerify results with Rodrigues Formula Generate optimal 3D coordinatesGenerate optimal 3D coordinates

8181

Supplement - Resources

Automatic Correction of Lens Distortion by Using Digital Image Automatic Correction of Lens Distortion by Using Digital Image ProcessingProcessing, Harri Ojanen, July 10, 1999, Harri Ojanen, July 10, 1999

3D Transformation & Camera Calibration3D Transformation & Camera Calibration, Jean-Yves Bouguet, April , Jean-Yves Bouguet, April 3, 19973, 1997

Structure and Motion ReconstructionStructure and Motion Reconstruction, Pietro Perona, April 17, Pietro Perona, April 17thth, 1997, 1997

Error Analysis in 3D ReconstructionError Analysis in 3D Reconstruction, Pietro Perona, Spring 1997, Pietro Perona, Spring 1997

Camera Calibration Toolbox for MatlabCamera Calibration Toolbox for Matlab, , http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/parameters.http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/parameters.htmlhtml

Camera CalibrationCamera Calibration, , http://mblog.lib.umich.edu/blimpbots/archives/2007/02/cameras.htmlhttp://mblog.lib.umich.edu/blimpbots/archives/2007/02/cameras.html

8282

Supplement –Applied Technology

Java Development Kit

Java Media Framework

Phidget21 Java JNI Library

Ant Automatic Build Tool