1
NIWeek Vision SummitAugust 2-3, 2011 Austin, Texas
www.ni.com/niweek/summit_vision
2
Breaking New Ground with Vision Inspection Systems
Dr. Dan MilkieSenior Developer
Coleman Technologies, Inc.
NIWeek Vision SummitAugust 2–3, 2011 Austin, Texas
What if there’s no existing solution?• Prototyping tips
• Planning guide
• Implementation
• Imaging algorithms– Finding small defects– 3D laser triangulation
• Final lessons
3
About us
• 16 year National Instruments Alliance Partner
• Basler Vision System Integrator
• NI Certified Developers & NI Professional Instructors
• Advanced engineering & science degrees
4
Vision Applications
5
Industrial
Research
– Dinnerware defect detection system – Dinnerware color pattern inspection system– Robotic seed germination classifier– High-speed seed counting system– Laser drilling inspection– Thin film defect identification– Mirror defect detection– PCB contact pad inspection system– Color tablet tracking– Particle size analysis (powders)– Particle size analysis (liquid suspension)– Glass rod inspection– Well plate inspection system– Bio-sample thermal imaging– Wellplate imaging system– Crystal finder/classifier– Biaxial tissue tester– Two-photon microscopy
Plate Inspection System
The ChallengeDinnerware Inspection
•Many different dishes
•1 dish per second
•Most difficult defect:– White bumps on
white plates
6
Defects
Can we image the defects?• Prototype with what you have
(or loaners)– Area-scan camera (GigE)– Lighting
• Directional : Desk lamp• Broad sources : Room lights, diffusers
– NI Vision Development Module
• Optimize lighting & camera placement– Tip : Replace camera with your eye
7
Okay, I see the defects, but can the computer?
8
• Start with NI Vision Assistant– Fastest way to test
processing functions– Estimate time using the
“Performance Meter”– Quickly turn scripts into
LabVIEW VIs
Okay, I see the defects, but can the computer?
• Start with NI Vision Assistant– Fastest way to test
processing functions– Estimate time using the
“Performance Meter”– Quickly turn scripts into
LabVIEW VIs
9
Proof of Principle• Are you confident in your plan yet?• If not, prototype in LabVIEW!
– Combine acquisition and analysis– Tip : Use tester with fresh data or saved image sets
• Practice good coding style– Prototypes Final version– Good code encourages trying new ideas– Documentation should be automatic
10
Going for it
Ready to build your system?•Know your test set, criteria
•Balance goals with sliding scales instead of all-or-nothing
•Solve most pressing issues first– Example : Our 1st generation machine tested 1 plate type
(their most popular) and only 4 defect types
11
Modular, Modular, ModularIndependent Stations1.Bottom view2.3D imaging3.Top view
Lesson learned : •Needed to add baffles•Found interference between stations•Changed how plates cross gaps•Vibration issues
12
NI PCIe-1430Dual CameraLink
NI PCIe-1430Dual CameraLink
NI PCIe-82354x GigE
NI PCIe-82354x GigE
PCPC
1 2 3
Small defect imaging• First version :
• Next version:
• Defects show up as dark spots:
13
LEDlightLEDlight
Line
scan
Ca
mer
a
Line
scan
Ca
mer
a
Glancing Angle
Transmission
Small Defects are Hard to Find!• Problems :
– Defects are small, low contrast– Large gradients in image– Plate-to-plate variations– Image size (8MP!)– Must process in < 0.5 second
• Solution:– A custom pixel-by-pixel threshold
for each image.
14
Defect!Defect!
Step 1 : Unwrap ImageProblem : We just need rim pixels•Use image masks?
– Longer process times– Still have large images
Solution : IMAQ Unwrap• 8M pixel square -> 1.5M pixel strip
•Aligns gradients, rim transitions in one direction
15
Step 2 : Create Custom Golden MasterWhat should this image look like if it were perfect?
•Remove defects using median smoothing– Tip : Use X Size >> Y Size to preserve gradients & edges– Performance Boost : Reduce image resolution , Smooth, then
Resample back to original size.
16
Unwrapped OriginalMedian Smoothed“Golden Master”
DefectDefect Defect Removed!Defect Removed!
Step 3 : Pixel-by-pixel Threshold
17
“Golden Master”
Results
Subtract aconstant to seta “lower”threshold.
Original
Defect Found!
Threshold original image using IMAQ Compare
Benefits of Pixel-by-pixel Thresholds
• Works with:– Gradients, edges, speckle, large dynamic contrasts
• Every image checks against itself – Robust against image-image variations, changing
lighting conditions
• Fast (<100ms for 1.5MP)– Limited by smoothing performance
18
3D Imaging
• Simple inspection for geometric errors– Gouges, bulges, warp
19
Warp examples
Laser Triangulation
20
LaserLine
LaserLine Area
CameraArea
Camera
Original
Thresholdapplied
Convert pixels to height
21
With a little more geometry, we can also correct for perspective :
WD
LaserLine
LaserLine Area
CameraArea
Camera
H
dyHeight =
3D Laser Height Measurement
22
• Each camera frame = 1 cross section (per laser line)– Tip : Add multiple laser lines for more cross sections
• 250 um cross section resolution (100 FPS @ 1” per sec)– Tip : Reduce ROI for fastest frame rates
• 100 um height resolution
Completed system
• Fast development with NI tools: – Completed in < 3 months
• Reliable– Inspected over 1 million dishes
• Multiple follow-up systems
23
Conclusions• Invest in good prototyping practices
• Prepare for unknown hurdles– Use modular, flexible architectures in hardware, layout,
software
• Defect finding algorithm– Pixel-by-pixel thresholds
• Simple 3D laser scanner– Inexpensive, easy to setup
24
Questions?
25
26
NIWeek Vision SummitAugust 2–3, 2011 Austin, TexasKeynotes
Industry Keynote: How NI Technology Powers the Space Elevator LaserMotive Tuesday, August 21:00-2:00
Academic Keynote: Industry Trends and Intelligent Production Systems of the Future Interdisciplinary Imaging and Vision Institute, RWTH Aachen University
Wednesday, August 31:00-2:00
Tuesday, August 2
3D Vision and the Kinect National Instruments 10:30-11:30
Wacky Optical Tricks for Machine Vision GE Global Research Center 2:15-3:15
Panel Discussion: Latest in Camera Technologies
Cyth SystemsBasler Vision TechnologiesPleora TechnologiesToshiba TELI
3:30-4:30
Machine Vision and Industrial Robotics: From Design Concepts to Factory Floor Deployment ImagingLab 4:45-5:45
Wednesday, August 3
Precision Metrology National Instruments 10:30-11:30
Is LabVIEW FPGA Right for My Vision Application? National Instruments 2:15-2:45
Autofocus System for an Ellipsometer Nanometrics Inc. 2:45-3:15
Developing a Quality Inspection Method for Selective Laser Melting of Metals Using a High-Speed NIR Camera Katholieke Universiteit Leuven 3:30-4:00
Web Inspection of Optical and Medical Fibers Adsys Controls Inc. 4:00-4:30
Breaking New Ground with Vision Inspection Systems Coleman Technologies Inc. 4:45-5:15
Development of a Digitally Multiplexed Bioassay Reader with Magnetic Bead Technology MoviMED 4:15-4:45
27