View
220
Download
0
Category
Preview:
Citation preview
7/28/2019 Jryde Robotics Pr
1/24
PR in
Robotics1/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Pattern Recognition in Mobile Robotics
CSE 455/555 Introduction to Pattern Recognition
Dr. Julian Ryde
SUNY at Buffalo
January 21, 2011
7/28/2019 Jryde Robotics Pr
2/24
PR in
Robotics2/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Abstract/Lecture Overview
A more realistic set of motivating application examplesbased in Mobile Robotics
Motivations for mobile roboticsChallenges of mobile roboticsExamples involving pattern recognitionNot going to talk about Industrial/static robotics
Even thought there are many applications of patternrecognition in this area of robotics
7/28/2019 Jryde Robotics Pr
3/24
PR in
Robotics3/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
What is Robotics?
DenitionsIntelligent connection of perception to action
Perception - Acquisition and interpretation of sensordataIntelligence?
Decision makingLearning
Actuation - Effecting changes in the physical world
LocomotionManipulationRoles of pattern recognition in robotics
Pattern recognition also has a role to play in intelligencePattern recognition forms part of the interpretation
7/28/2019 Jryde Robotics Pr
4/24
PR in
Robotics4/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Robotics Areas
StaticIndustrial roboticsControlled environmentsMore commercially successful
Mobile
Indoor RoboticsPersonal roboticsOfce robots
Remote telepresenceDomestic robots
Roomba - most successful consumer mobile robot?
Field RoboticsMiningMilitary
Health careHome care for the elderly (Especially Japan)
Wherever it is Dirty, Dull or Dangerous
7/28/2019 Jryde Robotics Pr
5/24
PR in
Robotics5/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Willow Garage
Long term funded company that does not need to makea prot
Adheres to the open source development modelRobot Operating System (ROS)OpenCV
Computer VisionMachine Learning algorithms
PR2 RobotPR2 Beta Overview Video
http://www.youtube.com/?v=UyLq4lfBsI0http://www.youtube.com/?v=UyLq4lfBsI07/28/2019 Jryde Robotics Pr
6/24
PR in
Robotics6/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Sensors
Internal sensorsGlobal position system - GPSInertial Measurement Unit - IMU
Vision basedStandard Cameras
OmnidirectionalRange sensors
Time of ightTriangulation based
Structured light
Stereo visionUltrasonic
KinectStructured light 3D depth sensorReturns RGBD - Red, Green, Blue and DepthInexpensive at 150$
7/28/2019 Jryde Robotics Pr
7/24
PR in
Robotics7/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Pattern Recognition in 3D Data versus Pictures
More difcult for humansMore suitable for computers?
3D data is more independent of observer positionMany objects are rigid or articulated so 3D shape is
important and invariantFor features invariant is goodimplies reliable/repeatable extraction
Camera images susceptible toillumination changesloss of information in conversion from 3D to 2D
7/28/2019 Jryde Robotics Pr
8/24
PR in
Robotics8/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Considerations specic to mobile robotics
AutonomyThere can be no guarantee of continuous connection.Therefore a certain level of autonomy is required forreliable continuous operation.
Power consumptionRestricted processing power
Especially for indoor and smaller robots that run onbatteries
Real-time constraintsPedestrian recognition has to be fast and reliable forautomotive applicationsBatch algorithms not really suitable
For long term operation robot ultimately has to processdata as fast as it receives it.
7/28/2019 Jryde Robotics Pr
9/24
PR in
Robotics9/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Localisation and Mapping
LocalisationFor most applications need to have an awareness oflocationAugment the robots workspace with features
Possible but expensive for factories and other controlledenvironments
MappingLearning a model of the operatingworkspace/environmentAids localisationAids path planning
Simultaneous Localisation and Mapping (SLAM)The process of both inferring location from a map whilstsimultaneously updating the map with new observedinformation.
7/28/2019 Jryde Robotics Pr
10/24
PR in
Robotics10/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Simultaneous Localisation and Mapping(SLAM)
SLAM is vital for indoor robotsInterestingly not used for the iRobot roomba
Outside not as necessary where GPS coverage isavailableVarious avours of SLAM
MetricTopological
Matching2D/3D scan matchingImage
7/28/2019 Jryde Robotics Pr
11/24
PR in
Robotics11/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Loop Closing
Closing of an open loop in a large (80m by 25m) cyclicenvironment from Konolige and Gutmann (1999)Improving map accuracy through loop closing
Have I been here before?Recognising previously visited locations
7/28/2019 Jryde Robotics Pr
12/24
PR in
Robotics12/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Scan matching
Two scans taken from different posesFind features present in each scan
Determine the correspondence between featuresUse this information to align the scansThis gives the pose change between scansMuch research done with 2D range scans
Example with 3D scans or an indoor environmentScan matching Ilustration
http://jcr/csiro/introseminar/presentation/msmpeg4v2/scanmatching.avihttp://jcr/csiro/introseminar/presentation/msmpeg4v2/scanmatching.avi7/28/2019 Jryde Robotics Pr
13/24
PR in
Robotics13/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
3D mapping indoors
3D map y around video
http://jcr/csiro/introseminar/presentation/msmpeg4v2/flyaround-lowres.avihttp://jcr/csiro/introseminar/presentation/msmpeg4v2/flyaround-lowres.avi7/28/2019 Jryde Robotics Pr
14/24
PR in
Robotics14/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Probabilistic robotics
Return not only an answer but some idea of theprobability distribution
For example the pose distribution can be multimodalMany ways of representing these probablity densityfunctionsThus you have some idea of how likely the result is tobe correct
Algorithms in robotics need to be robustRobust statistics e.g. median
contrast with sufcient statisticsInsensitive to outliers
Sufcient statistics e.g. meanMathematically analyticComputational convenientSensitive to outliers
7/28/2019 Jryde Robotics Pr
15/24
PR in
Robotics15/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
DARPA Grand and Urban Challenges
Grand Challenge$2 million Prize awarded to Stanford Racing Team132 mile desert course in just under 7 hours
Urban challengeFollow on from the easier grand challenge
Autonomous vehicle drive through an urbanenvironmentDrive in trafcManeuvers merging, passing, parking, negotiatingintersections
Featured the rst autonomous car crashReliable perception and interpretation vitalFinal highlights
DARPA Urban challenge highlights videoKey Enablers
Velodyne 3D laser scannerNear to far learning
http://www.darpa.mil/grandchallenge/video/DARPA_highlight_preview3_low.wmvhttp://www.darpa.mil/grandchallenge/video/DARPA_highlight_preview3_low.wmv7/28/2019 Jryde Robotics Pr
16/24
PR in
Robotics16/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Learning for mobile autonomous systems
Pattern recognition enables systems to learn andgeneralise from new data.Interaction with humans provides plenty of opportunityfor supervised learning.Amazons mechanical turk
Humans working for the robots!DARPA urban challenge videos
near to far learning in images e.g. using laser data andcorresponding vision results to train and so classify far
image pixels.Self supervised learning
Training data might arrives at a later time for somecasesTraining data provided by others sensors
e.g. Near to far learning
7/28/2019 Jryde Robotics Pr
17/24
PR in
Robotics17/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Near to Far Learning
Image and research from Raia HadsellOnline machine learning
Either laser or stereo provides reliable near range 5mclassication (driveable or not based on smoothness)Then used as training data for appearance based
classier for pixels beyond 5m
7/28/2019 Jryde Robotics Pr
18/24
PR in
Robotics18/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Solutions in Perception Challenge
GoalRigid objects recognized and their 6DOF posedetermined with the Kinect sensor
Kinect sensorBest solutions likely to combine conventional imageinformation with depth informationIdeal opportunity to put into practice what you learn inthis course
7/28/2019 Jryde Robotics Pr
19/24
PR inRobotics
19/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Field robotics
Work undertaken at the Australian CommonwealthScientic and Industrial Research Organisation(CSIRO)Autonomous earth moving
http://home/jryde/demo/bobcatdemo150210.avihttp://home/jryde/demo/bobcatdemo150210.avi7/28/2019 Jryde Robotics Pr
20/24
PR inRobotics
20/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Autonomous Skid Steer loader (Bobcat)
Earth moving taskMove soil material from one location to another
Ultimate goal is the design the terrain you want in aCAD package and have it replicated in the physicalworld by autonomous excavation and earth movingmachines
7/28/2019 Jryde Robotics Pr
21/24
PR inRobotics
21/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Reliable Human/Novelty Detection
Learn the operational environmentHighlight those areas that have changedEssential for safety reasons
If unknown objects detected or the environment hasbeen tampered with need to potentially stop operation
Handle noise, how much variation constitutes change?
Some active research questions related to
7/28/2019 Jryde Robotics Pr
22/24
PR inRobotics
22/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Some active research questions related toPattern Recognition
SLAMMonocular camera based visual SLAMSLAM with RGBD sensors
Correspondence problem
Finding corresponding points in multiple imagesFor stereo vision
Object recognition in 3D data3D shape based retrieve and lookupView and occlusion invariant hashingLocality sensitive hashing
Object recognition from a moving camera sensorDifferent to object recognition in random imagesBackground subtraction for moving cameras
7/28/2019 Jryde Robotics Pr
23/24
PR inRobotics
23/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Recap
What is (mobile) robotics?Example applications involving pattern recognition
LocalisationMapping
Scan matchingLoop closing
Terrain classication for path planningNear to far learningSelf supervised machine learning
Field roboticsAutonomous Earth moving
Current work in the research communitySolutions in Perception Challenge
7/28/2019 Jryde Robotics Pr
24/24
PR inRobotics
24/24
Dr. JulianRyde
Introduction
Challenges
ExampleApplications
Summary
Questions for consideration?
What is holding back mobile robotics?Was the lack of low cost depth sensors, but now withthe Kinect?Low cost manipulator platforms? The PR2 is $400,000
Motivated by the recent debut of the Kinect how canexisting computer vision pattern recognition algorithmsbe applied to RGBD data?
Recommended