Upload
others
View
28
Download
0
Embed Size (px)
Citation preview
Hyun Myung, Ph.D.Professor, School of Electrical Engineering
Head, KAIST Robotics [email protected]
http://urobot.kaist.ac.kr
Urban Robotics Lab“IT & Robotics for smart cities”
6-DoF SHM robot system
Vision / LiDAR / Magnetic field / RF-based 3D SLAM
JEROS (Jellyfish Elimination RObotic Swarm)
Low-cost self-driving
Oil spill protection robot
CAROS (Climbing Aerial Robot System)
Green algae removal robot
UAV
Green Algae
FAROS (Fireproof Aerial Robot System)
Robot Navigation
Structural Health Monitoring
Environmental Robotics
Autonomous valet parking Mole-bot
AI & Robot IntelligenceAutonomous robot navigation Vision, LiDAR, Magnetic field-based SLAM (Simultaneous Localization And Mapping) Indoor, Outdoor (Ground, Underwater, Underground, Aerial) SLAM
Object recognition UAV image-based jellyfish recognition, green algae recognition, underwater jellyfish recognition Object recognition using low-resolution aerial images
Human behavior recognition for HRI (Human Robot Interaction) Human pose estimation Sequence-based human behavior recognition
Developed robot system in our lab JEROS (Jellyfish removal robot), ARROS (Green algae removal robot) CAROS, FAROS (Wall climbing drones)
2
Autonomous Robot Navigation
Mapping•Feature map•Grid map
Localization•Localization•Relocation
Motion Control•Path Planning•Reactive Control
ExplorationActive Localization
SLAMIntegrated Approach•Active SLAM
(Source: Makarenko et al. ’02)
Core technologies of autonomous navigation
Graph-based sensor fusion for dynamic and feature-poor environment False constraints removal in dynamic environment (low + high dynamic) SLAM performance improvement using low-cost sensor fusion
• ex1) 2D LiDAR + 2D camera = 3D matching for robustness• ex2) Magnetic feature + 2D LiDAR = minimize planar localization error
KAIST’s technology
5
GP-SLAM for low-dynamic environmentGrouping nodes
Pruning false constraints
Normalized covariance values of the odometry edges. Result obtained after grouping the nodes
Error metric (average Mahalanobis distances of the edges) based on
the grouped nodes.Optimized trajectory after pruning false constraints
Edges btw. groups 2 & 6 can be pruned
6
GP-SLAM for low-dynamic environmentVisual odometry
Conventional SLAM
GP SLAM
Donghwa Lee, Hyun Myung, "Solution to the SLAM Problem in Low Dynamic Environments Using a Pose Graph and an RGB-D Sensor," Sensors, vol.14, no.7, pp.12467-12496, [Link], July 2014.
7
SLAM in dynamic environment (LiDAR sensor)Real-time recognition of high dynamic (human) and low dynamic objectsGraph-based SLAM using 3D feature matching & LRF scan matchingTested on KAIST Geocentrifuge center building (10 × 10m) with 2 low dynamic and 5 high dynamic objects
8
3D SLAM for autonomous driving
Stereo camera GPS
3D point cloud map
Odometry
Robust localization for autonomous driving
Hyungjin Kim, Bingbing Liu, Chi Yuan Goh, Serin Lee, Hyun Myung, "Robust Vehicle Localization using Entropy-weighted Particle Filter-based Data Fusion of Vertical and Road Intensity Information for a Large Scale Urban Area," IEEE RA-L (Robotics and Automation Letters), vol.2, no.3, pp.1518-1524, [DOI], July 2017.
Velodyne, GPS, IMU, and wheel odometry
Mean error ≤ 0.1 m10
3D SLAM using a tilted 2D LiDAR sensor
3D localization and mapping (SLAM) using a tilted 2D LiDAR sensorImproving accuracy through hierarchical graph optimization
Before global optimization
Global optimization result
Super node generation
Super node optimization
Sensor system
Hierarchical graph optimization
Robot motion
Tilted LiDAR
Local map
graph
Local map
graph
Local map
graph
Local map
graph
Local map
graph
Local map
graph
Global map graph
Local point cloud maps
3D SLAM using a tilted 2D LiDAR sensor
3D SLAM using a tilted 2D LiDAR sensor
Earth’s magnetic field-based SLAM
Motivation Feature-poor (Vision/LiDAR)
environment Anomaly of earth’s magnetic field
(20~80 μT) is used as features
<3-axis magnetic field distribution>
14
Magnetic sequence-based graph SLAM
Jongdae Jung, Taekjun Oh, and Hyun Myung, "Magnetic field constraints and sequence-based matching for indoor pose graph SLAM," Robotics and Autonomous Systems, vol.70, pp.92-105, [DOI], Aug. 2015
Information matrix Num of constraints
Robot localization
- Cyan: reference (SICK Gmapping)- Magenta: wheel odometry- Blue: optimized pose graph
Localization (KI building) Magnetic field map
15
Mole-bot
Directional drilling
Directional drilling for shale gas extraction requires: Underground localization RSS (Rotary Steerable System) mechanism Control algorithm
17
source: Katie Mazerov , “New directional drilling system combines improved mud motor, MWD technologies”, Drilling Contractor (2011), Hallibuton Hompage
Mole-bot
• Embedded directional drilling
• Underground localization
Underground comm. & monitoring system
As-is To-be
18
Mole-bot
Digging mechanism
Source: NY Times (https://youtu.be/toARdZKs-IE)
19Scapula as a lever
20
Mole-bot
Front part Rear part
21
Mole-bot
x15 x10
Forward drilling Directional drilling
Underground localization using Graph SLAM
The magnetic sensor can measure similar features againDesigned for loop closing at both direction movement
Forward movement Backward movement
Concurrent normalized cross-correlation
22
Underground localization using Graph SLAM
Results Comparison of RMSE
OrientationPosition
Dead reckoning 77.8cm
12.5cm
10.2º
Proposed algorithm [1] 2.4º84.0%↓ 76.5%↓
[1] Byeolteo Park and Hyun Myung, "Resilient underground localization using magnetic field anomalies for drilling environment," IEEE Trans. Industrial Electronics, [DOI], Feb. 2018.
23
Structural Health Monitoring
SLAM in aerial environment
Bridge inspection using UAVs
25
UAV SLAM Framework
GPS + Camera + IMU + 3D LiDAR
Camera + IMU
GPS
3D LiDAR
Visual Inertial
OdometryNode
generationNormal
Distribution Transform
Graph optimization
Poseestimation
pose
Pose (only if it is available)
Generalized ICP
with voxels
Constraint
node
pose
node
3D point cloud (2.6s/frame)
Voxel of 3D point cloud (0.7s/frame)
NDT3D point cloud
UAV SLAMDownward camera
Upward camera
CAROS: Wall-climbing drone (robust to wind)CAROS (BBC News, May 2015)
Tilt-rotor-based low-impact perching
Significant decrease of impact in perching
5o/s
87o/s
Applications of a wall-climbing droneFAROS (MBC News, Apr. 2016)
Environmental Robotics
JEROS (Jellyfish Elimination RObotic Swarm)
USV(unmanned surface vehicle) + Jellyfish shredding device Autonomous navigation Formation control: leader-follower Vision-based jellyfish detection &
distribution recognition Jellyfish removal performance: 216
kg/hour
Damage by jellyfish in South Korea: over 300M USD/year (fishery: 230M, power plants: 70M)
PROBLEM
DEVELOPMENT
JEROS (BBC News, 2015)
Formation control test
Jellyfish distribution recognitionJellyfish images using drones
Jellyfish distribution recognition
Deep learning GPS/IMU aided image-based localization
Jellyfish images
Localized Jellyfish distribution
JEROS formation control
Jellyfish removal
Coverage path planning ASV launching and docking
Jellyfish distribution recognition using DNN
Proposed LeNet-5 AlexNet GoogLeNetAccuracy [%] 94.06 95.11 95.60 98.37
Average frame rate[Hz] 8.23 4.89 0.01 0.02
Proposed network
Hanguen Kim, Jungmo Koo, Donghoon Kim, Sungwook Jung, Jae-Uk Shin, Serin Lee, and Hyun Myung, "Image-Based Monitoring of Jellyfish using Deep Learning Architecture," IEEE Sensors, vol.16, no.8, pp.2215-2216, Apr. 2016.
ARROS (Algal bloom Removal RObot System)ECF (electrocoagulation and floatation) reactor
Sungwook Jung, Hoon Cho, Donghoon Kim, Kyukwang Kim, Jong-In Han, and Hyun Myung, "Development of Algal Bloom Removal System Using Unmanned Aerial Vehicle and Surface Vehicle," IEEE Access, Dec. 2017.
[email protected] http://urobot.kaist.ac.kr