Slide 1

Verification of specifications and aptitude for short-range applications of the Kinect v2 depth sensor Cecilia Chen, Cornell University Lewis Educational and Research Collaborative Internship Project (LERCIP) NASA Glenn Research Center, Graphics & Visualization/Dr. Herb Schilling 7/22/2014 Slide 2 Purpose of this study Validate the published specifications for the Microsoft Kinect v2 depth sensor Resolution and x-y position accuracy Depth-sensing accuracy Near-range sensing limit Determine the sensors potential for use in short-range applications Feasibility of repurposing the Kinect for functions requiring an operating distance of approximately 0.5 m from the sensor Slide 3 Overview of topics discussed I.Introduction A.Microsoft Kinect v2 B.Infrared and Depth streams C.Time-of-flight II.Preliminary Calculations A.Is the depth cameras error range acceptable? B.Possibility of errors in position due to low resolution C.Possibility of errors in angle due to noise Slide 4 Overview of topics discussed III.Calibration and Experimental Verification A.Software preparation B.Calibration C.Experimental verification IV.Conclusions A.Findings B.Sponsors Slide 5 Introduction Slide 6 Microsoft Kinect v2 Slide 7 Infrared and Depth streams Slide 8 Time-of-flight Slide 9 Preliminary Calculations Slide 10 Is the depth cameras error range acceptable? Microsoft claims depth measurements are accurate to within 1 mm Guidelines for short-range use of the depth sensor Defined by an operating distance (between the Kinect and object) of approximately 0.5 m Near-range of the Kinect v2 supposedly starts at 0.5 m Given a surface oriented perpendicular to the depth axis: Position be able to locate a point on the surface to within 2 mm Angle be able to measure inclinations on the surface to within 10 Both to be confirmed through calculations using an upper error bound Slide 11 Possibility of errors in position due to low resolution y z x Slide 12 Slide 13 35 x/2 0.7 m x/2 35 Horizontal slice of FOV Slide 14 Possibility of errors in position due to low resolution y/2 0.7 m y/2 30 Vertical slice of FOV 30 Slide 15 Possibility of errors in position due to low resolution Slide 16 50 mm diameter Slide 17 Possibility of errors in angle due to noise For simplicity, take a plane defined by two points 50 mm apart representing the same circular surface: 50 mm diameter x y z Axis orientation Slide 18 Possibility of errors in angle due to noise z x y 50 mm Kinect zz Slide 19 Possibility of errors in angle due to noise zz 50 mm Slide 20 Calibration and Experimental Verification Slide 21 Software preparation Use C# to interact with Microsoft Visual Studio and the Kinect for Windows SDK Modify and expand the Depth Basics code included in the SDK (Preview 1404) Decreasing the grayscale gradient range for easier visual distinguishability between depths Making the window recognize a hovering cursor Writing the (x, y, z) coordinates of the cursor to the image window in real-time Averaging depth readings at a given point to smooth out jumpy data Capturing depth arrays of multiple frames Outputting collected data to CSV files for further analysis Slide 22 BeforeAfter Slide 23 Calibration Step 0: Design and assemble a rig to hold the Kinect steady Step 1: Level the sensor Step 2: Set up a base surface for measurements at a height similar to the intended distance between the Kinect and interaction area Slide 24 Calibration Step 3: Check for irregularities Slide 25 Experimental verification Step 4: Measure a sloped calibration block of known height 50 mm Slide 26 Base, z 1 Top of block, z 2 Height, z Actual (mm)-- 50 Measured (mm)5710 (538 minimum)Error Slide 27 Experimental verification Step 5: Lower the base height Step 6: Measure a sloped calibration block of known height 50 mm Slide 28 Base, z 1 Top of block, z 2 Height, z Actual (mm)-- 50 Measured (mm)71266250 Slide 29 Experimental verification Step 7: Repeat with calibration block(s) of different heights 64 mm Slide 30 Base, z 1 Top of block, z 2 Height, z Actual (mm)-- 64 Measured (mm)71264765 Slide 31 Conclusions Slide 32 Findings Successful verification of the published specs Kinect v2 appears to be a promising depth sensor for short-range applications Expectation or Guideline Experimental confirmation Acceptable? Position (resolution)2 mm(1.9 mm) 2 per pixel resolution Yes Angle (noise) 10Requires z 8.8 mm at z = 0.7 m Very Depth accuracy1 mm Very Near-range0.5 m0.538 mNo Slide 33