Click here to load reader
Upload
the-khronos-group-inc
View
256
Download
0
Embed Size (px)
DESCRIPTION
The Khronos Camera working group is creating an open, royalty-free standard for advanced, low-level control of mobile and embedded cameras and sensors.
Citation preview
© Copyright Khronos Group 2013 - Page 1
Camera Control API Provisional Working Group
May 2013
© Copyright Khronos Group 2013 - Page 2
Advanced Camera Control Use Cases • High-dynamic range (HDR) and computational flash photography
- High-speed burst with individual frame control over exposure and flash
• Rolling shutter elimination
- High-precision intra-frame synchronization between camera and motion sensor
• HDR Panorama, photo-spheres
- Continuous frame capture with constant exposure and white balance
• Subject isolation and depth detection
• High-speed burst with individual frame control over focus
• Time-of-flight or structured light depth camera processing
- Aligned stacking of data from multiple sensors
• Augmented Reality
- 60Hz, low-latency capture with motion sensor synchronization
- Multiple Region of Interest (ROI) capture
- Multiple sensors for scene scaling
- Detailed feedback on camera operation per frame
© Copyright Khronos Group 2013 - Page 3
Camera Control API Complements Acceleration
Pre-ISP
Processing (Bayer Space)
ISP
Post-ISP
Processing (YUV Space)
Application
Need Camera API to feed
processing pipeline for
advanced use cases
Image/Vision
Acceleration
APIs ?
Sensor Lens
Flash
© Copyright Khronos Group 2013 - Page 4
Precursor APIs for Camera Control Initiative • FCAM – Open source project
- Capture of stream of camera images with precision control - A pipeline that converts requests into image stream
- All parameters packed into the requests - no global state
- Programmer has full control over sensor settings for each frame in stream
- Control over focus and flash - No hidden daemon running
- Control ISP - Can access supplemental
statistics from ISP if available
• Android New Camera HAL (2013)
- Uses some of these concepts
© Copyright Khronos Group 2013 - Page 5
Potential Camera Control API Functionality • Burst Sensor Control
- Exposure, time, gain, CFA pattern …
• Burst Lens control
- Target focus distance, aperture, focal length, position state …
• Burst Flash Control
- Brightness, duration, burst, activity state …
• ISP Control
- Demosaic-ing quality, denoising quality, 3A, CCM, Gamma, color space …
• Output(s) Control
- Resolution, ROI extraction, quality, format (including Bayer, YUV) …
• Frame timestamp
- To be used to synch with motion sensors
• Parameter access
- Standard-defined parameters AND Vendor-specific extensions registry
• Feedback Statistics
- Histogram, sharpness map …
• Multi-sensor control
- Synchronization, master sensor controlling other sensors, image stacking
• Metadata
- Per frame: Focal length (fx, fy), principal point (cx, cy), skew (s), image resolution (h, w), exposure
- Per device: Cameras and sensors physical layout, calibration and lens distortion
© Copyright Khronos Group 2013 - Page 6
Camera Control API Usage
Burst control of
sensor, flash, lens
with precision
timestamps on frames
Application
Statistics
Feedback
Output
Scalers
and Formatters
Downstream
Processing ISP Image
Processing
MEMS
Sensors
Precision timestamps
on positional sensor
samples
Sensor Lens
Flash
EGLStream
and EGL access
to sample
timestamps
Camera Control API