Embed Size (px)
Citation preview
Presented by: For:
© ETSI 2019
15-16 April 2019
ETSI Industry Specification Group Augmented Reality Framework (ISG ARF):
activities overview and outlook
Ludovic Noblet Immersive Media meets 5G Workshop
2© ETSI 2019
The context
AR is a growing and promising market with high revenue forecast e.g. 120$B by 2021
Clear and long-term RoI benefits based on improvements and efficiency gains
Acquisition of AR solutions by large companies
Risk of control of AR technologies by only a few large actors -> vertical siloes
A set of common key components: Tracking, localization, pose estimation, 3D reconstruction, cloud storage and more
Lack of standards adoption:
Adoption of standards bring interoperability
Creation of an Industry Specification Group at ETSI - a European standards organisation with a global impact
3© ETSI 2019
Overall Objective of the ISG ARF
Defining a framework for the interoperability of AR applications and services, in order to prevent market fragmentation and enable providers to offer part of an overall AR solution
To encourage an ecosystem with a diverse range of solution providers including smaller players, new entrants and academics
To define a modular architecture and achieve interoperability at the interface of building blocks
The focus is on interfaces, no standardisation of core technologies
The building blocks can be open source, proprietary or standardised technologies
Achieving Interoperability:
Will give end-users confidence to invest in and use (future proof) AR solutions
Can prevent lock-in situation with a single vendor
Can ease technology swapping, e.g. to improve applications performance and services
4© ETSI 2019
ARF Members and Participants
BasinLogic
5© ETSI 2019
Definition of AR
Augmented Reality (AR) is the ability to mix in real-time spatially-registered digital content with the real world.
Combines real and virtual
Interactive in real-time
Registered in 3D
6© ETSI 2019
What does an AR system require?
Mix digital content with the real world : Rendering interfaces
VisionVideo see-through displays (smartphones, tablets, etc.)
Optical see-through displays (AR glasses, AR windshield, etc.)
Projection
Spatialized Audio
Haptics
Spatially registered : Localisation Sensors & Computer Vision / AI algorithms
Vision (RGB cameras, Depths sensors, etc.)
Exteroceptive (GPS, Wifi, Lifi, cellular, etc.)
Proprioceptive (IMU, odometer, etc.)
7© ETSI 2019
What does an AR system require?
In real-time : Powerful processing units
CPU, GPU, Vision Processing Unit, Tensor Unit
Interactive content : Interaction devices
Tactile
3D gestures
Vocal
Eye-tracking
Cognition and Emotion analysis (Brain Computer Interfaces, etc.)
Haptics
Digital Content : 3D contents, scenarios, and a rendering engine
3D rendering engine (Unity, Unreal Engine, etc.)
3D Content (export from PLM, BIM, designed by computer graphics designers, scripts, etc.)
8© ETSI 2019
AR Framework architecture - Draft
Vision Engine
Localization, Mapping, Scene
analysis, etc.
Planes, occlusion meshes,lights,etc.
Central ProcessingUnit
Graphics Processing Unit
Vision ProcessingUnit
Processing Units
Hard
ware Tracking Sensors
Vision
Proprioceptive
Exteroceptive
Interaction Interfaces
Gesture
Vocal
Eye Tracking
Emotion
World
knowledge
Interactive
Contents
3D Rendering Engine
Rendering Interface
Pose Estimation
Interactive
Scenario
Non interactive
contents
Possibly distant
Distant with difficulty
(very low latency)
Local
Cloudification
So
ftw
are
Content Simplification,Interaction, Fusion, etc.
Data
Video see-throughAR
Optical see-through AR
Projected AR
Audio AR
© ETSI 2019
9© ETSI 2019
Proposed functional architecture
© ETSI 2019
10© ETSI 2019
ISG ARF Workplan
Work Item 4 InteroperabiltyRequirements
Needs for standards
Work item 1AR standards
Landscape
Work Item 3 AR Framework
Architecture
Use Case Survey
Work item 2AR industrialUse cases
Define new ETSI Specifications
Recommendexistingstandards
Definerequirements to extend existingstandards
11© ETSI 2019
AR standards landscape
A group report identifying the role of existing standards relevant to augmented reality
various standards setting organisations and fora reviewed: ISO/IEC, W3C, Khronos, IEEE, IETF, OGC, …
Rapporteur : Marius Preda, Institut Mines Telecom
Standards under review are:
directly addressing AR as a whole,
or addressing key technological components that can be useful to increase interoperability of AR components and services.
Report expected to be published before the summer
12© ETSI 2019
Industrial use cases for AR applications and services
A group report describing typical industrial use cases for AR applications and services
Rapporteur: Ralf Schäfer, Fraunhofer HHI
A questionnaire was developed to collect information about typical AR use cases and the conditions under which AR is used.
The survey, available in English, German and French, was online between 28/02/2018 and 01/05/2018.
13© ETSI 2019
Structure of the Questionnaire
Usage area
Expectations of users on AR
Level of experience with AR
Use Case analysis
Benefits of AR
Usage conditions
Level of data protection
Sources of augmentation data
Technical usage conditions
Operating conditions
Feedback options
Level of experience in AR usage
Experienced bottlenecks & limitations in AR usage
General perception of the development of AR during the last 2 years
14© ETSI 2019
Participants profileWithout academics
55% of the participants define themselves as Technology PROVIDERS over Technology USERS
AR technology user38%
AR technology provider
55%
None7%
AR technology user AR technology provider None
15© ETSI 2019
AR use-cases Profiles
0%
20%
40%
60%
80%
100%
120%
inspection / quality Maintenance Training Manufacturing
Tech Provider Tech User
Without academics
16© ETSI 2019
AR use-cases - Accuracy
Do the augmentations need to be precisely located relative to a real equipment or object ?
14%
40%
88%
43%
100%
60%
43%
13%
inspection / quality Maintenance Training Manufacturing
under 1 millimeter a few millimeter a few centimeter up to 10 centimeter
Without academics
17© ETSI 2019
1,38%
2,07%
3,45%
4,14%
4,14%
6,90%
6,90%
8,97%
8,97%
8,97%
9,66%
10,34%
11,72%
12,41%
Other (please specify)
Equipment cost
User security while using AR solution
AR Devices robustness
Incompatibility with protective equipments
Battery lifetime of the AR system
User interaction methods with the AR solution
Tracking robustness issues
Availability or adaptation of the data (3D models, 2D…
User acceptance (perceived reliability, utility, usability,…
Authoring time/cost of the AR application
Methods for initial spatial positioning of augmentations
Ergonomics of the AR display solution (weight, comfort ...)
Tracking accuracy issues
Mentioned challenges of ARWithout academics
18© ETSI 2019
1,38%
2,07%
3,45%
4,14%
4,14%
6,90%
6,90%
8,97%
8,97%
8,97%
9,66%
10,34%
11,72%
12,41%
Other (please specify)
Equipment cost
User security while using AR solution
AR Devices robustness
Incompatibility with protective equipments
Battery lifetime of the AR system
User interaction methods with the AR solution
Tracking robustness issues
Availability or adaptation of the data (3D models, 2D…
User acceptance (perceived reliability, utility, usability,…
Authoring time/cost of the AR application
Methods for initial spatial positioning of augmentations
Ergonomics of the AR display solution (weight, comfort ...)
Tracking accuracy issues
Tracking30%
Mentioned challenges of ARWithout academics
19© ETSI 2019
1,38%
2,07%
3,45%
4,14%
4,14%
6,90%
6,90%
8,97%
8,97%
8,97%
9,66%
10,34%
11,72%
12,41%
Other (please specify)
Equipment cost
User security while using AR solution
AR Devices robustness
Incompatibility with protective equipments
Battery lifetime of the AR system
User interaction methods with the AR solution
Tracking robustness issues
Availability or adaptation of the data (3D models, 2D…
User acceptance (perceived reliability, utility, usability,…
Authoring time/cost of the AR application
Methods for initial spatial positioning of augmentations
Ergonomics of the AR display solution (weight, comfort ...)
Tracking accuracy issues
Ergonomy27%
Mentioned challenges of ARWithout academics
20© ETSI 2019
1,38%
2,07%
3,45%
4,14%
4,14%
6,90%
6,90%
8,97%
8,97%
8,97%
9,66%
10,34%
11,72%
12,41%
Other (please specify)
Equipment cost
User security while using AR solution
AR Devices robustness
Incompatibility with protective equipments
Battery lifetime of the AR system
User interaction methods with the AR solution
Tracking robustness issues
Availability or adaptation of the data (3D models, 2D…
User acceptance (perceived reliability, utility, usability,…
Authoring time/cost of the AR application
Methods for initial spatial positioning of augmentations
Ergonomics of the AR display solution (weight, comfort ...)
Tracking accuracy issues
Cost : 11%
Mentioned challenges of ARWithout academics
21© ETSI 2019
1,38%
2,07%
3,45%
4,14%
4,14%
6,90%
6,90%
8,97%
8,97%
8,97%
9,66%
10,34%
11,72%
12,41%
Other (please specify)
Equipment cost
User security while using AR solution
AR Devices robustness
Incompatibility with protective equipments
Battery lifetime of the AR system
User interaction methods with the AR solution
Tracking robustness issues
Availability or adaptation of the data (3D models, 2D…
User acceptance (perceived reliability, utility, usability,…
Authoring time/cost of the AR application
Methods for initial spatial positioning of augmentations
Ergonomics of the AR display solution (weight, comfort ...)
Tracking accuracy issues
Mentioned challenges of ARWithout academics
Tracking 30%
Ergonomy 27%
Cost : 11%
Data availability: 9%
22© ETSI 2019
Interoperability Requirements for AR
A group specification specifying requirements for the interoperability of AR components, systems and services
Will build on industrial use cases identified in previous group report
Rapporteur: Eric Villain, Orange
Work at an early stage
23© ETSI 2019
Benefits to the ecosystem and outlook
A reference interoperability framework for AR components and services taking into account existing AR models and compliant with relevant AR specifications
Availability of high performance AR components portable between different hardware vendors, different providers of software solutionsand platforms
AR services and platforms simpler to design, deploy and operate
Incremental migration of AR components to the edge starting with storage in the cloud all the way to remote rendering ofcomplex scenes and other functions e.g. relocalisationwhere latency is critical.
24© ETSI 2019
Why are we doing that ?
No solution fits all requirements !
Custom development is essential to meet the needs of industry 4.0 (and others)
Being able to play LegoTM with AR components is crucial for AR deployment and adoption.
Prevent lock-in situation with a single vendor
Swap easily technologies, e.g. improve applications performance and services
Improve confidence to invest in and use AR solutions
encourage an ecosystem with a diverse range of solution providers including smaller players, new entrants and academics
If you share this vision, join the ISG ARF !
25© ETSI 2019
Do get involved!
ToR : Terms of Reference
If your company is an ETSI member, you can join the ISG ARF after signing the ARF member agreement; no additional cost
If your company is not an ETSI member, you can participate after signing the ARF participant agreement
Participation to online conference calls and subscription to mailing list are free
A fee of 200€ per day per physical meeting (F2F/online participation) per delegate; (capped to 700€)
26© ETSI 2019
ISG ARF Management Team
Chair: Ms. Muriel Deschanel, [email protected]
Vice-chair: Dr. Ralf Schäfer, [email protected]
Secretary: Nicole Leminous, [email protected]
Technical Officer: Sylwia Korycinska, [email protected]
Rapporteurs for:
AR standards landscape report: Marius Preda, [email protected]
Industrial use cases for AR applications and services report: Dr. Ralf Schäfer, [email protected]
AR framework architecture specification: Jérôme Royan, [email protected]
AR interoperability requirements specification: Eric Villain, [email protected]
27© ETSI 2019
THANK YOU!
