Trends and Outlook for Sight and Sound in Simulation

Neil WitteringBusiness Development Manager – SimulationBarco

neil.wittering@barco.com

Recent MemoryA brief history of recent changes

2

▪ COTS evolution

▪ New standards for resolution

▪ Advances in illumination

Display Considerations

3

▪ The “good enough” debate continues◦ How to deliver performance

◦ How to meet the budget

▪ Image fidelity

◦ Contrast, color and brightness

◦Which light source

◦Which imaging device

▪ Benefits for operators over time

◦ Higher rates of availability

◦ Lower maintenance costs

Illumination TechnologyOptions and Choices

4

▪ All solid state?

▪ Rapid market switch to laser/phosphor

◦ All projector markets since 2016/17

◦ Brightness and longevity

▪ HLD – the next generation LED

◦ Color performance and longevity

◦ Higher brightness replacing “gen 2” LED

▪ Lamps switched off?

◦ Where cost and size matter lamps work

▪ RGB LASER?

Resolution

5

▪ 4K vs “4K-UHD” debate continues◦ Cost effective with pixel shift

▪ Many sites using WQXGA

◦ When to upgrade?

▪ What about 8K

◦ How to feed the beast

▪ Image Generator performance

◦ Advanced video processing

▪ IG Pixel Shift

◦ 4K@120Hz with todays hardware

Evolution doesn’t stop

6

▪ Display

◦ 8K displays driven by consumer applications

◦ Can IGs deliver 8K

◦ Do trainers need 8K

◦ At what cost

▪ Illumination

◦ Extend color gamut – rec2020?

• Enhanced realism in scenes

◦ HLD and hybrid designs

• color performance is great

• Demand for brighter output

◦ RGB laser to the rescue?

The Forgotten Element of Simulation Training:

Immersive Audio

7

Redefining the limits of reality

Introducing object-based, real-time spatial audio processing

9

▪ It positions sound correctly relative to the

trainee’s position and object’s behavior in 3D space (x,y,z).

▪ No sweet spot.

▪ Independent of speaker setup.

Object-based sound means:

The Dynamic Audio Environment

▪ A true separation from consumer-level audio systems

▪ A completely separate modelling environment from

Visual

◦ Modelled audio to reflect specific characteristics

◦ Audio modelled to be acoustically perfect

◦ Reverberation and doppler

◦ Positional and in-motion processing

▪ Can be fully correlated to the visual environment for

suspension of disbelief

10

Immersive audio

▪ Stereo vs Surround sound vs Immersive audio

◦ Stereo sound uses 2 loudspeakers to create an image appearing to come from between 2 loudspeakers.

◦ Surround sound let’s the listener hear sounds from a certain direction and sound can also come from behind. Usually as a channel based output solution (5.1/7.1 etc.) .

◦ Immersive audio require accurate reproduction of sound scene and localization to:

• Immerse the listener in the scene to suspend the feeling of disbelief.

• Correlate audio with image exactly (HMD and Dome projection applications)

• Give the listener (limited) freedom to move around.

▪ The problem with surround sound

◦ Traditional stereo or surround sound layouts depend on an exact predetermined positioning of the loudspeakers and the listener (defined sweet spot).

◦ The individual channels have the spatial information (ILD, ITD and possibly HRTF) embedded in the signal forcing predetermined loudspeaker arrangement

◦ Changing the positioning of the loudspeakers or moving out of the sweet spot breaks down the perception of immersiveness.

◦ Most simulation applications generate sound scenes in real-time for few predetermined loudspeaker arrangements. (often the OS offered implementation) However without processing the scene for the exact loudspeaker setup, still, correct localization cannot be achieved.

▪ Immersive audio

◦ Realistic behaviour of sound across the loudspeaker arrangement needed to transfer the listener to the virtual world and create a sense of “being there”.

◦ Moving around the space specifically in domes and HMD applications enhances immersive aspects.

◦ Object based sound or treating individual sounds in the scene as objects detaches positional information from the sound signal and introduces meta data that can be processed separately. Interfacing directly with the simulation.

◦ Vital for training sound related skills like shot detection and night operations as well as other low visibility environments and certain failure identification scenarios.

Spatial audio reproduction

11

Immersive Audio

12

▪ Auralize objects beyond the visual threshold.

Objects can be realistically cued even if they are out of

sight.

◦ Can be correlated with real-time visual, or time-synched with video

or live exercises.

▪ Immerse the trainee in the synthetic environment providing

a lifelike experience.

▪ Create disruptive audio environments to induce stress and

train decision making under pressure.

▪ Train for low visibility environments and nocturnal scenarios.

Training benefits of immersive audio

First Responder

13

14

Battlefield Care

15

Command &Control Centers

16

Multi-crew Environments

Barco AudioCue

Redefining the limits of reality

18

| youtube.com/BarcoTV

| linkedin.com/company/Barco

| twitter.com/Barco

| facebook.com/Barco

19