ISMAR 2012: Tailoring the Adaptive Augmented Reality (A²R) Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

Tailoring the Adaptive Augmented Reality (A²R) Museum Visit:

Identifying Cultural Heritage Professionals Motivations and Needs

Dr Areti DAMALA, Dr Nenad STOJANOVIC

Centre d’Etude et de Recherche en Informatique et Communications (CEDRIC) Conservatoire National

des Arts et Métiers (CNAM), Paris, France &

Forschungszentrum Informatik Karlsruhe (FZI), Karlsruhe, Germany

The 11th IEEE International Symposium on Mixed and Augmented Reality

Museums and Theater Session

Atlanta, Georgia, USA

Wednesday, November 7th 2012

Dr Areti Damala, Dr Nenad Stojanovic

Tailoring the A²R Museum Visit: Identifying Cultural Heritage Professionals Motivations and Needs

How can we improve the AR museum visiting experience ?



Plan of the presentation

� The ARtSENSE project

� Adaptive Augmented Reality: Main components

� Theoretical background

� System Architecture

� Design challenges within a Cultural Heritage context

� A multi-layered and participatory design approach

� Identifying main motivations and needs

� Conclusions and directions for future work



� Augmented Reality Supported adaptive and

personalized Experience in a museum based on

processing real-time Sensor Events

� European Union co-funded project

� 36 months duration

� www.artsense.eu

� See, hear, feel, experience!



Consortium Partners



Cultural Heritage Partners

� An Art and Design museum

The National Museum of Decorative Arts, Madrid, Spain (MNAD)

� A History of Science and Technology museum

The National Museum of the History of Science and Technology, Paris, France (MAM)

� A Contemporary Gallery and New Media Arts Center

Foundation for Art and Creative Technology, Liverpool, UK (FACT)



From AR to A²R: The ARtSENSE approach

Audio BioVisual

1. Visual Monitoring and Augmentation

2. Acoustic Monitoring and Augmentation

3. Affective Monitoring and Augmentation



From AR to A²R: A Theoretical Framework

“Human users need to be treated as centers, immersed in AR interfaces, rather than as viewers, seeing interfaces or graphical

overlays”Guidelines

1. Center the human participant

2. Consider the idea of sound as much as the idea of sight

3. Acknowledge that human participants move, think and existsimultaneously

[Pedersen, 2009]



From AR to A²RAdaptive Augmented Reality =

Visual + Acoustic + Affective Monitoring andAugmentation

Vision bears the heaviest responsibility among allsenses. In a CH context: Linking the signifier withthe signified

Fine Arts Museum, Rennes, France Street Museum, Museum of London, UK

Stedelijk AR(t): Jan Rothuizen



From AR to A²R

Adaptive Augmented Reality = Visual + Acoustic + Affective

Monitoring and Augmentation

Musée des arts et métiers, Paris, France [Kaghat et al., 2011]



From AR to A²R

Adaptive Augmented Reality = Visual + Acoustic + Affective

Monitoring and Augmentation

Cognitive Engagement

Attention

Knowledge Acquisition

Emotion

Activation/Stimulation

Positive Affect

Negative Affect



ARtSENSE: System Architecture Overview



Design Challenges

For the CH professionals

� Understand the potential of the technology� Difficult to express needs as AR scenarios

� Understand how it works

� Apply the AR potential in the design phase� Emerging technologies = inedited design challenges

� No established design guidelines

� No standardized AR displays

� Conceive and create content for the AR application

� Evaluate and test the learning materials to be delivered

&

� AR and A²R challenges



Design challenges: Creating the third space…

Questions and issues to resolve

� What is the potential of AR for the museum visit?

� What is the look and feel of AR displays?

� How can A²R can be exploited?

� Which are the possible narratives and scenarios that can bedeveloped?

� Early design process, open to:� Technical partners

� CH professionals

� New Media Artists

� Museum visitors



Design challenges: Creating the third space…A Multi-layered, participatory design approach

Museum visitors

� On site, inobtrusive observations of museum visits

� Participatory observations of guided visits

� Inobtrusive observations of creative workshops

CH professionals

� Brainstorming sessions

� Interviews

� Testing of several AR displays in the museum environment and gesture interaction tests



Design challenges: Creating the third space…

Technical partners

� Immersion in the sensitive museum ecology

� Collaboration exercises among technical and CH partners

New Media Artists

� Collaboration of the Consortium with the International Artists’collective Manifest.AR



�Rendering visible the invisible

�Observing and comprehending artifacts from

multiple viewpoints

MNAD, Spain

MAM, FranceFACT, UK



Identifying motivations and needs: MNAD,

Madrid, Spain• Museo Nacional de Artes Decorativas, Madrid, Spain

• 3000 objects in the permanent exhibition, ca 30.000 visitors per year

• Spanish decorative arts: pottery, glass, textiles, carpets, jewellery, furniture, oriental collections (especially Chinese), avant-garde european and contemporary design

• Representative of History of Art, Design and Decorative Art museums



Motivations and needs: MNAD, Madrid, Spain

1. Observing artefacts from multiple viewpoints

2. Open hidden compartments

3. Contextualize

4. Facilitate the comparison of objects dispersed in different

collections

5. Linking with non exposed objects

6. Linking with other objects of the collection

(Compare “parallels”)



MNAD Selected Exhibit

Jorge 3D audio kitchen

Late 18th century tiled kitchen from Valencia



Gaze Tracking



The woman is speaking directly to me. Her voice comes from her head.

3D Audio spatialization

Position 3Position 1 Position 2



Motivations and needs: MAM, Paris,

France• Musée des arts et métiers, Paris, France

• 4.000 objects displayed, approximately 250.000 visitors per year

• A University museum

• The French National Museum of the History of Science and Technology



Identifying motivations and needs: MAM,

France, Paris

1. Reveal and assist visitors visualize complex physical

phenomena

2. Seeing the machines working again

3. Creating sound-scapes, reconstructing sounds

(intangible cultural heritage)

4. Historical context surrounding the invention



MAM Selected Exhibit

The Lavoisier Laboratory and the synthesis of water (1743-1794)

Activation of Right HemisphereNegative affect/avoidance

Activation of Left HemispherePositive affect/approach



Motivations and needs: FACT, Liverpool,

UK

• Leading organisation for the

development, support and exhibition

of film, video, new and emerging

media (established in 1989)

• 300.000 entries per year

• Non-existence of a permanent,

physical exhibition



Motivations and needs: FACT, Liverpool,

UK

1. Encourage different interpretations of the idea of an artifact

2. Present non-static works (video, narratives, interactives)

3. Expose inner workings of a building

4. Provide access to “denied” areas

5. Annotation of the architectural building as a public and social place



VIP Signature Pillar

Artists and auteurs, filmakers and talks

A growing, ongoing archive

A “hidden” story is revealed that is also the

history of events and programme

FACT history, major events, activities

This rich media content including moving image

Physiological data analysis based on moving image

media experience

FACT selected artifact



FACT, Liverpool, UK: Artwork commission

to Manifest.AR1. Implication of the International Artists Collective Manifest.AR

2. Artistic creation and technological innovation

3. A dedicated exhibition opening in June 2013 featuring eight mobile AR projects

4. More lightweight approach, “technology demistification”

http://manifestarblog.wordpress.com/manifest-ar-fact/



Conclusions• From “designing for” to “designing with”

• One step back, two steps forward?

• First impression = life circle of conceptual modeling and prototyping longer

• Better comprehension of technology = better approach towards the development of interactive features

• Better comprehension of the needs of CH professionals = pushing technology to the limits in a meaningful way

• A more enjoyable experience for all project stakeholders

• Museum professionals learning and inspired from each other



The potential of A²R for Art and Cultural

Heritage

• Visual monitoring and augmentation as important as audio and affective monitoring and augmentation

• Physiological (bio) sensors: monitoring of the affective impact of the AR visit

• How to interact once the interaction surface disappears?

• Gaining a better understanding of a visitor’s gaze upon a work of art (eye-tracking and creation of heat-maps)

• Learning about the physiology of museum visiting

• Content adaptation according to the visitors’ emotions and states

• Can A²R assist us for the shaping of a rewarding museum visit?

• What can we learn about the interrelations among emotion, cognition and learning?



www.artsense.eu

Thank you for your attention!

Questions?

ISMAR ARtSENSE Contacts

[email protected]

[email protected]

[email protected]

[email protected]