Digital humanities visualisation & playful

learningA joint Centre for Culture and Technology and Australia-Asia-Pacific Institute presentation!

Seminar 3 !

@Curtin HIVE (Hub for Immersive Visualisation and eResearch) John Curtin Gallery, Building 200A

Original abstract (non-core promise)

• DH 'big tent' or cash-grab in a crowded campsite? In his chapter 'The Digital Humanities or a Digital Humanism’ in Debates in the Digital Humanities, (Gold, 2012), Dave Parry raises the controversial question as to whether DH would be best considered as the application of computing, or an inquiry as to how digital media has irrevocably changed the Humanities.

• IMO: DH at its core considers how to integrate computing with humanities and attempts to understand how both computing and humanities must change.

• DH is/are computing services + tools for digitalization and processing of text or literature (Baldwin, 2013).

• digital humanities is/are primarily text-based and non-text based media is not part of DH?

• visualisation cannot provide suitable scholarly arguments;

• clear separation between written language and images;

• to be a humanist or a humanistic scholar (not the same thing) one has to have high levels of literacy.

• WAYWARD PROOF: examples through immersive environments and interaction, (both playful and serious).

Second part

• Digital humanities heritage games & supercomputing for hobbits

• What can humanities academics and students do with visualisation and interaction tools?

DH tools or services• http://www.europeana.eu/

• http://www.dariah.eu/

• http://clarin.eu/

• http://www.openedition.org/

• http://dirt.projectbamboo.org/

• NEDIMAH

• http://www.hastac.org/

• http://www.neh.gov/divisions/odh

• http://digitalhumanities.org/centernet/about/ esp dhcommons.org

http://chcta.wordpress.com/

IS DH text-based?: Lascaux-Marc Azéma

• Stone Age Art Caves May Have Been Concert Halls http://news.nationalgeographic.com.au/news/2008/07/080702-cave-paintings.html

• Rock art acoustics https://sites.google.com/site/rockartacoustics/

• Using Subterranean Acoustics To Explore Ancient Cave Art http://www.gizmodo.com.au/2014/02/using-subterranean-acoustics-to-explore-ancient-cave-art/

• http://www.landscape-perception.com/archaeoacoustics/

• http://gtr.rcuk.ac.uk/project/F6E2B372-A47B-4756-AD23-31F3E2F43F26

http://museuprehistoriavalencia.com/ficha_actividades.html?cnt_id=4141 https://www.youtube.com/watch?v=f4HbcdISySY https://www.youtube.com/watch?v=axQSc9P0DsI

LASCAUX “The 12,000-year-old cave paintings appear to be placed at acoustically resonant locations, and

researchers now theorise they may have been inspired by musical performances held in caves.”

http://www.abc.net.au/science/articles/2008/07/03/2293114.htm

Visual Scholarly Argumentshttp://pelagios-project.blogspot.com.au/

http://pelagios.github.io/pelagios-heatmap/

Problem: Integrate Text+Modelhttp://gap.alexandriaarchive.org/gapvis/

index.html#index

What is Spatial History? Richard White

• One of the important points that I want to make about visualizations, spatial relations, and spatial history is something that I did not fully understand until I started doing this work and which I have had a hard time communicating fully to my colleagues: visualization and spatial history are not about producing illustrations or maps to communicate things that you have discovered by other means. It is a means of doing research; it generates questions that might otherwise go unasked, it reveals historical relations that might otherwise go unnoticed, and it undermines, or substantiates, stories upon which we build our own versions of the past.

Archiving is very importantfrom:

http://imgace.com/pic/2013/04/north-koreas-secret-weapon/

instantar.org marker less augmented drawing and stories

free Bologna in Blenderhttp://blenderprops.altervista.org/works/apa-the-etruscan-and-270-years-of-bolognas-history-test/

Film Events (Machinima)

http://www.sourcefilmmaker.com/

http://moviesandbox.com

http://www.thesims.com/de-de

Combine images, panos http://www.petermorse.com.au/vrar/vr/

Iphone: http://www.youtube.com/watch?v=s9sBtuCuju0 Technical description http://paulbourke.net/dome/UnityiDome/

Other pano examples http://paulbourke.net/transient/Beacon/beacontour.html

hobbit supercomputer• Can we access supercomputing for media

studies and production?

• http://news.bbc.co.uk/2/hi/technology/3672887.stm

• 2004. runs the third largest supercomputer on the planet..The ones that beat Weta WERE the Japanese Earth Simulator (5120 processors) and Los Alamos National Laboratory's supercomputer (8192 processors).

• http://www.stuff.co.nz/technology/digital-living/3763453/Weta-a-super-power-in-super-computers

• IN 201, 6 times in top 500, most in top 300

• NB More Data in LOTR 3 than LOTR1+2

Martin Freeman - Hobbit by Karolina5n on deviantART, karolina5n.deviantart.com1024 × 1533Search by image Martin Freeman - Hobbit by karolina5nhttps://encrypted-tbn1.gstatic.com/images?q=tbn:ANd9GcS-Utf5lVdrEErcSp_nNAClkZs6II7JGtvaG2AEj72gRcrSbtVs

game engines

LBP1+2

• http://www.mediamolecule.com/blog/article/kareems_talk_from_learning_without_frontiers_2011/

Playing History Plague – Slave trade -

VikingsChallenge: ..the belief that it is exciting

to learn about history. Integrates learning and playing in a way that engages pupils and gives them a concrete feel for the historical time and

setting Solution:

Platform: Mac/PC, single player, browser Technology: 3D Unity game engine

Playtime: Per game 60 minutes Target group: 9-14 years old

Alternative game modesTurkey Maiden Educational Computer Game

http://www.thenightjourney.com/statement.htmhttp://digitalethnography.dm.ucf.edu/

Games and Role-playhttp://publicVR.org

OR video at http://vimeo.com/25901467

iVEs from VR systems

http://www.ntnu.no/ub/omubit/bibliotekene/gunnerus-1/mubil

CAVE via game enginehttp://cryve.id.tue.nl

games..• Games are great learning environments

• Except for Cultural Significance, history and heritage

• Conclusion: problems and solutions

• Technology=barrier but not issue: learning is the problem.

• Which historical principles are used, learnt and applied?

• Inhabitants’ points of view (heritage) is missing

• Scholarly cycle incomplete, community cycle inextensible

VH is not VR• “Virtual reality is the use of computers and human-

computer interfaces to create the effect of a three- dimensional world containing interactive objects with a strong sense of three-dimensional presence.”

• The importance of using HMDs or CAVEs, for VR apparently requires “a head-tracked, usually a stereoscopic, display that presents the virtual world from the user’s current head position, including the visual cues required..”

What Is Left Out?• Beliefs, rituals, other cultural behaviours and activities?

• Traces the level of certainty, and authenticity of reproduction and reveals process? I.e. scholastic rigor?

• Sensitive to the needs of audience & shareholder?

• Virtual heritage is the attempt to convey not just the appearance but also the meaning and significance of cultural artifacts and the associated social agency that designed and used them, through the use of interactive and immersive digital media.

4 Issues With Culture• Definition (and relation to

place and inhabitation)?

• How is culture transmitted?

• Transmit local situated cultural knowledge to “others”?

• VH: how to transmit via digital & augmented media?

Why 3D?• To evoke +communicate historical situations or heritage values find

deeper understandings not simply memorize facts (Bloom, 1956).

• Place is a cultural setting, it gives cultural interaction a time and a location, Crang (1998, p.103), “Spaces become places as they become ‘time-thickened’”

• Places do not just organise space, they orient, identify, and animate the bodies, minds, and feelings of both inhabitants and visitors.

• Cultural presence: a feeling in a virtual environment that people with a different cultural perspective occupy or have occupied that virtual environment as a ‘place’.

my interest..• Background in architecture, (art

history) and philosophy

• PhD with Lonely Planet in VEs for travel and tourism

• Taught interaction design and game design at UQ, UNSW, Massey

• Project Manager at DIGHUMLAB (Denmark)

• Professor of Cultural Visualisation, Curtin University, Perth, Western Australia

New mediathe act of reshaping the user experience of exploring realms or worlds

through the innovative use of digital media.

other mods

Palenque ported to UnrealAdobe Atmosphere, Poser, 3D Studio Max, Arc GIS,

PHP, Javascript, HTML

Design past artefacts, events, rituals or customsUnreal Tournament / Xibalba-Palenque

Surround projection 2005

•

Games aren’t Challenging?• A rule-based formal system with a variable and quantifiable

outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the outcome, the player feels attached to the outcome, and the consequences of the activity are optional and negotiable. (Juul 2003, para 15).

• A system in which players engage in an artificial conflict, defined by rules, that results in a quantifiable outcome. (Salen and Zimmerman, 2004).

• A challenge that offers up the possibility of temporary or permanent tactical resolution without harmful outcomes to the real world situation of the participant (Champion, 2006).

Games are culturally significant?• For evoking +communicating historical situations or

heritage values we must deeper understandings rather than simply memorizing facts (Bloom, 1956).

• What is the cultural significance of what is represented and interacted with?

• Cultural presence, a feeling in a virtual environment that people with a different cultural perspective occupy or have occupied that virtual environment as a ‘place’.

Prescriptive / Procedural learning?

Gamer: Reach objectives as quickly & vividly as possible. Activity

Tourist: Enjoy highlights safely and conveniently. Viz: Weekend in Capri

Traveller: complete tasks via local affordances. Activity: Myst

Archaeologist: Discover past via examining material remains, geographical changes, epigraphy etc.

Viz of process: ArcDig, detective series?

Anthropologist: Understand the beliefs roles and relationships of inhabitants in context. geographical

changes, epigraphy etc.

Hermeneutic: Myst, Sims? Oblivion?

Problem: Inhabitants’ PoV• Can users learn via interaction the meanings and values

of others, do we need to interact as the original inhabitants did?

• How can we find out how they interacted?

• Can the limited and constraining nature of current technology help interaction become more meaningful, educational and enjoyable (Handron & Jacobson, 2010)?

• How do we even know when meaningful learning is reached?

Problem: Rituals• Attention/focus

• Social judgement

• Territoriality

• Social Proxemics

• Being “in the flow”

• Physical delineation (profane vs sacred)

• Event-based or regularhttp://www.youtube.com/watch?v=MqMXIRwQniA

Image: http://www.virtualtripping.com/google-earths-rome-reborn/ 2008

Problem: Sensory immersion

Affective Process

Biofeedback middleware

Kinect 1/2: voice + skeletonSkyrim has motion tracking and voice commands

http://uwnews.washington.edu/ni/article.asp?articleID=37134

Vocal Joystick surfs the Web• Listen in as someone uses

Vocal Joystick to browse the Web.

• Eight vowel sounds move the cursor in different directions.

• Louder noises move the cursor more quickly.

• The sounds “k” and “ch” simulate clicking and releasing the mouse buttons.

Problem: Violence• No realistic humans

• No social judgement

• No time to think

• Gun based genres are commonplace

• Weaponry skill can be easily levelled up

• Typical single player

• Demographics

Alternatives To Violence• Reflexivity: A reflective space, where players relax & consider the consequences of their actions

• Performativity: Players asked to perform or orate and present their experience of the VE in class.

• RPG Virtue Ethics: Characters change in relation to development of virtue ethics.

• Consequentialism: Consequences of player actions affect their future gameplay. through the game.

• Creative Uses For Weapons.

• NPC distaste and disparagement: they discourage violence.

• Biofeedback: Performance based on calmness.

• Expressive and embodied modes of interaction.

• Non-violent competition.

• Players become morally accountable for their actions.

• Ritual or mythical use of weapons.

Touchscreen Taoism

Chinese Taoism Touch Screen by Neil Wang and Erik Champion

(VSMM2012 conference) Opening:

Game Hua - http://youtu.be/DiGDezTM8hY

Game Qi1 - http://youtu.be/jP9nfdUFDTU

Game Qi2 - http://youtu.be/orCga2CQBjs

Game Qin - http://youtu.be/iC2BGT5IbDE

Game Shu - http://youtu.be/dv_TOnl_sbc

• Collaborative learning: HACK4LT, VILNIUS LITHUANIA

Problem: Book-based?• Technology or evaluation is not the fundamental problem.

Skeates (2000) warned that archaeologists need to reconsider their field as a communication medium, and not just as a closed scientific discipline.

• Books presuppose a vast domain of knowledge, a certain learnt yet creative technique of extrapolation..

• ..They typically do not cover the experiential detective work of experts that visit the real site (Gillings, 2002).

• An academic publication is also a simplification and metaphorical extension of the remains and ruins it describes.

Problem: Book-based?• Ideally, virtual environments may help the general public to

• create, and share and discuss hypothetical or counterfactual places

• meet virtually in these places with colleagues to discuss them

• work in these recreations to understand limitations forced on their predecessors

• develop experiential ways to entice a potential new audience to both admire the content and the methods of their area of research

• Scholarly knowledge does not easily translate to audience knowledge; nor does it always best engage the public.

• IF we can use digital worlds for teaching +learning about heritage &history, is it preferable to learn about a collection of culturally situated past experiences, or a strictly academic procession of historical events?

• Smith: confusion between history as meaning the past, and history as being something produced by historians.

• Given that even philosophers such as Goldstein (1964) and Gale (1962) disagreed on what constitutes history and what constitutes recollection of the past; how can students or the general public reliably distinguish between the two?

• How it can be or should be accessed?

Problem: Evaluation• The Siren call of technological determinism and fancy pictures.

• Digital media as purely a shop façade for the serious and scholarly past time of reading and writing books (Parry 2005; Gillings, 2002).

• Yet if we avoid teaching with digital media, how will the changing attention spans and learning patterns of new generations be best addressed (Mehegan, 2007)?

• Even if we decide on what we are evaluating, it is not clear how to evaluate. !

• The ethnographic techniques used by researchers may be effective in recording activity, but they do not directly indicate the potential mental transformations of perspective that result from being subjectively immersed in a different type of cultural presence (Benford et al, 2002).

Evaluating VES - People• Task performance (quantitative or

qualitative)

• Likert or statistical evaluation

• Extrapolated understanding

• Personal ‘sense’ of cultural presence

• What do they choose next (exit strategies)

• ‘Teach the teacher’ et al methods

• Excitement recorded from biofeedback

http://lilypadarduino.org/

Mixed Reality

http://virtual.vtt.fi/virtual/proj2/multimedia/projects/mrconference.html

http://ael.gatech.edu/lab/research/arsecondlife/using-the-ar-second-life-client/

publicVr.org Jeffrey jacobson

puppet control

The cost of Stereo VR

motion capture

Conclusion• Games as Virtual Environments may connect more people,

more thematically without competing with book learning.

• Background research needed for public vs. scholar needs.

• Game conventions ‘work’ but meaningful learning elusive.

• We lack interactive and immersive digital history projects that are meaningful and engaging learning experiences.

• Mixed reality in history and heritage has many advantages but few working exemplars.

References• BBC Ancient History Section. (Undated). Death in Sakkara: An Egyptian Adventure, http://www.bbc.co.uk/history/interactive/

animations/ironage_roundhouse/index.shtml

• Benford, S., Fraser, M., Reynard, G. Koleva, B., and Drozd, A. (2002). Staging and Evaluating Public Performances as an Approach to CVE Research, Proceedings of the 4th international conference on Collaborative virtual environments, ACM New York.

• Bloom, B. S. (1956). ‘Taxonomy of Educational Objectives’, Book 1 Cognitive Domain. New York: Longman Inc.

• Champion, E. (2008). ‘Otherness of place: Game-based interaction and learning in virtual heritage projects’, International Journal of Heritage Studies, 14(3), 210-228.

• Dondlinger, M. J. (2007). ‘Educational Video Game Design: A Review of the Literature’, Journal of Applied Educational Technology, 4(1), 21-31.

• Handron, K., & Jacobson, J. (2010). Extending Physical Collections Into the Virtual Space of a Digital Dome,. Paper presented at the 11th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST), Paris, France.

• Hight, J. (2006). ‘TEXT: Narrative Archaeology: reading the landscape’, newmediafix, http://newmediafix.net/daily/?p=638

• Leader-Elliott, L. (2003). ‘Community Heritage Interpretation Games: A Case Study from Angaston, South Australia’, International Journal of Heritage Studies, 11:2, 161-71.

• Gale, R.M. (1962). ‘Dewey and the Problem of the Alleged Futurity of Yesterday’, Philosophy and Phenomenological Research, 22(4), 501-511.

• Gillings, M. (2002). Virtual archaeologies and the hyper-real, in P. Fisher, D. Unwin, (eds.), Virtual Reality in Geography (London & New York: Taylor & Francis, 2002), 17-32.