PROMOTING GENERALIZATION USING IMMERSIVE …...PROMOTING GENERALIZATION USING IMMERSIVE TECHNOLOGIES 5 Figure 1. View of 3D CVLE personal pods providing a visual cue indicating where

PROMOTING GENERALIZATION USING IMMERSIVE TECHNOLOGIES 1

Promoting Acquisition and Generalization of Skills for Individuals Severely Impacted by Autism Using Immersive Technologies

Matthew Schmidt, PhD*

Noah Glaser* Carla Schmidt, PhD* Dennis Beck, PhD**

Heath Palmer*

* University of Cincinnati ** University of Arkansas

Abstract. This chapter outlines the influences and supports for Virtuoso, an immersive 3D learning intervention for adults severely impacted by autism. In addition, the design of Virtuoso’s prototype intervention focusing on learning to use public transportation is described, along with descriptions of a spherical video-based virtual reality mobile app and a three-dimensional collaborative virtual learning environment. Clear background is provided on autism spectrum disorders and their impacts on individuals’ ability to transition from secondary education into adulthood, including the use of information and communication technologies as a means for providing interventions for this population. Research literature is explicated that provides insight into how to promote behavior and skill generalizations with an explanation of how specific skills generalization heuristics map onto the design topography of Virtuoso’s public transportation prototype. The chapter concludes with a discussion of the promise of immersive technologies and possible future intersections related to the strengths and challenges of current research. Keywords. Immersive learning, Autism Spectrum Disorder, Three-dimensional collaborative virtual learning environments, Virtual reality, Spherical video-based virtual reality, Video modeling


Introduction This chapter presents the theoretical and conceptual grounding for a suite of prototype

immersive technology supports designed for individuals with Autism Spectrum Disorder (ASD) who are transitioning from secondary education into adulthood. Entitled Virtuoso (a play on the words “virtual” and “social”), this project is an immersive technology-based intervention for individuals with ASD who are enrolled in the Impact Innovation adult day program at the University of Cincinnati (UC). The general focus of Virtuoso is to promote the acquisition and generalization of adaptive skills in safe, completely controllable virtual reality environments. In its current form, Virtuoso provides virtual training on the use of public transportation. This suite of immersive technology prototypes includes a three dimensional collaborative virtual learning environment (3D CVLE) designed to support the acquisition and generalization of adaptive skills related to use of the UC shuttle system.

Project description

Virtuoso is a a subset of the Impact Innovation program at the University of Cincinnati. Impact Innovation is an adult day program for individuals with significant communication and behavioral challenges associated with ASD. Over 20 adults participate in Impact Innovation year-round. Participants follow a uniquely designed daily schedule with the assistance of a peer mentor. Participants take part in a variety of vocational internships in a number of domains as part of an employment exploration process. Healthy lifestyles are encouraged through a variety of activities, including exposure to the Health Matters curriculum (Marks, Sisirak, & Heller, 2010), daily life skills training, hygiene interventions, and time at the UC recreation center. Finally, as a means to foster a high quality of life, the Impact Innovation program encourages lifelong learning based on participant’s individual interests.

Virtuoso serves the Impact Innovation program by providing technological interventions and training for Impact associates. The focus of this chapter is the public transportation training prototype that combines curriculum, technology, and applied behavior analysis techniques to promote the overarching goal of helping Impact associates become more independent users of public transportation. The overarching learning objective is for learners to be able to use the UC shuttle. Impact associates complete four stages of curriculum to complete the public transportation training prototype. The first two stages focus on acquisition of the skill and the second two stages on generalization of the skill. First, associates review a social narrative presented on an iPad. Second, associates engage in computer-based video instruction using an Android-based spherical video-based virtual reality (SVVR) app. Third, associates practice the skill in an interactive 3D CVLE. Fourth and finally, associates perform the skill in the real-world. Each stage of the curriculum can be repeated as many times as needed. This provides a way to learn about and virtually practice using the public transportation that is safe and predictable. In the following sections, a detailed description of how associates engage with the Virtuoso 3D CVLE is provided.

Background and Context ASD affects one in 68 people in the United States (Centers for Disease Control, 2014).

Individuals with ASD require supports because they struggle in varying degrees with a triad of impairments commonly associated with the disorder (Wing, 1988). These impairments typically manifest across (1) social, (2) communicative, and (3) behavioral domains (American Psychiatric Association, 2013). Challenges across these domains lead to deficits in adaptive skills, that is, practical, everyday skills needed to function and meet the demands of one's environment,


including the skills necessary to effectively and independently take care of oneself and to interact with other people. Hence, individuals with ASD can benefit from supports during their transition from secondary education to adulthood. Before transitioning and while enrolled in the U.S. public education system, they typically receive multiple supports. The individualized education program (IEP) plays a central role in preparing for the transition from secondary education to adulthood. At the age of 16, the IEP process is required to include services to help plan for this transition (20 U.S.C. § 614(d)(1)(A)(VIII)). However, the legal entitlements mandated by the IEP end at the age of 21. This absence can lead to significant obstacles for young adults related to employment (Wei, Wagner, Hudson, Yu, & Shattuck, 2015). The majority of these individuals are unemployed or underemployed after completing their secondary education (Taylor & Seltzer, 2011). To promote more positive outcomes, continued supports are needed (Friedman, Warfield, & Parish, 2013).

Three-dimensional Virtual Learning Environments for Individuals with ASD

Information and communications technologies (ICT) have the potential to address some of the challenges associated with providing continuing supports after completing postsecondary education. ICT for individuals with ASD is an area that has received substantial interest in the literature (Goodwin, 2008; Odom, et al., 2015), demonstrating strong potential as an intervention modality for treating the impairments associated with ASD (Grynszpan, Weiss, Perez-Diaz, & Gal, 2014). Virtuoso utilizes a variety of technologies to provide continued supports for transitioning adults with ASD, including an interactive 3D CVLE.

Three-dimensional virtual learning environments (3D VLE) are a family of immersive technologies that include virtual reality, video games, and virtual worlds. 3D VLE can be manipulated at a deep level, thus providing researchers with a high level of experimental control (Strickland, 1997); they can provide realistic contexts within which skills can be practiced safely and without physical danger (Strickland, 1997; Standen & Brown, 2005), they can promote new insights into collaborative learning using computational methods such as data mining (Schmidt & Laffey, 2012), etc. The majority of 3D VLE interventions have focused on single-user applications (Rutten, Cobb, Neale, Kerr, Leonard, & Parsons, et al., 2003; Trepagnier, Olsen, Boteler, & Bell, 2011). However, researchers slowly have begun to incorporate multi-user, collaborative activities into their interventions (e.g., Jarrold, Mundy, Gwaltney, Bailenson, Hatt, & McIntyre, et. al 2013; Lorenzo, Pomares, & Lledó, 2013; Moore, Cheng, McGrath, & Powell, 2005). 3D CVLE technologies may convey concepts, meanings, and a symbolic measure of representing real world activities through photographic realism as well as embodiment as avatars that promote socio-communicative activity (Wang, Laffey, Xing, Ma, & Stichter, 2016; Wallace, Parsons, & Bailey, 2017). The ability to control input stimuli, visual fidelity, and interactivity within 3D CVLE systems at a remarkably refined level allows designers and researchers to create interventions that are uniquely attuned to the particular needs of learners with ASD. Technology Interventions for Using Public Transportation

Central to promoting the goals of the Impact Innovation program is transportation. Transportation is provided to Impact program associates via the UC shuttle system. Shuttle training is included in Impact Innovations’ training repertoire because the ability to use public transportation such as the UC shuttle bus can increase independence through access to employment, medical care, community, etc. However, transportation is among the most cited barriers across a variety of settings for individuals with disabilities (Allen & Moore, 1997; Carmien, et al., 2005). Davies and his colleagues (2010) maintain:


Public transit systems (e.g., fixed route public buses) provide the most commonly used or available transportation option and provide, probably, the best option for those living in urban areas for independent, timely, integrated, inexpensive, and relatively unrestricted mobility for people with intellectual and developmental disabilities. However, transit buses also present a unique set of barriers due to route complexity, transfer requirements, unfamiliar destinations, schedule complexity, and other cognitively loaded requirements needed for successful transit system navigation. (p. 455)

An example of a public transportation intervention using immersive technology for adolescents with ASD is provided in Parsons, Leonard, and Mitchell’s (2006) description of a technology-based training that focused on social skills specifically targeting the skill of finding a place to sit in two contexts: a virtual cafe and a virtual bus. The virtual cafe context utilized a 3D VLE for delivery of the intervention, and the bus context utilized computer-based video instruction. Two participants underwent the training, during which the level of complexity was increased after each successful completion of the corresponding training scenario. Researchers found some evidence of skills generalization in terms of participants self-reporting potential ability to relate the training to situations in the real world. These researchers suggest that generalization of skills learned in a virtual world is an area for future research.

Use Case Example: Virtuoso 3D CVLE Participants with ASD engage in the 3D CVLE portion of Virtuoso by logging into the

High Fidelity virtual reality toolkit to connect to a privately hosted domain. Learners use avatars as representations of themselves to interact with others and the environment and speak using microphone-equipped headsets. Activities are led by an online guide (similar to a coach). The virtual space is structured to allow the practice of the skills needed to catch a ride on the UC shuttle. Learners explore and complete their training tasks within a virtual replica of the University of Cincinnati. Today, Jonah, an Impact Innovation associate, has returned to the Impact Innovation office, where his staff member, Shana, reminds him that he will be using Virtuoso for shuttle training. Jonah has used the 3D CVLE in the past and is familiar how to use it. As Jonah dons a head-mounted display (HMD), his display is mirrored onto a desktop computer so Shana is able to observe his progress. She sees that Jonah is now standing inside of the Impact office and that there is another avatar standing in the office with him. That person is an online guide that is being controlled by Carla. Carla introduces herself and directs Jonah’s attention to a series of personal pods positioned on the floor in the office suite. Jonah is instructed to go stand on one and to look at the nearby image of their daily schedule. He moves his avatar onto a personal pod, which changes color to red after he enters it. Carla asks Jonah to read the schedule and state what today’s task will be (Figure 1).


Figure 1. View of 3D CVLE personal pods providing a visual cue indicating where to stand while receiving instruction (1); pods turning red when occupied provide a visual cue to stop avatar movement (2). Jonah reads the schedule, and Carla provides specific positive praise: “Great job, Jonah. I really like how you were able to correctly identify your next task.” She then directs Jonah to look at the campus map (Figure 2).

Figure 2. View of an Online Guide gesturing to the map in the virtual Impact office; two associates stand on their personal pods as they receive instruction.


Jonah turns his avatar, where he sees a path from the Impact office to the shuttle stop indicated with a red dotted line. Carla asks Jonah if he can tell her where they will be walking to catch the shuttle bus today, and what cues from the map might help him answer the question. Jonah audibly references the dotted lines and the bus stop that is highlighted on the map. Carla confirms that what Jonah has said is correct and provides specific positive praise. Carla then prompts Jonah to exit the Impact office suite. Jonah walks through a door and arrives on the UC campus. Carla joins him on campus and asks if he needs a break. Jonah does not, so they continue through the tasks. Personal pods are situated throughout the environment to serve as visual reminders of where Jonah should stand to receive verbal instruction from Carla. Navigational components such as dotted lines and signs are present to act as scaffolding that can be faded as Jonah becomes more familiar with the intervention (Figure 3).

Figure 3. Navigational tools used throughout the 3D CVLE including personal pods and navigation aides such as clearly demarcated pathways and signage. Carla and Jonah walk to the shuttle stop. Jonah follows the path, using signs as cues of where to go. Once at the shuttle stop, they review the shuttle schedule using a shared virtual display and then wait on a bench until the shuttle arrives. Carla asks Jonah to confirm if it is the correct bus. Jonah confirms and Carla provides specific positive praise. She them prompts Jonah to board the shuttle. As Jonah and Carla board the bus, they are greeted with an digital achievement that expresses congratulations for completing the session.

Figure 4. Jonah boards the shuttle (1) and is then presented with a digital achievement (2).

Design Considerations: Applying Heuristics for Generalization Technology-mediated interventions can influence participants’ acquisition of skills and behaviors, but generalization of skills to novel contexts presents a challenge. Stokes and Osnes (2016) define generalization as observable changes in behavior in settings different from the


training environment and “across stimuli, responses, and time” (p. 721). Stokes and Baer (1977) identified nine techniques designed to promote generalization, which were later categorized into three broad principles (Stokes & Osnes, 2016), including taking advantage of natural communities of reinforcement, training diversely, and incorporating functional mediators. This section considers how those heuristics are embodied across Virtuoso’s UC shuttle system training prototype.

Take advantage of natural communities of reinforcement. The first principle, taking advantage of natural communities of reinforcement, refers to using elements of the natural environment that already function to maintain the target behavior (Stokes, Fowler, Baer, 1978). Natural communities of reinforcement are aspects of the natural environment that serve to increase or decrease behavior. In order for someone to benefit from the reinforcing aspects of the natural environment, teaching of a skill or behavior is needed. In the public transportation training prototype, Virtuoso teaches the skill by first introducing it using a social narrative and then representing those skills through video modeling. It is also the case that sometime participants are not aware of how fully to take advantage of the reinforcing aspects of a naturally occurring situation. For example, an Impact associate might not be aware of how to find a shuttle stop or how to check the shuttle schedule. To account for this problem, Virtuoso teaches associates to use the UC map to find a bus stop and to check the UC shuttle app as part of their training (Figure 5).

Figure 5. Impact associates learn to use the UC shuttle app in a shared web browser (1) and to navigate to the shuttle stop using a UC map (2).

Train diversely. The second principle, train diversely, refers to maintaining the minimal level of training control possible while still producing behavior change (Stokes and Osnes, 2016). If training is too controlled and focused, then effects are likewise very focused. Hence, the likelihood of generalization is diminished for skills and behaviors that are learned this way. Correspondingly, the focus and outcomes of training should be broader, and training should be more diverse. One way to do this is to diversify the training conditions such as setting, trainer, etc. Virtuoso provides diversity across settings by intentionally varying locations and methods of representation. For example, training takes place in the Impact offices, on the UC campus, at the shuttle stop, etc. and is represented across a continuum of technologies. Further, using multiple exemplars of skills in teaching, modeling, practice and practice and exerting looser control over the situations that are intended to elicit target behaviors can promote generalization. Virtuoso employs a variety of exemplars to provide diversity and presents these exemplars across a variety of contexts. For example, when engaging in structured practice in the 3D CVLE, the tasks are


practiced with other people and across different contexts, causing them to become less uniform and are therefore less predictable. Varying the circumstances of the training scenario makes the situations less predictable.

Incorporate functional mediators. The third principle, incorporate functional

mediators, refers to taking advantage of relevant discriminative stimuli in the training environment that can be transferred to other environments to promote generalizations (Stokes & Osnes, 1986, 2016). The technologies employed in Virtuoso are examples of a functional mediator. That is, the four-tiered scaffolding strategy of first completing a social narrative, then reviewing video models, then practicing in the 3D CVLE, and finally practicing in the real-world serves to mediate the generalization of skills from the training context to the real-world. To this end, Virtuoso parallel constructs salient physical, social and self-mediated physical stimuli found in the real-world across all technology scaffolds. According to Stokes and Osnes (2016), salient physical stimuli “may be a physical object that is present in both the training and the generalization setting, or at least very similar items are present in both settings” (p. 727). Salient social stimuli can be “the characteristics of a person, such as a certain gesture, or the presence of the person” (p. 728). Salient self-mediated physical stimuli might be “the use of a notebook that specifies how to perform in a certain setting” (p. 728). Virtuoso embodies these principles across technology scaffolds. For example, social narratives employ real-world photographs of Impact staff members, physical settings, and job aids (e.g., annotated maps, schedules). In the 3D CVLE, the Impact offices and the UC campus have been modeled after their real-world counterparts with a high degree of photographic fidelity and also provide the same stimuli as provided in other scaffolds. As an embodiment of Stokes and Baer’s (1977) heuristics to promote generalization, Virtuoso’s public transportation training prototype seeks to promote the acquisition and generalization of associated skills.

Discussion and Conclusion

The purpose of this paper was to describe the influences and supports for Virtuoso, an immersive 3D learning intervention for young adults with ASD that adopts a multidimensional approach for acquiring adaptive skills in a safe environment that can be manipulated so as to reduce input stimuli and adapt to learner needs. A description of ASD and its impact on individuals’ ability to transition from a secondary education environment to adulthood was provided, with an emphasis on the skills necessary to do so. A background on the use of ICTs as interventions for this population was provided, with an emphasis on the use of 3D CVLE to support adaptive skills training. The background on the Impact Innovation program was provided, as well as the target skill area for Virtuoso. Finally, heuristics that provide insight into how to promote skills generalizations and how specific skills generalization heuristics map onto the topography of Virtuoso were considered.

While the current state of Virtuoso and its implementation is clear, its future is not. How will Virtuoso leverage the promise of immersive technologies for data mining and customization against the very real problems of user-generated environments and privacy? Immersive environments are challenging to implement in practice due to expense and skill required for customization, their inability to scale to multiple geographic locations, and their impacts on privacy issues. Unfortunately, immersive environments are not customizable without spending much money and effort. Privacy and control over personal data is another increasing concern for all technology users (World Economic Forum, 2017). Current features of immersive


environments enable data to be collected from eye-movements, facial expressions, and haptic data, as well as from more traditional means of data mining (i.e., automated collection of trace data). Unfortunately, this fact means that the personal data at risk in immersive environments is even more invasive and personal than the data that is currently available on the Internet, making concerns about user privacy a far more serious issue. Forty-seven percent of people in six nations have stopped or avoided using a technology based service because of inadequate user controls over personal data (World Economic Forum, 2017). This statistic is of concern because it may mean that caregivers of and individuals with ASD may be pushed from the long-term adoption of immersive technologies due to these privacy issues. This result would be unfortunate due to the incredible potential of these technologies to increase quality of life for individuals with ASD. For Virtuoso, this trend highlights a need for continuous and rigorous attention to Institutional Review Board criteria and a commitment to store data in secure, offline locations. It also means that Impact Innovations Program at UC, as well as other future programs that Virtuoso partners with, maintain similar, highly secure, data storage facilities.

References Allen, S. M., & Mor, V. (1997). The prevalence and consequences of unmet need: Contrasts

between older and younger adults with disability. Medical care, 35(11), 1132-1148. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders

(5th ed.) Washington, DC: Author. Carmien, S., Dawe, M., Fischer, G., Gorman, A., Kintsch, A., & Sullivan Jr, J. F. (2005). Socio-

technical environments supporting people with cognitive disabilities using public transportation. ACM Transactions on Computer-Human Interaction (TOCHI), 12(2), 233-262.

Centers for Disease Control and Prevention. (2014). Autism spectrum disorders: Data and statistics.

Council for Exceptional Children (2014). Policy manual. Retrieved from https://www.cec.sped.org/~/media/Files/Policy/2014%20Policy%20Manual.pdf

Davies, D. K., Stock, S. E., Holloway, S., & Wehmeyer, M. L. (2010). Evaluating a GPS-based transportation device to support independent bus travel by people with intellectual disability. Intellectual and Developmental Disabilities, 48(6), 454-463.

Friedman, N. D., Warfield, M. E., & Parish, S. L. (2013). Transition to adulthood for individuals with autism spectrum disorder: Current issues and future perspectives. Neuropsychiatry, 3(2), 181-192.

Goodwin, M. S. (2008). Enhancing and accelerating the pace of autism research and treatment: The promise of developing innovative technology. Focus on Autism and Other Developmental Disabilities, 23(2), 125-128.

Individuals with Disabilities Education Improvement Act of 2004, 20 U.S.C. § 614 et seq. Grynszpan, O., Weiss, P. L., Perez-Diaz, F., & Gal, E. (2014). Innovative technology-based

interventions for autism spectrum disorders: A meta-analysis. Autism, 18(4), 346–361. https://doi.org/10.1177/1362361313476767

Jarrold, W., Mundy, P., Gwaltney, M., Bailenson, J., Hatt, N., McIntyre, N., et al. (2013). Social attention in a virtual public speaking task in higher functioning children with autism. Autism Research, 6, 393-410.


Lorenzo, G., Pomares, J., & Lledó, A. (2013). Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with Asperger syndrome. Computers & Education, 62, 88-101.

Marks, B., Sisirak, J., Heller, T., & Wagner, M. (2010). Evaluation of community-based health promotion programs for special olympics athletes. Journal of Policy and Practice in Intellectual Disabilities, 7(2), 119-129.

Moore, D., Cheng, Y., McGrath, P., & Powell, N. J. (2005). Collaborative virtual environment technology for people with autism. Focus on autism and other developmental disabilities, 20(4), 231-243.

Odom, S. L., Thompson, J. L., Hedges, S., Boyd, B. A., Dykstra, J. R., Duda, M. A., ... & Bord, A. (2015). Technology-aided interventions and instruction for adolescents with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45(12), 3805-3819.

Parsons, S., Leonard, A., & Mitchell, P. (2006). Virtual environments for social skills training: Comments from two adolescents with autistic spectrum disorder. Computers & Education, 47, 186-206. http://dx.doi.org/10.1016/j.compedu.2004.10.003.

Rutten, A., Cobb, S., Neale, H., Kerr, S., Leonard, A., Parsons, S., & Mitchell, P. (2003). The AS interactive project: Single-user and collaborative virtual environments for people with high-functioning autistic spectrum disorders. Computer Animation and Virtual Worlds, 14(5), 233-241.

Schmidt, M., & Laffey, J. (2012). Visualizing behavioral data from a 3D virtual learning environment: A preliminary study In System Science (HICSS), 2012 45th Hawaii International Conference on (pp. 3387-3394). IEEE. https://doi.org/10.1109/HICSS.2012.639

Standen, P. J., & Brown, D. J. (2005). Virtual reality in the rehabilitation of people with intellectual disabilities. Cyberpsychology & Behavior, 8(3), 272–282.

Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of applied behavior analysis, 10(2), 349-367.

Stokes, T. F., & Osnes, P. G. (2016). An operant pursuit of generalization – Republished article. Behavior Therapy, 47, 720-732.

Strickland, D. (1997). Virtual reality for the treatment of autism. Studies in Health Technology and Informatics, 81–86.

Taylor, J. L., & Seltzer, M. M. (2011). Employment and post-secondary educational activities for young adults with autism spectrum disorders during the transition to adulthood. Journal of Autism and Developmental Disorders, 41(5), 566-574.

Trepagnier, C. Y., Olsen, D. E., Boteler, L., & Bell, C. A. (2011). Virtual conversation partner for adults with autism. Cyberpsychology, Behavior, and Social Networking, 14(1-2), 21-27.

Wallace, S., Parsons, S., & Bailey, A. (2017). Self-reported sense of presence and responses to social stimuli by adolescents with autism spectrum disorder in a collaborative virtual reality environment. Journal of Intellectual & Developmental Disability, 42(2), 131–141. https://doi.org/10.3109/13668250.2016.1234032

Wang, X., Laffey, J., Xing, W., Ma, Y., & Stichter, J. (2016). Exploring embodied social presence of youth with autism in 3D collaborative virtual learning environment: A case study. Computers in Human Behavior, 55, 310–321. https://doi.org/10.1016/j.chb.2015.09.006


Wei, X., Wagner, M., Hudson, L., Yu, J. W., & Shattuck, P. (2015). Transition to adulthood: Employment, education, and disengagement in individuals with autism spectrum disorders. Emerging Adulthood, 3(1), 37-45.

Wing, L., & Gould, J. (1979). Severe impairments of social interaction and associated abnormalities in children: Epidemiology and classification. Journal of Autism and Developmental Disorders, 9(1), 11–29. https://doi.org/10.1007/BF01531288

World Economic Forum (2017). Shaping the future implications of digital media for society: Valuing personal data and rebuilding trust. Retrieved May 25, 2018 from http://www3.weforum.org/docs/WEF_End_User_Perspective_on_Digital_Media_Survey_Summary_2017.pdf

Documents

PROMOTING GENERALIZATION USING IMMERSIVE …...PROMOTING GENERALIZATION USING IMMERSIVE TECHNOLOGIES 5 Figure 1. View of 3D CVLE personal pods providing a visual cue indicating where