Usability comparative report - SGSCCgames4competence.eu/wp-content/uploads/2013/04/SGSCC_BOS_… · Web viewHe is using computer every day for playing the games, search the internet,

531134-LLP-1-2012-1-BG-KA3-KA3MP

The SGSCC (Serious Games for Social &

Creativity Competencies)

Usability comparative report

TitleDeliverable No. 5.1. Work package No. WP5

Work Package No. 5Status (D: draft; RD: revised draft; F: final)

F

File Name: Usability comparative report

This project has been funded with support from the European Commission.This publication reflects the views only of the author, and the Commission cannot be held

responsible for any use which may be made of theInformation contained therein.

www.games4competence.eu

http://www.games4competence.eu/

531134-LLP-1-2012-1-BG-KA3-KA3MP

Version History table

Version no. Dates and comments1.0. 06/29/15 Aleksandra

ĐurovićDraft version

1.1. 06/30/15 Bojan Velev Revised version1.2. 07/16/2015 Aleksandra

ĐurovićUpdated draft

1.3.07/18/2015

Aleksandra Đurović

Revised according to received feedback of Manfred Pretis and Karel Van Isacker

1.4. 07/30/2015 David Brown Revised draft1.5. 07/30/2015 Aleksandra

ĐurovićFinal version



531134-LLP-1-2012-1-BG-KA3-KA3MP

1 INTRODUCTION

This report sets out to describe piloting methodology and to provide the analysis of the results of usability and evaluation process. In addition, it will include country-specific findings as well as comparative analysis and general conclusions.

This report includes:

Introduction Description of the piloting methodology Pilot findings Usability Evaluation Schedule of the Desktop Game Case studies General conclusions

2 DESCRIPTION OF THE PILOTING METHODOLOGY

Piloting methodology was outlined in the Usability and evaluation planning document which was accompanied by ten appendices.

Specifically, usability, evaluation and piloting process consisted of two parts. The first part was the analysis of the effects of social competence training as a whole package (educational workshops and educational games (desktop and mobile)) on the development of social competences and creativity. The soft outcome measurement has been carried out using the pre-test - post-test design, namely, through assessment of the difference between ratings of teachers/trainers and self-assessment of trainees before and after the training. Pre-test training has been used to establish a baseline against which the progress could be observed.

The second part of the piloting process consisted of the usability testing of educational games as well as evaluation of handbook/training materials and the quality of educational workshops. Throughout the piloting usability testing has been carried out in order to identify issues that may prevent the users from interacting successfully with the system. In this process the observational method was used together with the post-test session questionnaire which included questions designed to assess the causes of the failure in the games. Some issues and system and user related events were directly entered in an online bug tracker tool Red Mine (http://sgscc.isrg.org.uk), while the others were directly reported to the developers of the games. The observed issues have been continually used to upgrade the games, so the feedback has been incorporated in the final versions.

In the following section evaluation documents that have been developed or adapted in order to carry out piloting process will be presented.

Three questionnaires have been developed for recording the profile of participants and description of the training (and they were translated in the local languages):

1.1.Background questionnaire (Appendix 1);1.2.Form for following implementation of social competence training (Appendix

4)1.3.Questionnaire for evaluation of training sessions (Appendix 5)



http://sgscc.isrg.org.uk/

531134-LLP-1-2012-1-BG-KA3-KA3MP

Two questionnaires have been developed in order to estimate the effects of training on development of social competences and creativity (and they were translated in the local languages):

2.1. Questionnaire for self-assessment of social competence and creativity (Appendix 2)

2.2. Questionnaire for evaluation of social competence and creativity (Appendix 3)

Three questionnaires have been developed for evaluation of SGSCC outputs (learning and gaming material) (and they were translated in the local languages):

3.1 Questionnaire for evaluation of the handbook and training materials (Appendix 6);

3.2. Post-test questionnaire, to be used after games testing (Appendix 7)3.3. Engagement ladder concerning evaluation of the desktop and mobile games

(Appendix 10).

In addition, Guidelines for observers (Appendix 8) and Recording consent form (Appendix 9) were provided, both translated in the local languages. They have been created as supplementary documents to ensure that the applied methodology follows the ethical guidelines and that the piloting process has been carried out under standardised conditions on each of the pilot sites.

All collected data could subsequently be entered in the Data Entry Form (excel).

2.1 Description of the instruments

2.1.1. Background questionnaire (Appendix 1)

The Background questionnaire consists of questions regarding the frequency of using email and social networks, learning and playing games on the Internet as well as questions regarding possession of various devices (desktop computer, smartphone…). Conclusions regarding digital competency of the trainees can be indirectly made on the basis of the answers received from this questionnaire.

2.1.2. Form for following implementation of social competence training (Appendix 4)

This form contains a table with colons reserved for the following information: date & place, a number of the exercise in the Handbook, a number of participants present and comments and evaluation analysis. The last section – Comments and evaluation analysis is partially based on the trainees’ answers on the Questionnaire for evaluation of training sessions, however, it also includes additional relevant information (such as informal feedback or observed nonverbal expression of the trainees).

2.1.3. Questionnaire for evaluation of training sessions (Appendix 5)

This questionnaire consists of four questions, aimed as a tool for the facilitation of the process of getting trainees’ feedback. It is created for group administration. Trainees answer how they like the exercises, their difficulty and usefulness.



531134-LLP-1-2012-1-BG-KA3-KA3MP

2.1.4. Questionnaire for self-assessment of social competence and creativity (Appendix 2)1

This questionnaire consists of 10 questions and covers indicators of social competences and creativity. In order to adapt the instrument for trainees from disadvantaged groups, responses are designed as dichotomous, either “good” and “not good” or “easy” and “hard”. Three questions assess communication skills, two questions measure self-control while the conflict resolution, self-esteem, assertiveness and cooperation are assessed by only one item. The tenth question measures creativity.

2.1.5. Questionnaire for evaluation of social competence and creativity (Appendix 3)

This instrument consists of 37 items on a 5-point Likert response scale. It is designed for collecting ratings of teachers/trainers of the trainees regarding social competences and creativity. Answers ranged from “never” to “always”. Moreover, participants have had the option to choose the response “cannot assess”, which was needed in order to evaluate which aspects of social competence cannot be measured by the assessment of teachers/trainers.

Communication skills are assessed by ten items (for instance “Nods/smiles/says “Yes” to show he/she is listening”). Five items measure cooperation skills, such as “Attends to instruction”. Self-control is assessed by four items (e.g. Persists in tasks which require effort”). Conflict-resolution is measured by six items, for example, “Reacts aggressively when he/she is in a conflict situation”. Assertiveness is assessed by seven items, such as “Tells you when treated unfairly”. Finally, three items measure self-esteem (e.g. “Expects to succeed on the task he/she just started doing”). The additional two questions measure creativity. Four items are reversed (three in a category of assertiveness and one in the category of conflict resolution).

2.1.6. Questionnaire for evaluation of the handbook and training materials (Appendix 6)

This questionnaire consists of 10 items on a 5-point Likert response scale (ranging from “strongly disagree” to “strongly agree”). It includes questions regarding the coherency and organisation of the material and suitability for the work with trainees. Moreover, there is space for additional comments of the trainers. 2.1.7. Post-test questionnaire, to be used after games testing (Appendix 7)

This instrument consists of 27 items. Pictorial scale with five smiley faces, which correspond to a 5-point Likert response scale, was used. The items are divided into four groups: items regarding the engagement qualities of the games (6), questions regarding games’ usability (3), questions regarding the game’s manageability (8) and questions regarding the graphics (10).

2.1.8. Engagement ladder (Appendix 10)

1 Further evaluation of the characteristics of the instruments in Appendix 2 and 3 has been carried out within paper “Assessment of social competences” presented on the Interdisciplinary “Social Competences, Creativity and Wellbeing” Symposium held in June 2015 in Plovdiv, Bulgaria.



531134-LLP-1-2012-1-BG-KA3-KA3MP

This instrument is adapted from the existing instrument2; changes were mostly in the scale being used and the instructions for completion. Engagement, as defined by the authors of the instrument, is multi-dimensional, and encompasses awareness, curiosity, investigation, discovery, anticipation, persistence and initiation. This instrument is in a form of a ladder, and observer writes the name of each participant in the part of the Engagement ladder which corresponds to the level of his/her engagement.

The following scale was used: 4 - Fully Engaged: Completely engaged; 3 - Mostly engaged: Engagement occurring the majority of the time; 2 - Partly engaged: Emerging engagement but unpredictable; 1 - Emerging & Fleeting: Low & minimal levels of engagement, some evidence of awareness and 0 - No Focus: Inattentive and unresponsive.

2.2 Timeline of the piloting and evaluation activities

The chronological steps in evaluation of the social competence training were outlined in the Usability and evaluation planning document (diagram is in the appendix 1).

2.3 Data analysis

In order to ensure that the collected data is informative and comparable, the Excel file for entering data from above-mentioned instruments has been developed. In the data analysis Excel and SPSS 20 have been used.

2 Carpenter, B., Egerton, J., Brooks, T.,Cockbill, B.,Fotheringham, J. & Rawson, H. (2011). Research Project Report. http://complexld.ssatrust.org.uk/project-resources.html



http://complexld.ssatrust.org.uk/project-resources.html

531134-LLP-1-2012-1-BG-KA3-KA3MP

3 PILOT FINDINGS3.1 Description of the Pilot Sites

Within this project, seven pilot studies have been conducted in order to carry out usability testing and evaluation. Generally, the piloting was organised in the period from December 2014 to July 2015. In terms of the number of trainees/testers it was planned in the project proposal that there would be 15 testers from every country and 20 from Bulgaria. It can be noted that this number has been surpassed in some instances.

In total, 304 trainees and 67 trainers participated in the piloting. Depending on the pilot site, training sessions were completed weekly, in every two weeks or monthly. The number of training sessions ranged from 4 to 30 (Table 1). Several different organisations from seven countries were included in the piloting process (Austria, Belgium, Bulgaria, Lithuania, Serbia, Turkey and UK) as well as trainers from various backgrounds and trainees with different types of disability and from various disadvantaged groups. The training materials and the process of training were adapted according to the needs of participants. The overview of the number of trainees and trainers in different countries as well as the number of training sessions can be found in the table 1.

Austria Belgium Bulgaria Lithuania Serbia Turkey UKNumber of

trainees 19 17 123 14 78 32 21

Number of trainers 2 6 16 4 14 14 11

Number of training sessions

4 15 20-30 15 7-13 13 5

Table 1: The number of participants and training sessions on different pilot sites

The pilot involved trainees from different age groups, however, the majority of trainees belonged to the age group 16-20 (56%) (Figure1). The trainees of 21-30 comprised 17.1% of the whole group and the age group 31-40 composed 18.1% of the trainees who participated in the piloting. On the other hand, trainees who were younger than 16 were less represented in the piloting (4.9%) as well as trainees in the age group 41-50 (3.6%). The smallest number of trainees were older than 50 (0.3%).

Figure 1: Trainees’ age group



531134-LLP-1-2012-1-BG-KA3-KA3MP

In the piloting sample, there were more female participants (54%), however the difference in numbers of female and male participants is relatively small (Figure 2).

Figure 2: Gender of trainees

The pilot involved people with disabilities, which constituted the majority of the trainees’ group (70%) (Figure3)3. The main target group were people with learning disabilities and they did compromise the majority of the sample. However, some of them have complex disabilities where the main disability is learning disability but it is also combined some other type of impairment. Also, people with Autism spectrum disorder, mild and severe mental disorder, visual and hearing impairment and mental health problems were also included in the piloting. Apart from the people with learning disabilities, the people from other disadvantaged groups were also involved in the piloting, including students from families from economically deprived areas, one parent families and families with a history of domestic violence, minority groups and youngsters with deviant behaviour.

Figure 3: The type of disadvantaged group of the trainees

3



531134-LLP-1-2012-1-BG-KA3-KA3MP

It should be also noted that the trainees predominately had some experience in playing games. In fact, 78.8% of 250 participants who tested the games had previously engaged with them.

Moreover, the trainers who were involved in the piloting were predominately specialised in working with people with disabilities and other disadvantaged groups.

After the general description of pilot sites, in the following section the more detailed description of participants and the process of training for each pilot site will be provided. For each pilot site, firstly the profile of participants (trainees and trainers) will be described and then the process of training will be outlined. It should be noted that all trainers received their training before the start of the social competence training.

PILOT SITE - AUSTRIA

PROFILE OF PARTICIPANTS

The profile of trainees

The pilot involved 19 trainees. Eleven trainees belonged to the age group 16-20 and eight trainees belonged to the age group 21-30. Moreover, ten trainees were male while nine of them were female, which leads to the conclusion that there was an equal participation between male and female trainees.

All trainees have disabilities and in this respect they represented a heterogeneous group. Specifically, their disabilities range from mild learning difficulty, a mental health problem to autism spectrum disorder.

The majority of the trainees who participated in the testing of the games have a desktop computer or a laptop (79%), which is not the case with a smart mobile phone or a tablet (68.4%). Furthermore, the majority of trainees have had some experience in playing games (74%) while 42% play them daily. Taking this into account, it can be concluded that ICT skills of the trainees and specifically their skills related to playing games, are at the level which enables them to engage with the games in a productive way.

The profile of trainers

The pilot involved two trainers. One trainer was specialised in working with people with disabilities (people with combined developmental disorders and hyperkinetic disorders, delay of typical development and genetic disorders) while the other trainer was not specialised in this type of work.

DESCRIPTION OF TRAINING

There were four training sessions in total, divided into weekly sessions. The first training session included information about the project, exercises regarding the identification of emotions and evaluation and reflection. The second session entailed repetition of the lesson learnt on the previous session, exercises regarding body language and listening, playing the mobile game and evaluation and reflection. The third session included repetition, exercises regarding the initiation of conversation, using illustrated handbook and desktop game and evaluation and reflection. The last session was mainly dedicated to repetition and discussion of open issues.



531134-LLP-1-2012-1-BG-KA3-KA3MP

PILOT SITE – BELGIUM



Seventeen trainees participated in the pilot. Nine trainees were male (52.9%) while eight trainees were female (47.1%). With reference to the age of trainees, it should be noted that the majority of them belong to the age group 16-20 (41.2%) and 21-30 (35.3%) whereas the minority of them belong to the age group 31-40 (5.88%), 41-50(11.8%) or 50 and higher (5.88%). All trainees have learning disabilities with complex disabilities (the main disability is learning disability but is some cases it is also combined with visual, motor or some other type of impairment).

While the majority of them have a desktop computer or laptop (82.35%), it is not the case with a mobile phone or tablet (47.06%). In addition, the large percent of them play games (76.47%). All trainees participated in the online training, which enabled them to engage with the games adequately.


There were 6 trainers, among which 50% were female. They have backgrounds in psychology, sociology and they had worked with people with disabilities before.


The training sessions were held between December 2015 and May 2015. There was an induction training with trainees which was followed by 14 training sessions in total. They lasted approximately 3 to 4 hours split in 45 minutes units. During this training, all exercises from the Handbook were completed. Specifically, mobile games were played by all trainees in the last four training sessions, whereas desktop games were played by 4 people in the last two training sessions. Moreover, additional materials such as videos were used throughout the training.

PILOT SITE – BULGARIA



In the pilot 123 trainees participated from four cities: Plovdiv, Smolyan, Sliven & Syedinenie. The trainees were heterogeneous regarding age: 23.6% belonged to age group 16 -20, 29.3% to age group 21-30, 39.8% to age group 31-40 and 7.31% to age group 41-50. There were more female trainees (63.4%) than male trainees (36.6%).

The majority of trainees were trainees with disabilities (90.2%) whereas 12 of them (9.8%) were from disadvantaged group (youngsters with deviant behaviour). Specifically, the difficulties that the trainees have are mild learning difficulties, severe learning difficulties, hearing impairment and mobility impairment.



531134-LLP-1-2012-1-BG-KA3-KA3MP

The large percent of trainees have a desktop computer (79%). In contrast, only 39% have a smartphone. However, it is important to note that the majority of trainees (73%) have at least some experience with games. The 25% of trainees play games daily.


The pilot involved sixteen trainers: psychologists, social workers, adult trainers and teachers. All trainers were specialised in working with people with disabilities (mild learning difficulties, severe learning difficulties, people with hearing impairment, and people with mobility impairment and youngsters with deviant behaviour).


The training sessions were held between January 2015 and June 2015. There were in average between 20 and 30 individual training sessions between trainees and professionals who were working with them. The content of the training sessions was adjusted to the individual needs of the trainees. The training included exercises from the handbook at list two times per week (20-30 minutes) and gaming at least three times per week.

PILOT SITE – LITHUANIA



The piloting process involved 14 trainees, among which 35.7% were female and 64.3% were male. The age span of the trainees ranged from 13 to 17. It should be noted that 12 trainees were younger than 16. All of them had disabilities, namely mild and severe mental disorders, Autistic Spectrum Disorder and kinaesthetic disorders.

The half of the trainees has desktop computer or laptop, whereas only 21% have mobile phone or tablet. However the majority of them play games 86%. In fact, 79% of trainees play games every day.


There were four trainers, specialised in working with people with disabilities, specifically people with mild and severe mental disorders, Autistic Spectrum Disorder and kinaesthetic disorders.


The training consisted of eight training sessions in total, which were carried out in the period of May to June 2015. All training sessions were devoted to fostering communication skills. Namely, the development of the skills of recognising and expressing emotions, expressing trust and initiating and sustaining conversations was in the focus. Moreover, participants had four game sessions as well, where they have played the desktop and the mobile game. The games were played separately - in 7 sessions: three for PC games, three for mobile game and one session to discuss with children about the games and let them choose randomly to play what they choose.



531134-LLP-1-2012-1-BG-KA3-KA3MP

PILOT SITE – SERBIA



The pilot involved 78 trainees who were recruited from seven different secondary vocational schools in Serbia. Three schools were located in Belgrade while four schools were located in Veliko Gradište, Čačak, Subotica and Vrnjačka Banja.

All trainees belonged to the age group 16-20. Namely, all participants were recruited from the first and the second grade of school, which means that they were 16 at the time of the piloting or turning 16 at some point during this year. From this group, 57.7% were female and 42.3% were male, which leads to the conclusion that there was a fairly equal participation between male and female trainees.

The majority of the trainees who participated in the testing of the games have a desktop computer or a laptop (91.2%), which is not the case with a smart mobile phone or a tablet (47%). Moreover, the majority of trainees have had some experience in playing games (88%) while 54.6% play them daily. Taking this into account, it can be concluded that ICT skills of the trainees and specifically their skills related to playing games, are at the level which enables them to engage with the games in a productive way.

All trainees were chosen based on the estimation of their teachers or school psychologist that they need support for the development of social competences and creativity. The majority of them belonged to the disadvantaged groups (families from economically deprived areas, one parent families, Roma, families with a history of domestic violence).


There were 14 trainers in the main. In every school, there was at least one psychologist or pedagogue, specialised in working with students from disadvantaged groups during their formal education. Teams of trainers from schools included teachers as well. The majority of trainers were female (86%). It was assessed that the trainers need support regarding technical aspects of the training due to the lower level of ICT competence. Therefore, they were assisted by the IT teachers.


So as to respond to a wide range of trainees’ needs, the teams in every school had to adapt the training material which they used in the training sessions. The minimum number of workshops based on the exercises from the Handbook was 7, and the maximum number was 9. The specific content of the workshops varied from one school to another; however, it typically included at least one exercise for each of six targeted groups of social competencies. Unfortunately, due to technical difficulties, two schools, i.e. 36 trainees, did not test the games. Five schools that did test the games implemented 1 to 4 game sessions.



531134-LLP-1-2012-1-BG-KA3-KA3MP

PILOT SITE – TURKEY



Thirty two trainees participated in the training. All trainees belong to the age group 16-20. There was the same number of male and female participants: 50% were male and 50% were female. As for their profile, all of them have mild learning difficulties.

Less than half of the trainees have desktop computer or laptop (40%) similarly as in case of mobile phone or tablet (44%). The majority of the trainees play games (78%) and most of them play them daily (72%).


There were 14 trainers. All of them were specialised in working with people who have mild learning difficulties.


The training was organised from December 2014 until March 2015. It was divided into nine training sessions in total which covered exercises for communication skills, self-control, conflict resolution and assertiveness. Every training session included between one and four exercises. Moreover, four game sessions were implemented as well.

PILOT SITE – UK



There were two groups of trainees who participated in the piloting. The nine trainees from the first group were pupils or ex-pupils of the City co-educational day school for children aged 3- 19 years with special educational needs (Oak Field School), in particular severe and profound learning difficulties and/or physical difficulties. In this group, six of them were male and three of them were female. Moreover, two of them belonged to the age group 16-20, two of them to age group 21-30, whereas the rest of them belonged to age group 31-40. They all had disabilities, including Down’s syndrome, William’s Syndrome and Non-specific severe learning and communication disabilities,

The second group of trainees were pupils of the specialist day school for pupils and students with autism (Sutherland House School). The eleven of them were male, while the only one trainee was female. All of them belonged to the age group 16-20. Participants had Autism Spectrum Disorder or Pathological Demand Avoidance. The majority of participants had a desktop computer or laptop (92%), and the large percentage of them had a mobile phone or tablet (83%). Furthermore, they had extensive experience in playing games: all of them played games and 75% of them played them daily.


With the first piloting group (Oak Field), there were 10 trainers (one head teacher, one teaching assistant, two pedagogy experts, one carer, one social inclusion expert and four



531134-LLP-1-2012-1-BG-KA3-KA3MP

technology experts/developers). In the second piloting session, 2 trainers were involved (one teacher and one teaching assistant).


Five piloting sessions were organised at two different schools.. The sessions piloted the Handbook, mobile game and desktop game. All sections of the Handbook, Mobile Game and Desktop Game were tested.

The testing sessions varied in numbers, but there was a core group of testers who attended each of the repeated sessions (for Oak Field = 4 repeated sessions). Co-discovery pairs were used at Oak Field with more able students supporting the exploration of the handbook and games with less able students, and this approach worked well in a social competences context.

3.2 The development of social competences and creativity

3.2.1 Evaluation of social competences before and after the training

In the following section, results will be presented using diagrams that show average ratings before the training and after the training4. Moreover, the areas where the largest progress has been observed will be highlighted.

PILOT SITE – AUSTRIA

Due to unforeseen changes in the group of trainees during the training, ratings regarding social competences and creativity were collected for only six trainees before and after the training. As the training encompassed only exercises regarding communication skills, the difference in average ratings before and after the training are shown only for this subgroup of social competence. It can be observed on the diagram below (Figure 5) that there was some progress in this area after the training.

Figure 5: Progress in communication skills – Austria4 In the average ratings all items were included except four reversed items, 3.item in subgroup Assertiveness and 4.item in subgroup Self-control which did not have good metric characteristics.



531134-LLP-1-2012-1-BG-KA3-KA3MP


As it can be observed on the diagram below (Figure 6), there was a progress in the area of all social competences and creativity. The largest progress has been observed in the area of self-esteem. On the other hand, the lowest progress has been observed in the area of assertiveness.

Figure 6: Progress in social competences and creativity - Belgium


For every targeted social competence as well as for creativity, the major improvements and challenges were registered.

Communication skills

In the area of communication skills the improvements that have been observed are maintaining eye contact with the trainer and other students, sharing facts from his/hers life, initiation of conversations and expression of a trust in relationship with others. As for the challenges it should be noted that the trainees frequently ask clarifying questions and interrupt their trainers. Also they still find it difficult to share feelings/emotions.

Cooperation

As for the cooperation, improvements that are noted are in the area of participation in collaborative problem solving and ability to understand how his/her friends feel when they are angry, upset or sad. The trainees still find it challenging to express gratitude when it’s appropriate, to recognize achievement and merits of others and to attend to instructions.

Problem solving

After the training, it has been observed that the trainees show more interest towards the needs of another person, better explain their own needs and show more respect to another person. They still find it challenging to agree to try partner's solution and they still tend to insist on their solution.



531134-LLP-1-2012-1-BG-KA3-KA3MP

Self-esteem

As for the self-esteem, the improvements are that the trainees now tend to show that they feel happy about themselves and they expect to succeed on the task they just started. At the same time they find it somewhat difficult to compare with those who are similar to them and to recognise their own strengths.

Self-control

Following on from training, the trainees have shown better resistance on temptations/instant gratifications when solving task, whereas they still find it challenging to keep their anxiety/fear under control, and to use strategies to improve self-control and control the anger.

Assertiveness

In the area of assertiveness the improvements that have been observed are that the trainees are less aggressive when their needs are not satisfied, they tell to others when they are treated unfairly and react actively when their needs are not satisfied. On the other hand, the trainees still find it challenging to deal with a pressure from others in an assertive way.

Creativity

After the training it has been observed that the trainees enjoy doing creative activities more and show positive emotions when they have participated in creative activities. They still find it somewhat difficult to suggest creative ideas during problem - solving process and to express themselves by different means.


As indicated by the change in ratings before and after the training, there was a change in some social competences and creativity. Namely, there was a relatively small improvement in the self-esteem and creativity, but not in the other rated social competences.

Figure 7: Progress in social competences and creativity – Lithuania



531134-LLP-1-2012-1-BG-KA3-KA3MP

PILOT SITE – SERBIA

The difference between the average ratings of the teachers before and after the training indicates a progress in all target social competences as well as creativity5. The largest progress has been observed in the area of communication skills and creativity.

Figure 8: Progress in social competences and creativity - Serbia

On the other hand, teachers have observed the lowest progress in the area of self-esteem.


As it can be observed on the diagram below (Figure 9), there was a visible progress in the area of all social competences and creativity.

Figure 9: Progress in social competences and creativity - Turkey

5 Data before and after the training was gathered for 63 trainees.



531134-LLP-1-2012-1-BG-KA3-KA3MP

The largest progress has been observed in the area of communication skills and self-esteem, while the lowest progress has been reported in the area of cooperation skills.

PILOT SITE – UK

Trainer’s help was used to interpret the change in social competences over the course of piloting sessions as a result of using the Handbook and Games. The following improvements and points of interest were noted:

Communication Skills: the importance of maintaining eye contact was acknowledged by the students, and the games allowed the students to talk about the problems they experience in initiating a conversation, and trainers noted this is the first step in improving this skill. Some of the students clearly showed empathy for each other (especially in the use of the mobile game), and were sharing facts from their lives – especially difficult situations involving self-control. It was noted that the games and handbook were excellent in creating a ‘space’ for these conversations to happen, where the media provided a forum for the types of conversations concerning social competences that don’t happen elsewhere.

Cooperation: there was clear evidence of the development of cooperation skills. The students were clearly cooperating in collaborative problem solving – and this happened between students pairs, and between trainer-students pairs. The later was interesting in that the students clearly assisted the trainers on several occasions to solve problems, and suggest strategies to navigate through the desktop game (e.g., suggesting the player and non-player character stand on different objects to initiate interactions.) The trainer noted students clearly attending to instructions in the desktop game.

Self-Esteem: the students consistently acknowledged they felt happy about themselves. The trainers also noted improved abilities of students to recognize their own strengths.

Self-Control: Big improvements were shown in this competency. One of the students in the Oak Field testing session has major problems with self-control (over-eating, stealing). The Games and Handbook have helped her to further understand wrong choices, avoiding temptation, and be able to control temptation. The piloting sessions also generated conversations around strategies for self-control. There was also lots of evidence of persistence in tasks which require effort (such as in the Whispering Statues task in the Desktop Game).

Creativity: The use of the Games and the Handbook allowed the students to talk about how creativity makes them feel, and its importance to them. The piloting sessions initiated conversations around poetry and dance and the improvement in well-being these activities make.

COMPARATIVE ANALYSIS AND CONSOLIDATED FINDINGS

The obtained results imply that there was a progress in the area of social competences and creativity during the training, in the view of teachers/parents/trainers who assessed the trainees. Generally there was an improvement in the area of all targeted competences; however the results differ across pilot sites. Given that the content of the training, number and profile of trainees varied significantly, it is somewhat difficult to compare the findings.

However, it can be noted that there is some consistency in the obtained results. Figure 10 shows that on a scale from 1 to 5, the change in the average ratings of six social



531134-LLP-1-2012-1-BG-KA3-KA3MP

competences and creativity ranged from 0 and 0.1 to 1.03. The biggest progress was observed in the area of creativity, self-esteem and communication skills.

Figure 10: The observed change in ratings of social competences and creativity before and

after the training across pilot sites

The results are in good agreement with the SWOT analysis carried out within the project6 which found that opportunities that may be associated with using educational games are increase in creativity and self-esteem. As the use of educational games was in common for all pilot sites, this can be one possible explanation for the obtained results. Moreover, these three competences might have been developed even within those training sessions in which development of other competences has been a goal. In fact, it was planned before this training to foster creativity through training sessions which target some other competence7.

3.2.2 Self-assessment of social competences before and after the training

At the beginning of training, the analysis was conducted to determine what was seen as difficult for the majority of participants. Therefore, the analysis was carried out to assess

6Pretis, M. (2013). Consolidated Research Need Analysis Report. http://games4competence.eu/wp-content/uploads/2013/02/Consolidated-report.pdf 7 Stamatov, R. & Sariyska, S. (2014). Social competence and creativity guide. http://games4competence.eu/



http://games4competence.eu/

http://games4competence.eu/wp-content/uploads/2013/02/Consolidated-report.pdf


531134-LLP-1-2012-1-BG-KA3-KA3MP

whether there was a change in the self-assessment of those social competences which the majority of trainees (more than 60%) defined as their week points.


The main goal of piloting in this pilot site was the development of communication skills. Therefore, the self-assessment of trainees in this area is in the focus.

The results are:

67% - feel more able to listen to their classmates 33.3% - feel more able to initiate conversation 50% - feel more able to recognise emotions of other people

In general, these findings indicate that there was a progress in the area of communication skills in the view of trainees. The largest improvement was in listening skills while the minority of trainees perceive difference regarding initiating conversation. However, it should be noted that even in this aspect there was a progress for some trainees, as evident in the case study.


Before the start of the training, more than 60% of the trainees assessed communication skills (initiating conversation and recognising emotions) and self-control (regulation of emotions) as the areas in which they are experiencing difficulties. On the contrary, according to their answers creativity and self-esteem do not present difficulties for the majority of trainees.

The findings are as follows:

54.5% - feel more able to initiate conversation 72.7% - feel more able to recognise emotions of other people 54.5 % - believe that they are more able to calm down when they are angry or

upset

It can be noted that for all competencies that trainees themselves perceived as hard before the training, more than a half of them observed changes after the end of the training. The progress is the most evident in the case of recognising emotions, seeing that more than 70% of participants experienced progress.


Before the start of the training, the majority of trainees (over 60%) recognised that they have difficulties regarding communication skills (listening and recognising emotions), cooperation skills, self-control (regulating negative emotions and persistence), conflict resolution, assertiveness, and creativity.

The results are:

10% fell more able to listen to their peers 22% believe that they are better in regulating emotions 15% fell more able to calm down when they are angry or upset 8% believe that they are more able to persist (keep doing) tasks that are hard for them



531134-LLP-1-2012-1-BG-KA3-KA3MP

9% believe that they are more able to express feelings when they are wronged 12% feel that they are better in solving conflicts 6% believe that they are better in working with others 3% enjoy participating in creative activities more than they used to.

In general, these findings indicate that, in the view of some participants, there was a progress. However, there are not many trainees that perceived a change. There are several possible explanations. It might be that there was not a visible change from the trainees’ perspective or that there was a change but not the big one. However, this conclusion is not in accordance with the data gathered from trainers’ ratings, who did observe a significant change in several respects after the training had ended.


This pilot site is characteristic in that the majority of participants assessed before the training that they do not encounter difficulties in any of the listed social competences or creativity. In fact, there is not a question for which more than 60% trainees said that they find the explained competence hard. The only aspect which the half of the trainees assessed that it is difficult for them is initiating conversation (communication skills). However, none of the trainees experienced change after the training in this aspect.

PILOT SITE - SERBIA

At the beginning of the training, the trainees assessed initiating conversations (an aspect of communication skills), regulation of emotions, persistence in the face of difficulty (aspects of self-control) and assertiveness as challenging. In contrast, only the minority of trainees perceived conflict resolution, self-esteem, cooperation skills and creativity as hard.

The results are:

56% of those questioned feel that they are more able to initiate conversation than they were before the training

67% believe that they are more able to express feelings when they are wronged 47% feel more able to calm down when they are angry or upset 47% believe that they are more able to persist (keep doing) tasks that are hard

for them.

Overall, the data imply that there was a progress in the area of social competences due to the training in the view of the trainees themselves. The improvement is the most evident regarding assertiveness, i.e. expressing feelings when they are wronged.


In this pilot site, all participants experienced difficulties with all assessed competencies except listening.

The results are:

84.4% believe that they are better in regulating emotions 25% feel that they are better in initiating conversations 65.6% fell more able to calm down when they are angry or upset



531134-LLP-1-2012-1-BG-KA3-KA3MP

53.1% believe that they are more able to persist (keep doing) tasks that are hard for them

53.1% believe that they are more able to express feelings when they are wronged 81.3% feel that they are better in solving conflicts 93.8% feel better about themselves 100% believe that they are better in working with others 93.8% enjoy participating in creative activities more than they used to.

It can be observed that the majority of trainees believe that they have made progress regarding all competencies except initiating conversations.

PILOT SITE – UK

The questionnaire itself initiated some very interesting conversations. Trainers remarked on how students with severe learning disabilities produced insightful and accurate answers to these questions. Changes in the self-assessment of social competences include:

50% reported an improvement in listening to classmates50% improvement in recognising emotions in other people50% improvement in ability to calm down after being angry50% improvement in how students feel about themselves50% improvement at how good students are at working with classmates

Overall the questionnaire, as well as the Handbook and Games, produce a really interesting and useful method for students to reflect upon their social skills, and the problems they face regarding social competences.

COMPARATIVE ANALYSIS AND CONSOLIDATED FINDINGS

In case of the most pilot site there were some competencies for which the majority of trainees observed improvement. More than 50% trainees observed that their communication skills have been developed in three pilot sites (AT, BE, RS), which is the case in self-control in two sites (BE, TR).

It is interesting to note that in some cases the trainees had different perception about their social competences in comparison to the perception of their teachers/trainers. For instance, in one pilot site (BE) self-esteem was not perceived as problematic for the trainees, whereas their trainers have observed a change in this respect. In contrast, in other pilot site (RS) both trainees and trainers noted that there was progress in communications skills.

In general, the participants have not noticed an improvement in creativity and self-esteem, which are precisely the competences in which the other people who assessed them observed the progress. The difference between self-assessment and ratings of other people in terms of social competences has been reported in the previous research studies as well8. However, the self-assessment is vitally important part of the training not only for the

8 Junttila, N., Voeten, M., Kaukiainen, A., & Vauras, M. (2006). Multisource assessment of children's social competence. Educational and Psychological Measurement, 66(5), 874-895.



531134-LLP-1-2012-1-BG-KA3-KA3MP

evaluation process but also because it fosters the self-awareness of the participants and facilitates learning9.

3.3 Evaluation of the learning and gaming materialsIn order to gain insight in the usefulness of learning materials, the evaluation has been

carried out through assessment of the handbook and workshops within which the exercises from handbook have been implemented. As for the gaming material, results of usability testing and evaluation have been reported separately for the desktop and mobile game.

3.3.1 Evaluation of the Handbook - Social competence unit & Appendix Practices and exercises of social competence and creativity

Austria Belgium Bulgaria Lithuania Serbia Turkey UKAverage rating 3,83 4,21 4,49 4,55 4,27 4,55 4,55

The Handbook received positive feedback by the trainers (total number of trainers who assessed the handbook was 69). It was generally assessed as clear, understandable and useful for training in social competences and creativity. This conclusion is supported by the quantitative data gathered. The average rating for the handbook across different pilot sites is 4.35 out of 5. The average ratings for all sites separately (depicted in Table 2) ranged from 3.83 to 4.55.

Table 2: The ratings of the handbook across pilot sites

Not only is the whole handbook assessed positively, but also its individual aspects are generally rated with good marks (Figure 11).

Figure 11 – The assessment of handbook across different pilot sites

9 Stamatov, R. & Sariyska, S. (2014). Social competence and creativity guide. http://games4competence.eu/



http://games4competence.eu/wp-content/uploads/2013/02/Appendix-1.Practices-and-excercises-of-social-competence-and-creativity-appendix-of-D.3.1.-final-31.07.2014.docx

http://games4competence.eu/wp-content/uploads/2013/02/Appendix-1.Practices-and-excercises-of-social-competence-and-creativity-appendix-of-D.3.1.-final-31.07.2014.docx

http://games4competence.eu/wp-content/uploads/2013/02/SOCIAL-COMPETENCE-AND-CREATIVITY-ILLUSTRATED-HANDBOOK-FINAL-english.docx

http://games4competence.eu/wp-content/uploads/2013/02/SOCIAL-COMPETENCE-AND-CREATIVITY-ILLUSTRATED-HANDBOOK-FINAL-english.docx

http://games4competence.eu/

531134-LLP-1-2012-1-BG-KA3-KA3MP

It can be observed that there is a high consensus across pilot sites about the quality of handbook in almost all aspects, which implies that the handbook is appropriate in all local languages. There is a difference in the assessment of the suitability of handbook for work with the trainees and especially the difference in rating of the suitability of its difficulty level. However, with regards to these comments it should be pointed out that the exercises from handbook are envisaged as a starting point in training and that trainers should adapt them to match the individual needs of the learners. Moreover, the difficulty level of the exercises can be changed if it is necessary by the trainers, for example by introducing more complex aspects or making materials more accessible.

3.3.2 Evaluation of the workshops

Evaluation of the workshops will be shown separately for different pilot sites, since they differed with regard to the number and content of training sessions. It can be noted that the trainers in all training sites reported high engagement of the trainees and good learning and transfer effects.


The trainers reported the high involvement of the trainees. Specifically, it was highlighted that they were active and satisfied throughout training sessions.

Moreover, it was stressed that the trainees remembered well what they had learnt in the previous session. The retention of the material was accompanied with the good transfer effects.


The majority of participants reported that they liked the exercises. Namely, they highlighted the benefits from using different types of materials (such as videos) and providing explanations throughout the training sessions. They especially stressed motivation of teachers and using examples as important. The exercises were perceived as easy rather than hard, mainly because of the support they have received.

The following feedback is example of the way the training sessions were perceived:

“I usually get angry when someone criticizes me, but now I have a different look on this”.

“Conflicts often occur in the pub, at work or at home, and this session taught us how to deal with it”.

“I feel often isolated, but this session helped me very much to talk to others”.

“Expressing thoughts and feelings are very important, and this lesson thought me nothing is wrong with that”.

Generally, trainees assessed that they expanded their knowledge in the areas that were covered during the training. It should be noted that they recognised the applicability of what they had learnt in the situations at work. Also, they expressed readiness to use the knowledge they gained in their relationships with friends and partners, as evident from the following examples:



531134-LLP-1-2012-1-BG-KA3-KA3MP

“I need this for my work. When new colleagues arrive I like to talk to them in an easier way”.

“When I am being badly treated by colleagues, I should act against that. The same when a colleague is being badly treated”.


In terms of satisfaction 82% (from 123 participants) of the respondents were very happy to participate during sessions. They were engaged to cooperate with others which they really appreciated. The students confirmed that they could transfer what they have learned into the real world. The majority of the participants indicated that they have learned more about themselves in terms of their strengths. They have learned different types of emotions as well as how to share their emotions, how to understand the negative effects of personal behaviour, and how to identify and formulate the conflict situation and make requests.

The trainees liked the most team working while exercising. They also liked a collection of an award while completing a certain challenge during the games and the opportunities to share their stories. Specifically, they liked the most exercises regarding the recognition of emotions.


Generally, the trainees were enthusiastic and eager to learn more about the topics covered in the training. However, the trainers observed that some of the materials were too difficult to them, so they needed additional help and guidance. The topics that they were the most interested in were: understanding different emotions, initiating conversations and expressing acceptance. The following feedback is illustrative for the evaluation of the sessions:

“Students were eager to understand different emotions and they liked to follow instructions even it was not easy for them to grasp all differences. They were focused and challenged to name different emotions and follow the exercise. They stated that it was interesting and they like the lesson”

“Students were eager to initiate conversation, but in most cases, they faced some difficulties to do it in an appropriate way. They liked to follow instructions even it was not easy. At the end of session they could manage to initiate a conversation in a familiar environment (classroom).”

PILOT SITE - SERBIA

The trainers reported that the participants have been immensely interested throughout workshops. The exercises were generally assessed as useful and moderately hard. The following feedback is an example of the organisation of training sessions and the benefits from them in the view of trainers:

“Students have actively participated and tried to apply self-control techniques that they had learned. Several students had already known that they should count to ten and breathe when they are upset; however, majority of them had not used them. After the exercise, they said that they learned how to alleviate negative emotions. “



531134-LLP-1-2012-1-BG-KA3-KA3MP

“They learned that they do not need to slap someone and that conflicts can resolve in a different way too. There is a noticeable difference in the way they behave at class“.


Generally, the trainers observed that the trainees enjoyed the exercises. For instance, they said that it was enjoyable doing activity with mimics and movements without talking. In contrast, it was more challenging for them to do exercise about expressing emotions.

At the same time, they reported that they had learned new and useful things, especially how to ask for help in the face of a problem and how to communicate with strangers who they meet in their daily lives.

PILOT SITE – UK

One of the most interesting observations in the workshops was the high levels of engagement of students in both the sessions at Oak Field and Sutherland House School. Students consistently chose to continue to use the games after the official end of the piloting sessions, and asked when the next session would be. Teachers and carers also reported that they would like to use the Handbook and Games in the future within their lessons and practise.

Another interesting observation was how well students with an intellectual disability, and with reported memory problems, remembered progress and activities of previous piloting sessions, against expectations. We theorise that this is connected to the high levels of engagement (and therefore deep learning) observed in participants during the piloting sessions.

3.4 Usability and evaluation of the desktop game Inca’s Island

The desktop game Inca’s Island has been assessed as challenging and engaging by the trainees across pilot sites (Figure 12)10. Furthermore, the situations depicted in the game have been generally described as realistic and relevant.

Figure 12: Assessment of the engaging qualities of the desktop game10 In total 241 trainees have been questioned about their experience with the desktop game.



531134-LLP-1-2012-1-BG-KA3-KA3MP

With reference to game’s graphics and manageability, Figure 13 shows that the game was mostly assessed as accessible. Namely, the starting and closing of the game were described as easy process and the transition between levels was seen as clear. Moreover, the game’s look was generally described as appropriate and the connection between pictures and actions clear.

Figure 13 – Assessment of the desktop game’s graphics and manageability

In contrast, there is less agreement between trainees about the difficulty of using this game. In some piloting sites, such as Belgium the game was assessed as fairly hard. For instance, the following feedback was gathered from some trainees: “I liked the desktop games, but they are difficult to play alone” and “Three friends and I played the desktop games. We needed a lot of help from the teachers. But we did well”!

However, it should be noted that for some trainees can be highly motivating to engage with the game precisely because it is challenging and somewhat difficult. In fact, in most of the piloting sites, the trainers rated trainees’ engagement when they were playing desktop game as fairly high. In two pilot sites (UK and RS), it has been observed that participants do not want to stop playing after they were offered the choice to stop as the session ended. Also, there are many positive experiences with this game, as illustrated with the following two examples from UK:

“P3 appeared to thoroughly enjoy the chance to play the game and listened very well to advice from the researcher about how to progress with it. He shared his views with other classmates and asked them questions about their game“.

“P4 seemed to be very engaged with playing the game and was keen to express his positive thoughts of it. He confidently asked questions when unsure and enjoyed sharing with the group his personal experiences of gaming“.

The positive and challenging aspect of the game can be described as follows:

Positive aspects of the game:

The look of the game – It was described that it looks as “real computer game“, which was seen as great advantage.



531134-LLP-1-2012-1-BG-KA3-KA3MP

Motivating feedback – Gaining awards in the game, which served as a feedback for trainees, was greatly appreciated and proved to be immensely motivating. „The trainers have reported that the students who tested the desktop game were very enthusiastic to overcome each challenge in order to receive an award – the diamond after each room”.

Fostering cooperation – In the view of participants, the benefit of using the game was facilitation of peer contact and cooperation. “The scenario is based on cooperation with the companion which was appreciated by both trainers and students. The feeling that the students are not alone while finding a solution of a challenge was very motivated for them”. This was also seen as a potential strength of the games in the SWOT analysis that has been carried out before the start of piloting11.

Challenging aspects:

Level of difficulty – As it has been reported, the game was seen as difficult by some trainees. Based on the experience in piloting it can be suggested that the game can be made easier if it is necessary by using audio instead of written text and adapting the language that is being used. Moreover, trainers’ handbook can be useful for planning adaptation of the game so that it corresponds to individual needs of learners.

Maintaining self-control – Moreover, it has been reported that playing this game can sometimes be frustrating for learners, especially if they find it difficult. One suggestion that can be made based on piloting process is that exercises for expressing emotions as well as rewards can improve the self-control of learners. The following example is a case in point. “Sometimes they have problems with self-control in order to complete the games’ aims. Due to the fact that the institutions where the piloting was implemented have limited number of devices sometimes the students was “fighting” with each other that will play. In that case the red cube with the smileys which has been produced was very useful. Each student rolled the cube and the one who achieved the happy smiley should first play the game”.

3.5 Usability and evaluation of the mobile game Inca’s Quest

The mobile game was perceived as engaging and relatively easy (Figure 1412).

Figure 14: Assessment of the engaging qualities of the mobile game

11 Pretis, M. (2013). Consolidated research need analysis report. http://games4competence.eu/wp-content/uploads/2013/02/Consolidated-report.pdf12 In total 216 trainees have been questioned about their experience with the mobile game.




531134-LLP-1-2012-1-BG-KA3-KA3MP

Moreover, not only was the game assessed as engaging by trainees themselves, but also trainers noted that the participants were thoroughly engrossed in the game. Similarly as in case of desktop game, the situations depicted in the game were perceived as relevant and realistic.

Graphics of the game and its manageability were also assessed positively, according to data presented in Figure 15.

Figure 15 – Assessment of the desktop game’s graphics and manageability

Using the game was reported to be easy and it was generally said that the time left for correcting responses was adequate.

The positive and challenging aspect of the game can be described as follows:

Positive aspects of the game:

Interactive nature of the game: It was reported that the game was very entertaining and engaging, which is illustrated by the following feedback: “The mobile game was fun” and “Playing on a smartphone, I never did that before”!

Look of the game – The game’s graphics were said to be of great quality. For instances, it was said that: “This game was considered polished and intuitive” and “the graphics are very good”.

Easy to use –It has been noted that the game is very easy to use, which is supported by the quantitative data mentioned above.

Adequate feedback - Feedback given in the game was assessed as rewarding, which implies that it can be motivating for learners. This can be illustrated by the following quotes: “It creates a positive feeling to see the green thumbs” and “It has good and clear reward system”.

Challenging aspects:

Language difficulties – Especially in the earlier stages of game testing, it has been remarked that some formulations need paraphrasing. The corresponding changes have



531134-LLP-1-2012-1-BG-KA3-KA3MP

been made accordingly. For example, it was reported that: “Some students with severe intellectual difficulties could not understand the meaning of the double negation of some questions from the 2nd mobile game. That’s why the Bulgarian team has implemented changes in the latest version with only positive questions”.

Clarity of instructions – Trainers have observed that the instructions sometimes need elaborating. In this purpose, trainers’ manual which has been developed within the project can be used.

4 Usability Evaluation of the Desktop Game

4.1 Summary

In addition to Piloting, Usability Evaluation activities ran in parallel for the desktop game, as an extensive exercise of conducting tests, collating, prioritising and closing down usability issues reported by all partners. This section reports on the range and depth of this process.

Between November 2014 and the end of the project (30th June 2015), NTU (UK) released 17 iterations of the Inca Island game. Over this period, feedback from our own usability evaluation activities and partners’ usability evaluation reports were used to directly iterate the game development (usability, pedagogy, localised content etc). This has resulted in 423 commits to Github (version control system implemented for the SGSCC project by NTU) which comprised 15847 file changes, and 151044 changed lines of code. An issue tracking system was put in place, though this system was little used, and issue reporting fell back largely to email reports. The following summarises the release history, changes and developments made, ISRG’s Advisory board testing, and partner feedback for each release to demonstrate the breadth of usability evaluation testing, iteration and feedback carried out by NTU.

4.2 v0.1.7 (Initial release)Released: 2014-11-12 (261 MB)

Game engine & 3 rooms released YouTube video walkthrough

Known issues

Languages not in place Speech not present The 3 example rooms have no activities Map not implemented Admin dialog displays on first run

4.3 ISRG Testing– 2014-11-13 (v0.1.7)NICER Group meeting: Game should start in conversation with Companion, who can give some initial guidance; splash screen needs animation – appears frozen; users want to play with actual content in the game



http://sgscc.isrg.org.uk/projects/desktop-game/wiki/V017

531134-LLP-1-2012-1-BG-KA3-KA3MP

4.4 ISRG Testing – 2015-01-14 (v0.1.8 pre-release)NICER Group meeting: System dialog appears on first run; game should start with conversation with Companion NPC; Player surrounded by white box sometimes.

4.5 v0.1.8 Released: 2015-01-18 (259 MB)

Rooms 4, 5 & 6 (Cooperation) functional

No. of commits: 91; No. of file changes: 2272; Changed lines: 40555(+) 29287(-

Partner Issue Reports:

RS: Various UI issues; Room 4 text content problem; web browser started unexpectedly; game sometimes freezes on pressing continue; Player avatar should be able to run.

4.6 v0.2.0 Released 2015-02-03

Rooms 10, 11, 12, (Self Esteem) & 16, 17, 18 (Assertiveness) added Paths released as 3 separate packages: Self Esteem (276 MB); Assertiveness (276

MB); Cooperation (259 MB) (already released)

No. of commits: 21; No. of file changes: 1300; Changed lines: 91516(+) 14182(-)

Partner Issue Reports:DE: Problems getting statues to talk; Game “went black”; confusion over activity indicator figures; text hard to read and some upside down text in Room of Puzzles; game crashed – report generated automatically and sent to developers.

4.7 v0.2.8 Released: 2015-02-23 (316MB)

Updated Self Esteem & Assertiveness topics reflecting feedback


Partner Issue Reports:BG: Self-esteem room - how to escape while the border is coming; room with Non-assertive messages - diamond not revealed; could only reach Self-esteem and Assertiveness; map does not providing contextual help; modify the companion avatar – permanently open mouth is not right.

NL: - start screen: replace "België" should with "Nederlands" & change flag; the English flag is strange; movement with arrow keys is not indicated in the settings; no spoken audio; statue at entrance to temple gets stuck in a loop, only one possible answer, took 4 tries to continue; mouse click action is different to space bar; cloud particles setting is wrong; water in 2nd room traps player; no indicators or help how to solve the room with the blocks; some text needs spelling and grammar checking; sometimes not all answers are shown with lower res screen size - user is blocked; room with music blocks - text can hardly be read on block; blocked since the 2nd person keeps on giving advice in a loop, too difficult....






531134-LLP-1-2012-1-BG-KA3-KA3MP

4.8 v0.3.3 Released: 2015-02-25 (348MB)

Conversation text & separate conversation xml editor


Partner Issue Reports:DE: Does not run on 32 bit laptop

4.9 v0.3.5 Released: 2015-02-25 (347MB)

Exact clone of 0.3.3, built specifically for x86 computers.


Partner Issue Reports:DE: Very slow on test laptop; Landing statue doesn’t talk, so the first door does not open; using the pads is difficult (especially with a slow computer, the system does not give feedback and the player does not really know when he/she is on the pad; basic German version needed quickly; Room 4 -difficult to read the text on the floor; audio is confusing - you “hear” the opening of a door, but there is no door.

4.10 v0.3.6 Released: 2015-03-02 (347MB)

Removal of cloud particles option from settings Updated collision for a couple of the rooms so that you cannot enter the water. Provision for conversation audio playback (exemplar content in each language version

of4 English audio files for the first conversation) Updated NL and EN flags Local modifiable Instructions GUI Inclusion of lever model in some of the room interactions.


Partner Issue Reports:DE: Sometimes statues automatically talk, at other times, a “talk” symbol appears and you have to press space; M3, selecting paths it is confusing, 1 guardian talks to you, it appears that the path has been selected, but you must talk to the guardian, to select the path; 1st self-esteem challenge - text on the floor is very difficult to be read; once completed the text saying you successfully finished is blurred.

BG: Still need to fix the companion’s mouth; user could be confused where to put player and companion to activate challenges; mark pads with 1 or 2; statements in the Cooperation game are not readable – especially problematic with the BG as sentences are longer; limit player choices to three - user will forget what the question was by the third one; second challenge is too difficult to mark the appropriate one with a number - divide into three separate with three options only; the last challenge with the map is almost impossible - people with learning difficulties have very bad orientation - need to simplify it.






531134-LLP-1-2012-1-BG-KA3-KA3MP

4.11 v0.3.7 Released: 2015-03-09 (330MB)

Same as v0.3.6, but with available DE & RS text added A spreadsheet listing all the translatable game text, and those translations received (DE

& RS) created


Partner Issue Reports:DE: game on the PC used is very slow; major delay between the mouse movement and the movement of the characters; ego-perspective of the players should be reinforced to increase readability of text or to introduce an observers perspective; open mouth of the companion was emphasized; major “input- lag”; right-click commands could be visible all the time; better zoom-factor should considered; M2 info sign on the wall browser problem

4.12 ISRG Testing – 2015-03-11 (v0.3.7)NICER Group meeting: Companion (open) mouth is scary; need conversation text spoken; need hints indicating pad actions; “Use” function not working.

4.13 v0.3.8 Released: 2015-04-07 (336MB)

Rooms 1, 2 & 3 (Communication) added Further DE translation incorporated


Partner Issue Reports:DE: Assertiveness room - turning blocks (Q3) as soon as I step back, the blocks will start to turn again, cannot select an answer; instructions needed; very slow on 6 of 7 desktop computers.NL: - Replace “Belgian” with "Dutch"; movement should USE the arrow keys as well; game gets stuck after viewing the instructions (select "België", back to a previous instruction, nothing happens).RS: Interface text needs additional translation; info dropdowns need translation; after communication R1, companion gets stuck, keeps going round in circles; companion can pass through closed doors; companion got lost game cannot continue without him, (next puzzle room can't be solved it without him.

4.14 ISRG Testing – 2015-04-22 (v0.4.0 pre-release)NICER Group meeting: Would prefer the 3D game on tablet; want characters to speak; found the textual nature of interactions problematic; think that will need carer or teacher support to play; interested in but unsure of multiplayer aspects; accessibility problems likely for students with physical impairments; different interaction style suggested – point & click to navigate

UK: Communication R2 – better instructions needed – if attention not paid to the statue at entrance, may become stuck; interaction words disappear when trigger platform is left; improve statue instructions in comm R3; after R3 exit blocked, cannot collect reward. Cooperation R2 – exception occurs when companion steps on platform, must then to right click in order to select things; before coop R3, (maze) camera clips player’s head at junctions;





531134-LLP-1-2012-1-BG-KA3-KA3MP

Conflict resolution R1 – trigger boxes visible; R2 – same bug as coop R2; missing answer element; CR R3 conversation spelling; Self-esteem after R1 – no barrier blocking the (wrong) path; SE R2 – cannot get the rear bridge section to come down; Self-control R3 (traffic lights) door at other side doesn’t open, lights do not come on, only crystal colour changes.

4.15 v0.4.0 (Full content release)Released: 2015-04-23 (346MB)

All 18 rooms present (rooms 7, 8, 9 (Conflict Resolution) & 13, 14, 15 (Self Control) added)

All available translated text present (in English if no local translation available) The spreadsheet contains all in game conversations and GUI element text in English,

& identifies where translation/clarification is required.


Partner Issue Reports:BG: Conflict resolution R1 statue does not respond, cannot proceed further; Path selection (6 category statues) only one is complete, but all still talk; at the end of each game (after completion of the 3rd challenge) “feedback" screen needs content; interface text needs additional translation

4.16 v0.4.1 Released: 2015-04-24 (347MB)

Conflict resolution room 1 – fixed questions not appearing in non-English versions 6 path statues room – only the active statue now speaks


4.17 v0.4.2 Released: 2015-04-28 (346MB)

Exactly the same as 0.4.1 but contains (virtually) complete BG, NL & RS (male) translations.


4.18 v0.4.3 Released: 2015-05-12 (346MB)

This version has (nearly) full BG, DE, EN & RS text - NOTE, LT has quite a few fields missing, and has no Turkish at all.

Inventory screen removed, replaced with Instructions Language change applied to GUI when selected at first run


Partner Issue Reports:BG: Communication R1 - starting conversation with the black statue is confusing; R2 -, the diamond appears after correct answers, but the player cannot take it; Conflict res – R2, 4th







531134-LLP-1-2012-1-BG-KA3-KA3MP

statement (the longer one) is not visible - some words and not on the screen; CR R2 – selection does not reset (cannot try room again)

RS: Communication gameR2 - diamond appears but player cannot take it, cannot proceed; player pauses the game, pressing red “X” button players exit the game and not just the menu; some parts of the game still not translated, e.g. Continue button and Command mode.

4.19 v0.4.4 Released: 2015-05-15 (346MB)

New TR translated content (some TR text still needed) Updated LT text & minor BG, & RS changes Conflict resolution – 2nd challenge – Text truncation fixed Communication – 2nd challenge – Room would not unlock on completion except in

English Topic selection screen now localised displays competence topics


Partner Issue Reports:BG: Interface text issues - text length to long for display – “Continue” not in native language

NL: "Continue" should be "Ga verder"; English audio should disabled – speaks over Dutch text; Companion always has his mouth open; Dutch spelling mistakes; drop down info boxes - Dutch and English both appear, not readable; some text doesn’t fit in available space, end is cut off, needs to be resized to fit automatically.

4.20 v0.4.5 (Full translated content release)Released: 2015-05-26 (345MB)

Full TR translations added (all languages now present) Minor BG, & DE & NL updates. Web browser now will not open when the tablet "Where dead men lie" is read Language switching improved Some untranslated fields added to localised content (“Continue” fixed) Scary companion mouth fixed


4.21 ISRG Testing – 2015-06-04 (v0.4.5)Sutherland House (at NTU): Experienced games players (autism) very quickly picked up how to play and played very well throughout; Strafe doesn’t work as expected;

4.22 v0.4.6 Released: 2015-06-10 (450MB)

This version has

A streamlined start (no Ginger) BG characters for letters A-G in Cooperation room 1 Working compass & map Speech volume is set high, and setting will be remembered between settings





531134-LLP-1-2012-1-BG-KA3-KA3MP

End of level feedback text (currently EN & DE) for the final screen – please translate in the spreadsheet (Interface sheet)

An updated spreadsheet (unlabelled speech items have been given ElementIDs for audio recording)

Guidance for audio recording Some example audacity projects


4.23 ISRG Testing – 2015-06-10 (v0.4.6)NICER Group meeting: Audio is essential; alternative interaction styles desirable, e.g. touch screen on a tablet (outside scope of project)

4.24 v0.4.7 Released: 2015-06-15 (350MB)

Minor fixes, particularly broken navigation in the settings screen RS & NL text added to feedback screen (end of path)


Partner Issue Reports:DE: Audio file naming issues need clarifying.

BG: Text still clipped in some places – on the cubes, pavements and the settings page; M3 (path selection) – inactive paths started speaking again.

4.25 v0.5.0 (Full audio release)Released: 2015-07-11 (377MB)

Full English audio, and numerous glitch fixes Play speech audio on MouseOver for Player/NPC dialogue and response options All in game text areas updated to allow for longer text in other languages (shrink to fit) Meaningful icons added to Topic Selection; jewels overlay when that path has been

completed


Partner Issue Reports:UK: “Tell me about the missions” option plays “I’m OK thanks”; can walk into water (between the beach and the bridge)

4.26 v0.5.1 Released: 2015-07-13 (481MB)

English, Serbian and German audio. BG game text updated with revised content


4.27 ISRG Testing – 2015-07-13 (v0.5.1)Sutherland House: including in NTU piloting report.






531134-LLP-1-2012-1-BG-KA3-KA3MP

5 CASE STUDIES

In order to gain more accurate picture about the progress in social competences and creativity during the training and to evaluate the learning and gaming materials more accurately, the method of case studies was used. To this purpose, the trainees whose progress was seen as typical by the trainees were chosen for the case study. In the following section the case studies for different pilot sites are presented.

CASE STUDY – AUSTRIA

Introduction

Susan is a 22 - year - old woman, who in terms of the relevant laws for persons with disability, is attending a so called “sheltered workshop” (towards productive activities). Susan is attending this workshop due to her difficulties, e.g. to understand complex situations. For her mild intellectual impairment (F70) has been diagnosed during primary school. Within this context, she performed several interventions and support activities, including teacher assistance at school and leisure time assistance during her adolescence.

Her strengths are observed within her friendly open temperament, however, it also can be observed, that she appears quiet shy. It means that she is attending the sheltered workshop; however her social contacts with peers and the mentors are mostly initiated by the mentors. Interaction towards peers is a big challenge. Mostly Susan is in a waiting position and observes others. If by chance another colleague within the sheltered workshop addresses her or contacts her, she needs some time to react. Usually she will lower her eye contact and show signs of social withdrawal.

Because of this shyness her trainers and mentors are not sure, whether her learning outputs are connected with her difficulties to communicate with her peers or whether these observations are based on more severe problems concerning learning.

Susan still lives with her parents, some forms or ideas of independent living were still delayed, both by the parents of Susan who were not sure, whether Susan was able to live on her own (with some kind of assistance). But also Susan, when she was confronted with some ideas towards independent living showed signs of anxiety. However for the trainers and mentors the next step towards independent living is a big goal.

Still Susan is brought by a specific bus for persons with disability. She stays all the day in the sheltered workshop and returns home to her parents in the late afternoon. During the weekends Susan mostly spends her time with her parents, sometimes she goes out with a leisure time assistance, who was engaged, to provide some her specific leisure time activities (like going to cinema or playing mini golf).

Participation in the training sessions

Current situation

Within a so called individual family support plan meeting the situation of Susan was reflected with the team, with Susan and her parents. In this context new goals within the workshop were designed with and for her. Susan expressed her intimate wish, to be able to drink coffee with her colleagues; however she was not sure, if she would be able to remain with her colleagues of the sheltered workshop without clear structure during for example coffee breaks.



531134-LLP-1-2012-1-BG-KA3-KA3MP

So it was obvious, that one major educational goal within the assistance process was the reinforcement and support towards communication with peers. Based on the available materials Susan together with her peers was sensitised concerning, how to come into contact with others. She learned that especially contextual cues (e.g. the situation of drinking coffee, eating cake within the situation of coffee break, the weather, the clothes, activities during the weekend...) could represent a good trigger for her to come into contact with the peers. In the very first weeks of this sensitisation process Susan was supported by a mentor who also joined the coffee break situation. The mentor supported her by means of overall cues and sometimes also prompted her to ask a question.

The progress in the area of social competences and creativity

Follow up and prompts

After three weeks Susan was better able to start a conversation, in this context she mainly focused on so called environmental factors as mentioned before. Also her peers noticed that communication with her became more fluent. This had an important impact that also the peers started to express more interest in the life of Susan and e.g. asked her what she was doing during the weekends.

Generalisation

The next step consisted in generalising the effects gained within the sheltered workshop. So Susan was asked to make some small shopping for the workshop by buying milk or sugar for the coffee and she was asked to go to the local supermarket. There she should ask one selling assistance, where the sugar or where the milk was. In the first sessions Susan needed some prompts and support, however after three times Susan could autonomously ask where some items were.

Evaluation

In evaluation talks with the parents also they perceived changes: within dinner situations or within weekends Susan tried to initiate more talks. Especially the idea, to initiate a talk by means of focussing on environmental factors and surroundings was one strategy, which helped Susan to initiate a conversation.

Materials that were used

Exercises from the SGSCC materials (mainly concerning initiation of conversations)

CASE STUDY – BELGIUM

General introduction

PhoenixKM works on a continuous basis with local end-users in the region of Ypres and has also attracted users from the local community to the piloting of the SGSCC outcomes.

The participants have all learning difficulties, moderate to severe, while most display complex disabilities. Social skills are a serious barrier towards full integration of people with learning difficulties in society.

To ensure the running of the pilot and the associated training, people with a background in psychology and sociology have been involved.



531134-LLP-1-2012-1-BG-KA3-KA3MP

A must has been to evaluate the ICT competences of the participants, hence why the majority used the mobile games, while only some users have experienced the desktop games.

Case study 1

Introduction

X.W. is a male of 36 years old who at present works in a sheltered environment. He does so already for the past 15 years. However, a main issue with his work environment is that he has a history of violent behaviour and communication with his co-workers, and therefore has already been cautioned on a number of occasions.

X.W. feels often misunderstood, and admits that he has problems in expressing feelings, emotions, and in understanding and correctly interpreting the behaviour of others.


X.W. participated in almost all training sessions, whereby he obviously was most interested in those aspects of the training that contributed to him better understanding his counterparts.

Especially the mobile game whereby emotions were addressed was very popular with him.

Upon request, X.W.’s mentor also approached one of his supervisors and informed him that X.W. was working on controlling his anger. This was positively embraced.


X.W. benefited extremely from understanding his own strengths and weaknesses, and has understood the importance of also listening when communicating. As a direct outcome, X.W. has improved his behaviour considerable, especially also in understanding emotions, and how others saw him with his past behaviour.

A recent meeting between the mentor and the supervisor revealed that X.W. has become more careful in vocally blurring out feelings to colleagues, and conflicts have been drastically decreased.


“Social Competence and Creativity Unit" with "Appendix 1.Practices and exercises of social competence and creativity”

Mobile game “Inca Quest”

Case Study 2

Introduction

A.J. is a 34 old female who comes from a socially disadvantaged family, became pregnant when she was 17 and has only the right to see her kid during the weekend under supervision. She has a history of alcohol abuse, but has changed drastically (positively) since she started a relationship with another man from the user group. They are both also under budget control.



531134-LLP-1-2012-1-BG-KA3-KA3MP

Main issues she faces are anxiety to express what she thinks and feels (assertiveness), resulting something her ending in a situation at work where she does not feel comfortable in doing a certain part of the job, because she did not dare to say no. “I act like that because they consider me stupid and know I am too shy to react.”


Both A.J. and her friend participate in the training, and have expressed especially interest in assertiveness training. To achieve this, some specific situations were presented as they also faced in real life.


Both have drastically improved the way they started interacting with others, albeit that this part of the training has proven to be most challenging for all participants. Assertiveness and standing up for one’s rights is a difficult issue and evokes situations people with learning difficulties are daily faced with.

A.J. has improved her confidence and was eager to learn how she could proof her confidence in a manner that was not arrogant. “Last week there was a new issue popping up at work and they immediately said I could solve it. For the first time I objected and said I was not the right person to do this. My supervisor told me she was surprised of my reaction but said she would respect this.” And in the end, it is about respect among peer and among staff in a company.



Desktop game “Inca Island” Mobile game “Inca Quest” to some extent

CASE STUDY – BULGARIA

General introduction

Interprojects team maintains a long-term partnership with Sheltered House – Sliven where 8 women with moderate to severe intellectual disabilities are accommodated. Despite the severity of their disabilities the residents of the Sheltered House possess a preserved level of intellect, basic social skills and reactivity because of their active way of living. In addition the women have previous experience in working in groups developed by the social workers and the psychologist within the service.

These facts motivated Interprojects to select the team of the sheltered house and its residents to take part in the testing phase of SGSCC project. In December 2014 the social workers and the psychologist of the sheltered house participated in induction training which made them familiar with the piloting scheme of the project as well as with the draft versions of The Social Competence and Creativity Unit and The Illustrative Handbook. In the second part of the training the professionals observed a demonstration of the mobile and desktop games.

Thus the participants were prepared to lead the testing process as trainers and to support their users with disabilities in testing the SGSCC outputs. The trained professionals were free to



531134-LLP-1-2012-1-BG-KA3-KA3MP

choose those people with disabilities who in their opinion would manage to play the games and exercises. For instance, within the Sheltered House – Sliven only three women were able to test all games and the exercises marked in blue flag (the lowest level of difficulty).

General feedback from the trainers in Sheltered House – Sliven

The two social workers supported by the psychologist of the sheltered house have conducted iterative sessions within which the residents with disabilities performed the exercises from Social Competence and Creativity Unit indicated with a yellow flag.

In Communication subgroup the participants managed to play the exercises, they understood the instructions and followed them with willingness. They engaged in giving feedback; however they faced challenges to link what they have learned from the particular exercise with their personal life experience.

However, when the trainer gave additional clarifications and examples the participants achieved better training effect. The examples from the trainees’ daily life also contributed to the development of the trained social skills.

For example, in recognising of emotions the illustrations from the handbook helped a lot to the participants. After the women with disabilities were trained to recognise the emotions through the handbook, they started to play the Inca’s emotions mobile game. According to the trainers’ opinions exactly this structure of training contributed to the successful acquiring of social skills and sustainable interest towards the mobile game. “This proved that the SGSCC outputs complement each other and should be used together in order to boost the training effects” stated Diana Cvetanova the manager and the psychologist of the sheltered house.

Concerning the exercises about the development of empathy the women executed them swiftly, however the trainers noticed that the participants not just put themselves in the other’s shoes but they actually experienced their feelings and emotions. Therefore, Diana Nikolova, a social worker, stated that the trainer should follow the process actively to ensure gaming will become learning.

Diana Nikolova said also that the exercise for realising the advantages of the acceptance confirmed that the participants easily accepted the other’s positive features.

According to the trainers’ opinions in Cooperation subgroup the participants understood the instructions and the content of the exercises; however they faced challenges in performing the exercises about realizing the other’s point of view. Desislava Dimitrova who was the second trainer shared that the participants were successfully trained to express gratitude, however they managed to do this in particular daily situations only, for example when they received a gift or a reward such as SGSCC stress balls.

Both trainers shared that the group of exercises concerning the abilities for solving interpersonal problems is useful and might be extended with more exercises for solving conflicts. Diana Cvetanova shared that it would be beneficial if the trainer made smooth transition from the situations that are not directly linked to the conflict driven by their own emotions to solving of conflicts corresponding to their own life situations.



531134-LLP-1-2012-1-BG-KA3-KA3MP

Concerning the Self-control and the Self-evaluation subgroups the trainers shared that the participants were actively involved in the exercises however these subgroups proved to be the most challenging in terms of acquiring long-term social skills on these topics. The participants played the desktop game intuitively and they felt joy and amusement. What they have learnt was how to formulate their own interests, talents and abilities but the self-control needs more extensive and long-term training.

The trainers also commented Assertive behaviour subgroup which was trained subsequently by the exercises and the desktop and mobile games. The participants with disabilities learnt how to make requests and the trainers believed that they develop this social competence. The trainers gave a recommendation that in a future project other exercises about making requests can be developed with more emphasis on distinguishing requests from requirements. This means to construct exercises in which to focus on the different elements of stating the request – facial expressions, tone, gestures, and mood.

The trainers in the Sheltered House – Sliven have consolidated the following conclusions:

The women that took part in the SGSCC testing phase actively participated in all activities. They played the mobile and desktop games iteratively and they were useful for them because through gaming they find their own zone of comfort.

The variety of tools developed within SGSCC project such as exercises with different levels of difficulties, illustrations, simplified texts, theoretical guidance for trainers and desktop and mobile games guaranteed that the participants had different channels to utilise the learning content and to develop their social skills.

The SGSCC outputs meet the needs of people with different types with disabilities, however the trainer is the one that is responsible to personalise the content in a way that suits the individual needs and temperaments of each person with disability.

Case study 1

Introduction

B. is one out of the eight women with intellectual disabilities accommodated in Sheltered House – Sliven. She is 34 years old, single and unemployed. Her daily life is centred on the activities planned by the social workers within the residential service. Apart from the intellectual disability B. also has an Autistic spectrum disorder. This circumstance often makes her isolated from the daily routines as well as makes her a less communicative person. That is why B. was selected by the trainers to test the SGSCC desktop and mobile games. Her ICT skills were well-developed which considerably contributed to the easy and faster way of coping with the mobile and desktop games.


At the beginning B. started playing the three mobile games as they proved to be easier compared to the desktop game. She was supported in playing all three mobile games only in her first two consecutive attempts. Then B. continue playing independently every day.




531134-LLP-1-2012-1-BG-KA3-KA3MP

According to her words she felt at ease when playing and praised when finished successfully and got the diamond. B. also shared that the Android mobile phone given to her for the purposes of the testing became her “best friend” because when playing she has her own time with Inca character. Within the interview B. mentioned that her favourite game was Inca’s spirits. The most difficult game for her was Inca’s adventure. B. said “I prefer Inca’s quest because the games are short and I can easily follow them. Now I know some phrases that I prefer to use in my daily life.”

In the second part of the interview B. was asked about the desktop game. She tested it only twice with the support of her trainer Diana. Every time she reached the 3rd room she gave up because she felt tired. B. said “What I like the most in this game is that I was able to communicate with the statue and she even responded. I also like the colour scheme and the music background. I would like to learn other things together with the companion so I will keep on playing in the future.” At the end of the interview B. shared that for the first time she was involved in such activity. She thought that the computers and mobile phones connect people and serve for fun but now she knows that they can help her to learn new things in interesting and involving manner. Her trainer also added that soon B. will start a vocational course and both games were very useful for her and made her more concentrated and persistent. Also B. became more communicative and started expressing herself with longer sentences. The interview ended with the statement by B. that she wanted the SGSCC project to be continued and other games for mobile phones to be developed.



Desktop game “Inca Island” Mobile game “Inca Quest”

Case Study 2

Introduction

D. is 47 years old woman who is living in the Sheltered House – Sliven since its establishment. She has moderate intellectual disability characterised by learning difficulties and a high level of physical stress. In addition, she is over-communicative and open to experience with new things. These were the reasons why D. was involved in SGSCC testing. Another reason is that soon she will start working as a shop assistant in a supermarket near the sheltered house.


D. tested the exercises marked with a blue flag. Within the interview D. was very excited about her participation in the testing of the games and exercises.

D. also tried the desktop and mobile games but because of the consequences of her disability she wasn’t that much successful as she was with the exercises marked with blue flag. However, the testing of the games motivated her to start a computer course for the development of her ICT skills. After the completion of the course she believed that she would be more successful in playing the games.



531134-LLP-1-2012-1-BG-KA3-KA3MP


D. said that she took part in the group activities for testing of the exercises with great pleasure because it gave her a chance to cooperate with the rest of the women and it strengthened their relationships. According to D.’s words the exercises about the development of communication skills and cooperation were the most interesting and helpful for her. The one that she liked the least was about self-control. D. also said that she would like to perform different exercises regularly because what she has learnt can be otherwise easily forgotten. She also added that through the exercises she learnt how to react in different situations, how to follow instructions, and how to formulate requests, desires as well as how to recognise the others’ emotions.

Furthermore, D. shared that the most interesting game for her was Inca’s emotions because for the first time she realised that she can recognise facial expressions by other means and not only directly by people’s faces. She added that if she made a mistake in the game, she was able to correct it.

At the end of the interview D. said that she felt herself special when she was recruited to participate in piloting. And every time when she succeeded the trainer gave her a symbolic reward such as SGSCC bags and stress balls. Her trainer also confirmed that at the end of the testing she became more confident and able to control her way of expression (verbal and non-verbal). D. concluded that she is well-acquainted with the Illustrative handbook and she wanted to give it to other people alongside with the games in order to give them the same chance to play and learn as she had.

CASE STUDY – LITHUANIA

Case study 1

Introduction

The boy of 15 year old has mild mental disability. His parents have one more teenager son. They communicate with the boys using the common communication principles: by speaking, writing, using mimics and pictures. The social and financial conditions are well. The boy has a separate room and computer.


The boy is hyperactive. He confronts very often with the brother (leaning to aggression). While he was participating in the tests he was active, joyful and communicative.

He is using computer every day for playing the games, search the internet, watching movies. He knows how to use MS Word software (basic level). He does not use educational computer applications at home, only at school. He particularly likes applications for learning math.

The boy is very curious – he wants to know everything but has difficulties to concentrate. He can keep attention longer only if the activity is clear and interesting. During the introduction



531134-LLP-1-2012-1-BG-KA3-KA3MP

of SGSCC games he was very interested to pilot them and he eagerly tested all of them. He liked the most Memobile and the first level of PC game – Inca’s Adventure. He managed to understand the game rules and he played them individually. He liked the games and asked if he could have them in his computer to play at home. The students also made a lot of comments about PC game and in those situations he liked to show his knowledge about the 3D games and to identify the differences he observed.


The main achievements achieved by this student are related to his concentration on educational content and positive disposal concerning new skills. He did not learn too much from the particular content, but he was more self-confident and problem solving oriented. He eagerly repeated game sessions and was engaged in the process.


Social Competence and Creativity Handbook Desktop game “Inca Island”

Case study 2

Introduction

The 16 year old girl has severe mental and speech disabilities. The parents are wealthy; there are three children in the family. The parents take care about the daughter – she has individual room and nanny who stay during the day time with her at home, drives to the schools and to pick up after the lessons. Parents communicate with the daughter using the common communication principles: by speaking, writing, using gestures, mimics and imitation. The mother takes care about child as much as she can. The girl does not have any other physical disabilities but she faces learning difficulties: she needs consistent support and assistance. Because of the listed reasons she has difficulties to study and low motivation. In the special school she broadened her knowledge and developed cognitive and social skills. She has PC used at home for learning.


She enjoys using computer in the school to follow educational tasks in the lessons. The student was invited to test the SGSCC mobile game Inca’s Quest. She was happy to use mobile device but was concerned and had low self-confidence. However, when she understood instructions how to play the game, she became more self-confident. After playing three sessions, she got confident to play individually, without asking for a help. Her limited abilities to understand content presented in texts sometimes caused her lower interest (Quiz). The most useful for her was the game with the face expressions – even though she could not understand all explanations, she was eager to learn the content and to play the game. She managed to match correctly the pictures almost every time, and she was very happy because of that. Moreover, she was engaged and concentrated playing the game within lessons.




531134-LLP-1-2012-1-BG-KA3-KA3MP

The games encouraged the student to follow instructions, to be more self-confident, to express her mood and to try to name correctly the expressions of other students.


Social Competence and Creativity Handbook Mobile Game “Inca Quest”

CASE STUDY – SERBIA

Introduction

Trainee M. D., a sixteen-year-old student of the first year of Secondary school, is at this moment in the Out - of - Home Care system. He has been a user of a residential care system for three years. Before that, he had been living with his relatives but he had to be transferred due to an abject poverty they had been living in. His brother has an intellectual disability, which also made it difficult for relatives to take care of them.

Before the start of the training, he had been at risk of failing seven subjects, which would have resulted in his repetition of the first grade. Teachers were not sure whether his underachievement was connected to some learning difficulty or it was related to difficulties regarding social interaction. Furthermore, he had been having angry outbursts when his needs were not satisfied and he had often engaged in quarrels with his teachers. Moreover, when confronted with his poor school achievement, he would deny any responsibility for the situation. His guardian in the Out of Home Care described him as a person who was not emotionally involved when he was in contact with other people and who generally shied away from social interaction. His homeroom teacher expressed worry over the fact that he was not adapting to the secondary school.


He participated in the seven workshops and four game sessions, which covered all six social competences and creativity. Namely, his trainer addressed the following topics: listening, accepting yourself, following rules, strategies of self-control, conflict-resolution strategies, understanding the difference between assertive and non-assertive reactions and creating a story. During training sessions, the exercises were adapted for the needs of particular group and teamwork was emphasised. The trainee enjoyed workshops immensely. Moreover, he was highly engaged when he was playing computer games “Inca Island”, which is somewhat expected since he plays computer games daily. The atmosphere of the workshops was described as positive, students were laughing and giving examples from their everyday life. The teachers noted that the students were visibly looking forward to the workshops.


His teacher observed several differences in his behaviour after the training. Most notably, his marks in all courses have increased. Actually, at this moment, he is no longer under risk of failing a grade. Furthermore, there has been an observable progress in his social competences. Namely, his teacher has said that he is more relaxed in social interactions and that he can express feelings more easily. Whilst he was mostly observing his peers without communicating with them before the training, at this moment he is actively seeking company and engaging with classmates. Furthermore, his aggressive outbursts have reduced



531134-LLP-1-2012-1-BG-KA3-KA3MP

significantly. Specifically, he is cooperating with his peers and teachers and expressing his feelings in an adequate way. For instance, earlier he would approach his teacher and say “I demand that you give me a better grade, I deserve it”, whereas now he is calmed when he talks with her and gives arguments for his requests. This is also the area in which he observes improvement. He reported that he has begun using a strategy of counting from 1 to 10 when he is angry and that he founds it helpful.



Desktop game “Inca Island”

CASE STUDY – UK

Introduction

AT is a 37 year old woman with Down’s syndrome. She is an experienced and highly motivated game player. She works part time at a local special school. She has problems with self-control issues, leading to over eating and stealing which has a negative effect on further employment prospects

Participation in the Training Sessions

AT participated in four training sessions, and a feedback session. Firstly she went through the Handbook activities with her trainer, and then played both the Mobile and Desktop Game in repeated sessions. She played the Desktop Game firstly in a group situation, and then in the remaining repeated sessions in a co-discovery pair. She concentrated on the communication, cooperation, self-esteem and self-control social competences in the Desktop Game.


The Desktop Game generated lots of discussion on the meaning and issues surrounding resisting temptation, and instant gratification when solving task (specifically important for this student). These activities are particularly relevant for AT who has got repeatedly into trouble with stealing, and over eating – and the games generated good discussions around these issues. AT’s carer reported that these discussions were an important step on the road to acknowledging her self-control problems and how these behavioural issues are affecting her life in a negative way. Strategies for self-control were discussed and developed, and conversations initiated as to whether she could cope in new work situations where there is temptation present. She also talked about the strategies for self-control that would work for her (in response to some suggestions for trainers). AT’s carer commented on how the Games and Handbook are successful as a springboard to open up discussions on these difficult, but highly relevant, subjects. AT was also reported to have limited memory skills, but she remembered significant game features and mechanics from previous sessions, and this demonstrates a mechanism for developing memory skills via the use of such engaging games.


Social Competence and Creativity Handbook Desktop game “Inca Island”



531134-LLP-1-2012-1-BG-KA3-KA3MP

Mobile Game



531134-LLP-1-2012-1-BG-KA3-KA3MP

6 GENERAL CONCLUSIONS

During the project, the SGSCC materials were piloted with 304 people with learning difficulties and people from disadvantaged groups. All significant findings have been integrated into final products.

The participant learners’ backgrounds can be described as diverse. Many participants had complex disabilities (in addition to learning disability they also had one or more other disabilities). Moreover, students (16+) in schools were included in the training as well as people who live in a sheltered environment. In order to respond to these diverse needs of learners, the learning and gaming materials have been continuously upgraded and adapted.

The main conclusions are as follows:

The blended learning approach which includes using gaming and learning material has shown to be useful for improvement in the communication skills, self-esteem and creativity. Moreover, many trainees themselves observed a change in communication skills and self-control, which supports the conclusion that following the SGSCC curriculum can lead to a positive change, observable in everyday context.

The learning materials have proved to be useful for work with target group, as evident by the high average mark 4.35 (5) and highly positive feedback during sessions on which the handbook was used.

The desktop game Inca’s Island was seen as somewhat difficult and engaging. Its graphics, incorporated feedback and the way it fosters cooperation between trainees have been seen as especially beneficial.

The mobile game Inca’s Quest was reported to be easy to use. At the same time, it has been assessed as engaging. Its interactive nature, graphics and incorporated feedback have been highlighted as useful.

Following lessons were learned during the piloting process:

Lesson 1: The quality of training and transferability of the materials into real life situation greatly depends on the ability of trainers/mentors/teachers to ENGAGE the learners. ENGAGEMENT is correlated with the ATTITUDE and the PREPARATION of the trainers/mentors themselves.

Lesson 2: Materials within SGSCC project can serve as a starting point for trainers/mentors/teachers to FACLITATE A LEARNING SITUATION for the beneficiaries. Therefore trainers/mentors/teachers need to adapt these supporting materials towards the individual NEEDS OF THE LEARNERS. SYNERGIES created between the printed training materials, the mobile game, the desktop game and the illustrated handbook or between other useful materials (like videos, cartoons, stories, own prepared materials) were found to be extraordinarily useful.

Lessons 3: Some materials were found to be very useful in promoting transferability real life and/or into the labour market, other exercises showed some limitations towards transferability. These limitations were perceived to be associated with the diverse needs of the beneficiaries within the pilots. This means that the materials will



531134-LLP-1-2012-1-BG-KA3-KA3MP

have to be pre-assessed during the train-the-trainer preparation, based on the trainers/mentors/teachers professional experience. Exercises should be selected carefully, and must consider the personal strengths and abilities of the learners. Some exercises proved to be challenging e.g. for learners with Autism Spectrum Disorders (for example, in the recognition of emotions or meta-competences (seeing things as others do), or in the expression of trust).The training materials are intended to initiate transfer effects from the resources to the specific needs of the learners. Therefore the mentors / trainers / teachers are asked to carefully PRE-ASSESS the potential STRENGTHS AND CHALLENGES of the INDIVIDUAL Learners. The Questionnaire for self-assessments of social competences and creativity and Questionnaire for evaluation of social competences and creativity, developed within SGSCC project, can be used for this purpose.

Lessons learnt 4: Promoting social competences for people with disabilities requires PREPARATION of the trainers / mentors / teachers; sensitive MATCHING of exercises to the individual’s needs and TRANSFER of the learned content into daily life. Some exercises might allow a higher transfer into the labour market related fields (conflict resolution, cooperation), while others are more closely linked to daily life situations (trust, initiation of conversation).

In terms of sustainability we are also interested in your experience with the materials. Please feedback your usage to [email protected].



531134-LLP-1-2012-1-BG-KA3-KA3MP


Appendix 1: Diagram - Timeline of piloting and evaluation activities


Documents

Usability comparative report - SGSCCgames4competence.eu/wp-content/uploads/2013/04/SGSCC_BOS_… · Web viewHe is using computer every day for playing the games, search the internet,