[IEEE 2013 International Conference on Interactive Collaborative Learning (ICL) - Kazan, Russia (2013.09.25-2013.09.27)] 2013 International Conference on Interactive Collaborative

Musimage: using social multimedia to engage pupils in classical music appreciation

Laurent Moccozet, Camille Tardy Computer Science Centre (cui.unige.ch)

University of Geneva Geneva, Switzerland

{Laurent.Moccozet,Camille.Tardy}@unige.ch

Hassan Lakhdar, Anne Lakhdar Teenergy Production (www.teenergy.ch)

Montreux, Switzerland {hassan, anne}@teenergy.ch

Richard Rentsch, Michaël Wirth Gymnase de Beaulieu (www.gymnasedebeaulieu.ch)

Lausanne, Switzerland [email protected], [email protected]

Abstract—Music teachers in secondary schools are facing a great challenge when attempting to successfully engage pupils in classical music appreciation. The distance e-learning sector has for a long time neglected to address the specific needs of teaching and learning music. This paper describes the use of social multimedia technologies to engage pupils in classical music appreciation. The approach relies on different hypotheses: nowadays, classical music needs to be a global experience, combining audio and visual; classical music is actual and generates emotions; pupils need to be active and creative even though they are not able to compose or create music. The resulting social educative platform is described together with the experiments and results that were conducted to evaluate the approach.

emotion; movie; multimedia; music; music appreciation; social multimedia; social network

I. INTRODUCTION Music teachers in secondary schools are facing a great

challenge when attempting to successfully engage pupils in classical music appreciation. There are many reasons for this situation, but the most important ones are usually related to negative preconceived ideas: classical music is only for old or cultivated people; classical music is out of date and boring; classical music is not a popular music. Therefore, pupils are even reluctant at the very thought of listening to classical music. The distance e-learning sector has for a long time neglected to address the specific needs of teaching and learning music. Two reasons can explain this situation as raised in [1]. On the one hand, in educational institutions, learning music is generally underestimated as an optional subject thus receives little attention and support for the development of new teaching tools. On the other hand it appeals to visual, auditory and even tactile dimensions that were until recently poorly controlled and little available in the technologies being integrated into the standard educational platforms. In this paper, we investigate the use of social multimedia technologies to engage pupils in classical music appreciation. The approach relies on different

hypotheses: nowadays, classical music needs to be a global experience, combining audio and visual; classical music is actual and generates emotions; pupils need to be active and creative even though they are not able to compose or create music.

II. ICT IN MUSIC EDUCATION Information and communication technologies (ICT) are

widely used in the music industry and they have considerably changed the ways music is accessed and owned including for the young people that are learning music in secondary schools. According to [2], the rise of cloud computing [3] has introduced a radical change in the relationships between music and people. With online services such as Spotify1 or Rhapsody2, music is now getting ubiquitous [4]. Therefore, music education has naturally tended to follow this evolution and question its practice to take into account the integration of these technologies [5], [6], [7], [8].

In a survey about the way nine music teachers are integrating ICT into their teaching activities [9], it is acknowledged that, “computer-mediated activities, result in a high level of student engagement and achievement”. In a qualitative and quantitative study [10], students expressed that “most students believed that IT could provide motivation for music learning” and that “they used IT mainly in their listening activities but believed that it could also be helpful in learning both classical and popular musical styles.” This hint is also highlighted in [11] where it is advised that to engage pupils musically, it is needed to take them where they are by considering their actual experiences and skills and by creating collective spaces based on their own relationship to music where they can express themselves. Nevertheless, the level of integration of ICT in teaching activities deeply relies on teachers’ confidence and skills in ICT [9]. It also depends on the time required in learning the digital tools and the need to

1 http://www.spotify.com/ 2 http://www.rhapsody.com/

2013 International Conference on Interactive Collaborative Learning (ICL)Page 164

978-1-4799-0153-1/13/$31.00 ©2013 IEEE 25-27 September 2013, Kazan National Research Technological University, Kazan, Russia

adjust the teaching methods to appropriately integrate the digital tools [12].

One possible way to apply ICT for music education consists in using existing Learning Management Systems (LMSs) such as Moodle and enriching them with additional multimedia resources. At the University of Music and Performing Arts in Graz, such an experience is being developed [13]. LMSs are widely available nowadays in higher education institutions. Unfortunately, they do not offer a large range of multimedia tools. Traditional tools such as wiki, forums or blogs limit the possible assignments and to text-based activities. External platforms have been investigated such as video commenting students’ musical performance, podcasting and interactive animated scores. The author concludes that further research is required to connect and integrate the LMS with the multimedia tools in order to create a continuum for theoretical and practical learning and training. The MusicWeb3 project presents itself as the “musical eLearning tool”. It offers a dedicated e-learning and authoring environment. It allows producing music course resources based on text, images, scores and audio. Netmusic4 is an ad-hoc platform built on the open source Content Management System Drupal. It provides step-by-step tutorials based on text and pictures including quizzes about basic techniques and processes involved in computer music. Some systems are also relying on mobile devices that youths are extensively using. MOGCLASS [14], which stands for Musical mObile Group for Classroom Learning And Study in Schools is a multimodal musical teaching and learning environment with mobile devices based on Apple Ipod touch. The goal is to create a networked mobile environment to support children's collaborative perception, performance and production of music. It offers for example a range of simple musical instruments that can be played individually or collaboratively with mobile devices. The evaluations of the system have shown that actively engaging pupils in creative activities was effective to motivate interest and collaboration in music learning.

R. Orpheus advocates that the combination of online collaboration tools, social networking on the Web and the loosening of copyright restrictions such as the Creative Commons initiative will have a great impact on music education [15]. Therefore, this is not surprising that among all the possible ICT that can be used in classes, the online collaborative social and multimedia aspects provided by the Web 2.0 are particularly investigated [11]. In [16], the author surveys various online collaborative tools, from ones that are very common such as blogs or wikis to ones that are specifically dedicated to music education. In [17] the evaluation of podcasts reports a higher level of enjoyment and comfort with the course. Among the different purposes of the Netsound5 European project, one consists in facilitating the exchange and sharing of good practices in ICT-based innovation in music education. The project portal includes a repository of educational tools and good practices collected during the project. A survey on good practices and

3 http://musicweb.koncon.nl/en/ 4 http://netmusic.brightonart.eu/ 5 http://netsoundsproject.eu/

methodologies for music education through ICT is also available [18]. It analyses the area according to different identification criteria. One of them is social network and a large part of the good practices listed in the survey are dedicated to the Web 2.0 ecosystem. From all the available Web 2.0 categories of tools, social multimedia platforms show a great potential for music education. It opens the way to integrate at the same time social and creative work and classroom activities [19]. Teachers are for example using Youtube in their class [6], [9] for listening activities: pupils can listen to the music and watch musicians’ performance at the same time, or they can compare various interpretations of the same music piece.

Fig. 1. The test class configuration and the social multimedia platform

interface

Web 2.0 pedagogical tools ecosystem can be roughly sorted into three categories: audio resources for music listening and appreciation such as Musopen6; tools for producing and composing melodies such as SoundJunction7; and tools for learning and practicing an instrument or instrumental techniques, such as GuitarTricks8. There are very few alternatives to composing tools to make pupils active and producers of contents. Unfortunately, composing requires skills that are not available or even just relevant for classical musical education in secondary school. Moreover, the existing tools are widely dedicated to pupils that are already engaged and immersed in music. Our goal is to provide educational

6 https://musopen.org/ 7 http://www.soundjunction.org 8 http://www.guitartricks.com



activities that involve students who are not necessarily interested by default in classical music and which will lead them to get engaged.

III. SOCIAL MULTIMEDIA FRAMEWORK DESIGN The film industry makes extensive use of classical music in

movies’ soundtracks to support the expression of a wide range of emotions. The soundtracks of popular blockbusters such as Superman, Star Wars or The Lord of the Rings contain classical music pieces. These movies are part of the pupils’ culture and they represent a good catalyst to take them where they are and bring them to classical music. The main idea is to use pupils’ creativity by empowering them to create their own audio-video mix to arouse emotions that they can further share and compare with their peers.

A. Feel the music “Feel the music” is a popular expression that connects

music with feeling and emotions. The close relationship between emotions and music (and even art in general) is commonly recognized and even associated to brain research on emotions [20]. In her book [21], J. Robinson explains how our emotions help us to understand music and the impact of emotions for music education has also been investigated [22]. Therefore, emotions represent the facet that connects all the actors and components of the proposed musical educative platform. Emotions will be expressed as tags that can be associated to any of the content available on the platform.

There are many ways to express emotions verbally. This is why we need a common taxonomy, so that all stakeholders can share the same vocabulary, even if it can reduce the richness of the emotional impact. Among all possible formalization of emotions, one is being widely adopted in computer science to represent and process emotions: the Ekman’s extended list of emotions [23]. It is being used for example to recognize facial emotions and expressions from photos or videos [24]. There are six basic emotions: anger, happiness, surprise, disgust, sadness and fear. This initial list has been extended later with: amusement, contempt, contentment, embarrassment, excitement, guilt, pride in achievement, relief, satisfaction, sensory pleasure and shame.

B. Implementation There are available social multimedia public platforms such

as Youtube9 or Soundcloud10 that would provide the main features required to support our pedagogical scenarios and activities. However, few problems have to be taken into consideration regarding data privacy, the lack of selection of the contents available on the platforms and to some extent, the open access to these contents. These issues advocate for a dedicated platform, which allows the control of pupils’ personal data and a careful selection of the available contents. Moreover, social multimedia public platforms are mainly dedicated to entertainment and not to education. Therefore, they are lacking many of the features required for example to monitor and assess pupils’ activity.

9 http://www.youtube.com/ 10 http://www.soundcloud.com/

The platform Musimage has been developed with two open source platforms: Drupal11 and Kaltura12. The first one is used to organize the social and educative aspects of the platform and it integrates all the social features: profiles, comments, tags, blogs, forums… and all the pedagogical features: quiz, activity monitoring… The second one is dedicated to the management of the multimedia content: storage, mixing and other processing. A dedicated Drupal plug-in connect the two platforms so that Drupal can offer multimedia-processing features that are handled through the Kaltura server. Different user profiles are setup to define the roles of the teachers and pupils on the platform. Specific features have been implemented such as contents tagging with the predefined list of emotions based on the Ekman’s extended list.

The server is filled with selected multimedia contents: video, music and photo. The video section contains short movies excerpts. These excerpts illustrate passages of the movies with classical music pieces as soundtrack. Each movie excerpt is described with a summary and a set of few metadata: the title of the classical soundtrack, the director, the main actors, the type of movie and a list of emotion that the excerpt is expected to produce. The music section contains famous pieces of classical music in mp3 format such as Beethoven’s Moonlight Sonata or the Adagio Assai of Maurice Ravel's Piano Concerto in G major. Each music piece is also described with some metadata such as the composer and the main instrument. The photo section contains photos of various types of art pieces: paintings such as The Scream from Edward Munch; monuments such as St-Peter’s basilica; sculptures such as the David from Gianlorenzo Bernini; landscapes such as a cloudy sky or a sunny street. All the video and audio content can be played online thanks to an embedded multimedia player. All contents are tagged with keywords and a tag cloud is provided so that pupils can explore the multimedia contents according to different facets: composers, movies, actors, emotions…

Fig. 2. The online multimedia montage interface in action

C. Tests and experiments Various pedagogic scenarios can be defined based on this

framework. We have identified three types: discovery,

11 http://www.drupal.org/ 12 http://www.kaltura.org/



experimentation and creation. Discovery scenarios are mainly based on the available contents, which are used to discover, explore and study the impact of music in the cinema and the arts in general through the prism of emotions. Potential usages include:

• Knowing and recording the emotions generated by a particular movie clip,

• Using the movie clips in a course to illustrate a particular piece of music,

• Analyzing how the cinema and other media deal emotions visually and by the choice of music.

With experimentation scenarios, pupils are getting more active. They experiment the emotional potential of music pieces. Potential usages include:

• Discovering variety of emotions and feelings according to the associations,

• Experimenting and explaining the choice of associations between movie clip and music,

• Learning how atmospheres can be created to arouse emotions,

• Understanding the influence of sound and image on what we feel personally,

• Stimulating creativity by exploring the range of possible emotions that sound and image can arouse.

With creation scenarios, pupils are really getting actors of their learning. In this context, pupils can create their own multimedia mix by selecting different pieces of multimedia (in further version, pupils should be also able to add their own multimedia contents). Potential usages include:

• Creating a multimedia mix to illustrate a given emotion,

• Creating collaborative games where pupils have to discover the emotion that their peers expected to arouse with their multimedia mix,

• Creating contests of mixes on a given emotion.

Fig. 3. Musimage user interface

Musimage has been tested with two classes of 30 and 20 pupils of 17 to 18 years old (fig. 1). The tests have both lasted half a day. The two experiments were organized as face-to-face workshops and took place in a multimedia room equipped with personal computers. Pupils were provided with audio headsets in order to be able to listen to the music in good conditions without disturbing others or being disturbed by others. For both

tests, the music teacher started by showing a few mixes with explanations about the use of the classical music in the soundtrack. Pupils were then asked to take the time to explore the contents (pictures, music pieces and short movie clips). They could for example comment the emotions they felt while watching some of the movie clips and compare with their peers. After the exploration phase, they were requested to



experiment how simply changing the classical music piece of the soundtrack could change the emotion. This was the occasion to engage pupils in a first simple test with the multimedia mixing tool (fig. 2): pupils had to select a movie clip; select a music piece and insert it in the movie soundtrack instead of the original one. For the last exercise, they where asked to choose an emotion they would like to express and select pictures, movie clips and classical music pieces to create a multimedia mix to arouse that emotion. After publishing the mixes, pupils could share and comment the production of their peers. Fig. 3 shows the user interface of Musimage. In the left column, menus give access: to the contents by types or by ranking according to pupils’ evaluations; to the creation of contents, particularly multimedia mixes; and to the social features such as forums, blogs. The right column gives access to the tag clouds to explore the contents by keywords and to the comments that pupils shared about their productions. In the middle of the screen, the multimedia mixes are displayed with their title and thumbnail. The thumbnail is a single frame, which is extracted from the multimedia mix.

Fig. 4 shows an example of a movie with a modified soundtrack. The selected movie is Black Swan and the soundtrack is the Beethoven’s Moonlight Sonata. The pupil has entitled her mix with “The swan ends its race” and described it with “Fear, sadness, joy, fulfillment, resolution ... that's what I feel when I hear this music with this excerpt ... we understand everything ... the complete fulfillment”. Fig. 5 shows a more complex multimedia mix composed of a selection of four still pictures organized with transitions. The selected soundtrack is the Adagio Assai of Maurice Ravel's Piano Concerto in G. The pupil indicated that she wanted to express happiness.

Fig. 4. An example of a multimedia mix composed of two from the movie Black Swan. The original soundtrack is changed to Beethoven’s Moonlight

Sonata

The social multimedia platform includes the codes pupils already know and master. Therefore the undertaking of the platform occurs naturally and quickly. Very few guidelines about the platform have been required. A brief description and guided visit of the platform were provided and a help page dedicated to explain how to create a multimedia mix was available on the platform. Very few questions about the technical aspect of the platform were asked during the sessions.

Most of the pupils could produce a mix withouth requiring any help.

Pupils were interviewed after each session. They confirmed the observation that the platform was technically simple to undertake. They also expressed their interest for the whole concept and the proposed activities were effectively engaging them in classical music appreciation. For example, they noticed that they listened attentively to each music piece they have pre-selected to really feel the emotion that each one means to them before integrating one inside a montage.

Pupils were also requested to fill in a form with a few questions about the usability and ergonomy of the platform. The could also freely express their feeling about the experiment. We have selected a few of the comments we have collected, that we have translated here to give an overview of how they have experienced the purpose of the project:

• “This platform is very interesting, it allows us to learn to listen to classical music, that we do not usually when we watch movies. We can see the importance of classical music in our emotions and it rightly accentuates the emotion we feel. Indeed, according to the music we put on some images, anxiety, for example, can be emphasized or otherwise, be lowered.”

• “This is very informative and fun to do ... we realize the importance of music in films, while we usually do not notice that without the music of the film is very different emotion ...”

• “It is interesting to make your own mix to show your own feelings. This platform is a great idea to raise awareness of classical music and be aware of basic music in movies that usually we do not particularly care.”

• “I found it interesting to work on music related films. I think we do not pay enough attention, but many of the emotions we feel during a movie are caused by the music we hear.”

IV. CONCLUSION This paper describes the motivation, the design, the

implementation and the tests of the Musimage platform. It investigates how social multimedia can support pedagogical activities that stimulate pupils’ creativity and engage them in classical music appreciation. The approach relies on the experience, background and skills of pupils: emotions, popular movies and the Web 2.0 social multimedia. The first tests qualitatively validate the main hypotheses that have motivated the project. Further investigations and evaluation are required in order to develop a full-featured social learning environment.



Fig. 5. An example of a multimedia mix composed of four still pictures with

transitions and the Adagio Assai of Maurice Ravel's Piano Concerto in G major as soundtrack to express the hapiness emotion

REFERENCES [1] D. Robert, « Online Music Education—Elearning Development Tools »,

EzineMark.com, 2010. [2] J. Morris, « Sounds in the cloud: Cloud computing and the digital music

commodity », First Monday, vol. 16, no 5, avr. 2011.

[3] P. Wikström, The music industry: music in the cloud. Cambridge: Polity, 2009.

[4] Q. Virjee, « Ubiquity in the cloud provided for by Spotify et al. », indianelectronica.com, 2009. [Online]. Available at: http://indianelectronica.com/blog/251109/future-music-ubiquity-cloud-provided-spotify-et-al. [Consulté le: 07-août-2013].

[5] 1 J. Savage, “Reconstructing music education through ICT”, Research Education, no. 78, pp. 65-77, 2007.

[6] 2 M. Sedlá ek, “The multimedia technologies applications in music education”, in Proc. of the XXV music teacher Conf. Musica Viva in Schola, Masaryk University, 2010.

[7] 3 J. Savage, “Information communication technologies as a tool for re-imagining music education in the 21st century”, International Journal of Education & the Arts, vol. 6, no. 2, 2005.

[8] 4 T. Dillon, “Future music: investigating the role of technology in enhancing public appreciation of and participation in music. An Insight Paper”, Available from: http://www.immagic.com/eLibrary/ARCHIVES/GENERAL/FUTRLBUK/F060307D.pdf

[9] 5 S. Wise et al., “Teachers' use of digital technology in secondary music education: illustrations of changing classrooms”, British Journal of Music Education, vol. 28, pp. 117-134, 2007.

[10] 6 W.-C. Ho, “Students’ experiences with and preferences for using information technology in music learning in Shanghai’s secondary schools”, British Journal of Educational Technology, vol. 38, no. 4, pp. 699–714, 2007.

[11] 7 S. A. Ruthmann, “Music learning and new media in virtual and online environments”, in Oxford Handbook of Music Education, G. McPherson and G. Welch (Eds.), New York: Oxford University Press, 1999.

[12] 8 V. K. Vratulis, “Exploring literacy pedagogy with digital technologies in teacher education”, PhD dissertation, University of British Columbia, 2010.

[13] M. Rechberger, « Music and Arts E-Learning Graz », présenté à The International Conference on E-Learning in the Workplace, New-York, 2009.

[14] Y. Zhou, G. Percival, X. Wang, Y. Wang, et S. Zhao, « MOGCLASS: a collaborative system of mobile devices forclassroom music education », in Proceedings of the international conference on Multimedia, New York, NY, USA, 2010, p. 671–674.

[15] R. Orpheus, « Future scenarios: music, education, social networks and Web 2.0 ». 2007. [Online]. Available at http://netsoundsproject.eu/know-how/papers/future-scenarios-music-education-social-networks-and-web-20.

[16] 9 S. A. Ruthmann, “Strategies for supporting music learning through online collaborative technologies, music education with digital technology”, in Music Education and Technology, Continuum International Publishing Group, pp. 131–42, 2007.

[17] 10 J. A. Bugos et al., “Podcasting: a method of enhancing course perceptions and performance in music appreciation”, International journal of instructional technology & distance learning, 6(1), pp. 37-46, 2009.

[18] F. Ballanti et G. Fiocchetta, « Report on Good Practices and Methodologies for Music Education through ICT », Anicia srl, 2011.

[19] 11 J. C. Rosevear, “Engaging adolescents in high school music”, PhD dissertation, University of Adelaide, Elder Conservatorium of Music, 2008.

[20] R. Bennett. "New brain research on emotion and feeling: Dramatic implications for music education." Arts Education Policy Review, 106, pp. 21-30, 2004.

[21] J. Robinson, "Deeper than reason: Emotion and its role in literature, music, and art", Oxford: Clarendon Press, 2005.

[22] D. J. Elliott, « Musical Understanding, Musical Works, and Emotional Expression: Implications for education », Educational Philosophy and Theory, vol. 37, no 1, p. 93–103, 2005.

[23] 12 Ekman, P., "Basic emotions", in Handbook of cognition and emotion, T.Dalgleish & M. Power (Eds.), Sussex, U.K.: John Wiley & Sons, Ltd, pp. 45-60, 1999.



[24] 13 I. Cohen, N. Sebe, A. Garg, L. S. Chen, et T. S. Huang, « Facial expression recognition from video sequences: temporal and static

modeling », Computer Vision and Image Understanding, vol. 91, no 1-2, p. 160-187, juill. 2003.



Documents

[IEEE 2013 International Conference on Interactive Collaborative Learning (ICL) - Kazan, Russia (2013.09.25-2013.09.27)] 2013 International Conference on Interactive Collaborative