A Musical System For Emotional Expression

António Pedro OliveiraUniversity of Coimbra, Portugal

2

Outline of the Presentation

Introduction Background Emotion-Driven Music Engine

(EDME) Conclusion

3

Outline of the Presentation

Introduction Background Emotion-Driven Music Engine

(EDME) Conclusion

Motivation Emotions are widely accepted as being

an important factor in the society Music is almost everywhere and it is a

powerful stimulus capable of influencing our emotions

Computational systems with the capability of producing music with an appropriate emotional content have an enormous application potential

4

Introduction Background EDME Conclusion

Aim

Conceive a computational system for the control of the emotional content of produced music, so that it expresses a given emotional specification

Music: solely instrumental

5

Introduction Background EDME Conclusion

6

Outline of the Presentation

Introduction Background Emotion-Driven Music Engine

(EDME) Conclusion

Emotional Expression with Music

Four approaches: Music Transformation Music Composition Music Selection/Classification Hybrid Approaches

Our approach: Hybrid that consists in combining

selection/classification with transformation

7

Introduction Background EDME Conclusion

8

Outline of the Presentation

Introduction Background Emotion-Driven Music Engine

(EDME) Conclusion

Architecture – Offline Stage

9

Introduction Background EDME Conclusion

Architecture – Online Stage

10

Introduction Background EDME Conclusion

Experiments Initial phase

11

Introduction Background EDME Conclusion

Initial phase Manually Built Knowledge Base (short

version) Happy music: high loudness, major scale Sad music: violin, slow tempo Activating music: high loudness, fast

tempo Relaxing music: low loudness, slow tempo

12

Introduction Background EDME Conclusion

Experiments Stages of the experiments

13

Introduction Background EDME Conclusion

Experiments First Experiment - Preliminary

Evaluation of the Classification Module Hypothesis: There is a small amount of

features that may predict arousal/valence Valence: 2 features, CC – 0.76 Arousal: 4 features, CC – 0.77

14

Introduction Background EDME Conclusion

Experiments

Second experiment (two parts) First part - Extended Evaluation of the

Classification Module Hypothesis: There is a small amount of

features that may predict arousal/valence Valence: 4 features, CC – 0.70 Arousal: 3 features, CC – 0.77

15

Introduction Background EDME Conclusion

Experiments Second part - Analysis of Audio

Features Hypothesis: There are audio features

emotionally-relevant Valence: Spectral Sharpness and Loudness

are important Arousal: Spectral Similarity, Spectral

Dissonance and Spectral Sharpness are important

16

Introduction Background EDME Conclusion

Experiments

Third experiment (three parts) First part - Improvement of the

Classification Module Hypothesis: There is a small amount of

features that may predict arousal/valence Valence: 5 features, CC – 0.69 Arousal: 3 features, CC – 0.71

17

Introduction Background EDME Conclusion

Experiments Second part – Evaluation of the Transformation

Algorithms Hypothesis: It is possible to change musical features to

transform emotional content High positive coefficients for tempo were confirmed The increase of register correlates positively with valence

and negatively with arousal Some of the features can be helpful in finding scales more

appropriate to some emotions Instruments are essentially relevant to the arousal Change from normal to staccato articulation has a

correlation with the increase of valence

18

Introduction Background EDME Conclusion

Experiments Third part – Melodic Analysis

Hypothesis: The analysis of the melodic line alone turns the emotionally-relevant features more visible

19

Introduction Background EDME Conclusion

Emotional Dimension CC Features

Valence – data of first experiment 0.79 5 features

Valence – data of second experiment 0.62 5 features

Valence – data of third experiment 0.41 4 features

Arousal – data of first experiment 0.85 4 features

Arousal – data of second experiment 0.72 4 features

Arousal – data of third experiment 0.54 3 features

Calibration and Validation

Hypothesis: 13 features are enough to discriminate emotional expression

Valence: 7 features, CC – 0.85 Arousal: 6 features, CC – 0.83

Use SAM to obtain emotional answers Controlled environment Statistical analysis:

System’s classification and subject’s classification are probably measuring the same concept

20

Introduction Background EDME Conclusion

21

Outline of the Presentation

Introduction Background Emotion-Driven Music Engine

(EDME) Conclusion

Contributions The system proposed has the

advantage of being able to produce outputs of acceptable quality quite independently from the music base

It is also quite flexible: the music base can be completely redefined to adapt to the specific needs of a given scenario

22

Introduction Background EDME Conclusion

Contributions The system is also reliable, thanks to

the experimental calibration using different subjects

We adopted both the discrete and dimensional representation of emotions

We used techniques of human emotional recognition for validation and calibration of the system

23

Introduction Background EDME Conclusion

Future Work

Emotion-Driven Music Composition Artificial Intelligence approaches

Test in applications contexts Healthcare Entertainment

24

Introduction Background EDME Conclusion

The End

25