1

Music, Hearing Loss, andCochlear ImplantsThe Next Frontier

Charles J. Limb, M.D.

Francis A. Sooy ProfessorChief of Otology, Neurotology and Skull Base SurgeryDirector, Douglas Grant Cochlear Implant CenterDepartment of Otolaryngology-Head and Neck Surgery and Neurological Surgery

University of California San FranciscoSan Francisco, CA

Disclosures

• Advanced Bionics Corporation– Consultant (2006-present)– Research support

• Med-El Corporation– Research grants (2012-present)– Research support (2016)

• Oticon Medical– Conference chair/consultant, cochlear implants and music (2016)

• Frequency Therapeutics, Inc.– Consultant (2016)

2

3

Treatment for hearing loss has evolved

1800s 2000s

Methods

Secondary Reconstruction

4

Why music?

5

First musical instrument?Bird bone flute

Recovered from southern Germany (Giessenklösterle)~35,000 years old

Patel, et al., 2009 Current

‘Snowball’ gets his groove on

Patel, et al., 2009 Current

‘Snowball’ gets his groove on

6

Despite success in language perception, most cochlear implant users

cannot hear music well.

500-1500 Hz BPF

unfiltered stimuli

Percentage Correct Doesn’t Work For Music!

7

8

What does music sound like for a CI user?

In normal hearing, place and rate pitch mechanisms are seamlessly integrated and support one another.

In electric hearing, place and rate pitch mechanisms are both disrupted.

Difference between middle C and one semitone above or below ~15 Hz

Pitch perception is the single biggest obstaclefor CI-mediated perception of music

Relative pitch > absolute pitch

9

PURE TONES REAL TIME SPECTROGRAM

SUPERSTITION - REAL TIME SPECTROGRAM

10

11

Typical CI frequency mapcovers range of 60+ semitones

Rachmaninoff Prelude Op. 3, No. 2 in C# minor: original

Rachmaninoff Prelude Op. 3, No. 2 in C# minor: original

12

Rachmaninoff Prelude Op. 3, No. 2 in C# minor: +/- 1 s.t.

Rachmaninoff Prelude Op. 3, No. 2 in C# minor: +/- 1 s.t.

Caldwell M. et al, 2015.

13

CI Users Utilize Tempo Rather Than Modeto Interpret Musical Emotion

Caldwell M. et al, 2015.

14

Caldwell MT et al. 2016

15

NH Confusion MatrixPresented Identified

Cnt CorClass. %

Single Interval Chord

Single260

90.3%24

8.3%2

0.7%

Interval63

21.9%174

60.4%53

18.4%

Chord23

8.0%122

42.4%143

49.7%

CI Confusion MatrixPresented Identified

Cnt CorClass. %

Single Interval Chord

Single199

69.1%63

21.9%26

9.0%

Interval136

47.2%84

29.2%67

23.3%

Chord112

38.9%86

29.9%89

30.9%

Donnelly et al., 2009

CI Users Demonstrate Fusion of Polyphonic Pitch

16

How do we measure musicalsound quality in CI users?

Musical Sound Quality is Poorin CI Users

• CI users subjectively report poorer musical sound quality following implantation (Gfeller et al 2002; Lassaletta et al 2008)

• Sound quality traditionally assessed via questionnaires or rating scales (Lassaletta et al 2008; Gfeller et al 2008; Looi et al 2008, 2011)

• Assessment of sound quality can be used as a tool, not an indicator of preference or enjoyment

17

18

Cochlear Implant-MUltiple Stimulus with Hidden Reference and Anchor

(CI-MUSHRA)

• 25 full quality musical stimuli are increasingly degraded:– Example: Removal of bass frequencies

– Hidden reference (best sound quality)

– 200 Hz HPF

– 400 Hz HPF

– 600 Hz HPF

– 800 Hz HPF

– 1000 Hz HPF

– Anchor (1000-1200Hz band-pass filter)

No Alterations

Highly Altered

19

Roy et al., 201225 excerpts, from 5 genres (pop/rock, country, jazz, classical, hip hop)

CI-MUSHRA

MUSHRA: ITU-R Recommendation BS.1534For evaluation of lossy audio compression algorithms

Roy et al., 2012

20

How do we improve music for CI users?

• Change the CI• Change the brain• Change the music

Can we tune cochlear implants?

21

FPCT distinguishes all 216 individual electrode contacts

Jiam NT et al. Otol Neurotol

2016 (In press)

Frequency-place mismatch increasesin apical and basal electrodes

Jiam NT et al. Otol Neurotol 2016 (In press)

22

Kretzmer, E. A. et al. Arch Otolaryngol Head Neck Surg 2004;130:499-508.

Kretzmer, E. A. et al. Arch Otolaryngol Head Neck Surg 2004;130:499-508.

A radiograph of a cat with a cochlear implant shows the 6 electrode contacts(electrode 1 at white arrowhead) within the cochlea

Courtesy of the Ryugo laboratory,Center for Hearing Sciences

23

Parsons LM, 2001

PET is quiet, non-magnetic

Positron Emission Tomography

Positron-emitting [15]O atom

H2-[15]O molecules

Positron Emission Tomography

24

CI listeners

Normallisteners

Is CI listening similar to normal listening?

Limb et al., 201010 postlingual CI users / 10 controls

25

Roy et al., 2014

26

Conclusions

• Music is the pinnacle of hearing

• Critical impairments in pitch, timbre and sound quality are observed in CI users

• These impairments reveal limitations of CI processing that speech testing does not

• Cochlear implants are severely out of tune

• We must work to improve CI designs, processing strategy, musical training, and even the music itself

Thank You• Johns Hopkins University School of Medicine

• The Dana Foundation

• NIDCD/NIH

• The Brain Science Institute

• National Endowment for the Arts

• Peabody Conservatory of Music

• University of California San Francisco

• Baltimore Symphony Orchestra

• San Francisco Jazz Center

• San Francisco Conservatory of Music

• Advanced Bionics Corporation

• Cochlear Corporation

• Med El Corporation

• Oticon

27

Thank You

• Judy Doong• Alice He

• Nicole Jiam• Tina Munjal

• Meredith Caldwell• Patpong Jiradejvong

• Joseph Heng• Mickael Deroche

• Gabe Donnay• Stephen Dunlap

• Diane Hwang• Irene Kim

• Matthew Sachs• Karen Barrett Chan

• Nicholas Ryugo• Gabriela Cantarero

• Monica Lopez-Gonzalez• Lindsay Scattergood

• Jonathan Zwi• Juan Huang

• Michael Williams• Patrick Donnelly• Summer Rankin

• Malinda McPherson• Andrew Landau

• Fred Barrett• Alexis Roy

• Melanie Gilbert

Music, Hearing Loss, andCochlear ImplantsThe Next Frontier

Charles J. Limb, M.D.

Francis A. Sooy ProfessorChief of Otology, Neurotology and Skull Base SurgeryDirector, Douglas Grant Cochlear Implant CenterDepartment of Otolaryngology-Head and Neck Surgery and Neurological Surgery

University of California San FranciscoSan Francisco, CA