View
214
Download
0
Category
Preview:
Citation preview
1/20/2017
1
ELEC-E5620 - Audio Signal Processing, Lecture #1
Vesa Välimäki
History and Future of Audio Signal Processing
January 20, 2017
0. General issues (Vesa & Fabian) 13.1.2017
1. History and future of audio DSP (Vesa) 20.1.2017
2. Digital filters in audio (Vesa) 27.1.2017
3. Audio filter design (Vesa) 3.2.2017
4. Analysis of audio signals (Vesa) 10.2.2017
No lecture (Evaluation week for Period III) 17.2.2017
5. Audio effects processing (Fabian) 24.2.2017
6. Synthesis of audio signals (Fabian) 3.3.2017
7. 3-D sound (Prof. Ville Pulkki) 10.3.2017
8. Physics-based sound synthesis (Vesa) 17.3.2017
9. Sampling rate conversion (Vesa) 24.3.2017
10. Audio coding (Vesa) 31.3.2017
Course Schedule in 2017 (Periods III, IV)
January 20, 20172
Vesa Välimäki
1/20/2017
2
Outline
• The most important events in the history of DSP and its applications to audio, acoustics, and music• Publications
• Inventions
• Products
• Standards
• Some predictions of the future
January 20, 2017Vesa Välimäki3
Old Times
• Alexander Graham Bell, , 1876
• Elisha Gray, ‘musical telegraph’, 1876
• T. A. Edison, , 1877-
• Thaddeus Cahill, telharmonium, 1897...1906
• US commissioner of patents (1899): “Everything that can be invented has been invented”
• Lee De Forest, triode vacuum tube ( ), 1906
• Leon Theremin, theremin, 1919
• Rice & Kellogg, dynamic , 1925
• Laurens Hammond, electric organ, 1929...1935
January 20, 2017Vesa Välimäki4
1/20/2017
3
Phonograph
• Invented by Edison in 1877
• Birth of sound recording
• Storage of sound as a mechanical waveform
Source: http://www.americaslibrary.gov/pages/aa_edison_phonograph_1.html
January 20, 2017Vesa Välimäki5
Theremin
• Air violin based on two antennas
• Amplitude and frequency control
• Heard on soundtracks of old sci-fi films and in pop music
January 20, 2017Vesa Välimäki6
1/20/2017
4
1948: Birth of Signal Processing
• Claude Shannon, “A mathematical theory of communication,” Bell System Tech. Journal, 1948
• Oliver, Pierce & Shannon, “The philosophy of PCM,” 1948 (PCM = Pulse Code Modulation)
• Richard W. Hamming, error-correcting codes, 1948
• Bell Telephone Lab: transistor, 1948
• Pierre Schaeffer: musique concrète, 1948
• IRE Audio Group was founded by Leo Beranek & others
• Since 1963: IEEE Signal Processing Society
• Audio Engineering Society was founded in 1948
January 20, 2017Vesa Välimäki7
1950s
• Ragazzini, Zadeh: Z transform, 1952
• Texas Instruments: development of a digital processor for seismic data, 1956
• First graphic equalizer Langevin Model EQ-251A
• Stereo recording, 1956
• Sony: transistor radio TR-63, 1957
• Max Mathews: computer music experiments, 1957-• Using the world’s first D/A converter (12 bits)
• Kilby, Noyce: integrated circuit, 1959
January 20, 2017Vesa Välimäki8
1/20/2017
5
Sony TR-63 Transistor Radio
• The 1st transistor radio
• Popular during 1950’s
• Birth of personal electronics!• Pocket calculator
• Walkman
• Mobile phone
• MP3 player
• PDA
• Tablet computer
• etc.http://www.geocities.com/SoHo/Atrium/1031/trans/cool/Sonys.html
January 20, 2017Vesa Välimäki9
1960s: Early DSP Algorithms
• Manfred Schroeder: digital reverberation, 1961
• Atal & Schroeder: 3-D sound with 2 loudspeakers, 1962
• Moog: , 1964
• James Cooley & John Tukey: , 1965
• J. Flanagan & R. Golden: phase vocoder, 1965
• The first DSP text book by Gold & Rader, 1969
January 20, 2017Vesa Välimäki10
1/20/2017
6
3D Sound Using a Loudspeaker Pair
Patent by Schroeder & Atal, 1963
• Hear sound from any direction with just two speakers
• Within a restricted listening area
• Invention: cross-talk cancellation! Each ear receives only one
loudspeaker signal, like in binaural listening
Implemented with digital filters
January 20, 2017Vesa Välimäki11
Moog Synthesizers
• Analog electronics
• Subtractive synthesis Periodic waveforms and
filters
• Popular in pop and rock music and in commercials Wendy Carlos: Switched-on-
Bach (1968)
January 20, 2017Vesa Välimäki12
1/20/2017
7
1970s: Practical Applications
• Bishnu Atal: Linear predictive coding, 1970
• Blesser & Lee, real-time digital delay unit, 1971
• BBC: Digital sound recording started in 1971
• E. Kurenniemi: synthesizer DIMI-600, 1972
• Chowning: , 1973
• Stockham: Restoration of old Caruso recordings, 1975
• Digital audio recording for consumers U-matic video cassette + a Sony PCM-1600 adapter, 1978
• First digital reverb: EMT Model 250, 1976
• First samplers: Synclavier, 1975; Fairlight CMI, 1978
Source of figure: http://mixonline.com/TECnology-Hall-of-
Fame/1976-EMT-reverb/
January 20, 2017Vesa Välimäki13
Fairlight Computer Music Instrument
• Early digital synthesizer and sampler (1980-1984) Included a sequencer
• Expensive (30000 USD)
• Used in film industry and pop music in 1980s E.g. Kate Bush, Thomas
Dolby, Peter Gabriel, Jan Hammer, Stevie Wonder
Fairlight CMI IIx (1982)
(source: www.obsolete.com/120_years)
January 20, 2017Vesa Välimäki14
1/20/2017
8
Fairlight CMI Preset Sounds
• Typical samples– Drums
– A ”trained” dog
– Human voices
– Orchestra
– Sounds from nature
(source: www.obsolete.com/120_years)
January 20, 2017Vesa Välimäki15
Synclavier
• Manufactured by New England Digital (USA 1976-1992)
• Digital synthesizer and sampler Additive and FM synthesis
Spectrum analyzer
• Extremely expensive
• Used in the film and pop music industry E.g., Michael Jackson, Sting,
Frank Zappa
(source: www.obsolete.com/120_years)
January 20, 2017Vesa Välimäki16
1/20/2017
9
1980s: Audio DSP Became Mainstream
• Texas Instruments, Intel, NEC, Motorola: first digital signal processors, 1980-
• Sony: Walkman cassette player, 1980
• Philips & Sony: , 1982
• Musical Instrument Digital Interface, MIDI, 1983
• Popular digital music synth: , 1983
• Sony: Digital Audio Tape (DAT), 1984
• First digital mixing consoles, 1986
• Ingrid Daubechies: wavelets, 1988
January 20, 2017Vesa Välimäki17
CD
• Developed by Philips and Sony since late 1970s
• Brought audio DSP into homes since mid-1980s
• Contains much digital signal processing Decoding, error correction, oversampling, digital filters, DA converters
Sony prototype (1981) Sony CDP-101 (1982)
January 20, 2017Vesa Välimäki18
1/20/2017
10
Yamaha DX-7 Music Synthesizer
• The first affordable digital music synthesizer (2000 €)
• FM synthesis only Good for metallic, clear sounds, such as electric piano, tubular bells, celesta etc.
• Huge commercial success in mid-1980s
(source: www.obsolete.com/120_years)
January 20, 2017Vesa Välimäki19
1990s: Multi-Channel Sound and MPEG
• International standards by Moving Picture Experts Group (MPEG) MPEG-1, MPEG-2, and MPEG-4 standards for video and audio coding MPEG-1 layer 3 = MP3, 1990; popular since late 1990’s
• New music players: Philips DCC, 1992-1996; Sony MiniDisk, 1992-• Digital multichannel sound for movies and home theater (5.1), 1992• 3-D audio reproduction using HRTFs, ca. 1992• First applications of active noise control
Saab 2000 aircraft, Digisonix air-conditioning systems, expensive cars...
• Physical modeling synthesis: Yamaha VL-1, 1994• Digital audio with 24 bits at 96 kHz • First MP3 players, 1998• Auto-tune for the singing voice, 1998• European digital radio (Digital Audio Broadcasting, DAB)
January 20, 2017Vesa Välimäki20
1/20/2017
11
Korg Prophecy Synthesizer
• A physical and analog modeling synthesizer (1995)
January 20, 2017Vesa Välimäki21
2000-2009: Mobile Audio
• New disk formats: Super Audio CD, DVD Audio, Blu-ray
• New audio coding techniques WMA, mp3PRO, AAC+, AMR-WB+, and about 100 others
• Apple iPod music player, 2001
• MPEG-7 standard, “Multimedia Content Description Interface”, 2001 How to search and manage digitized audiovisual content
Requires high-level audio processing (sound recognition, classification,…)
• MPEG-21 standard, “Networked Multimedia Framework” Managing the whole multimedia delivery and consumption chain
• Nokia: Polyphonic ring tones, 2002
• Sony-Ericsson: Music phones, 2005
• Music games: GuitarHero 2005-, RockBand 2007-
• More music files sold than CDs, 2008-
January 20, 2017Vesa Välimäki22
1/20/2017
12
2010-2017: What Happened
• Your turn: Try to remember or grasp!
January 20, 2017Vesa Välimäki23
Current Audio DSP Applications
• Phones, music & media players (DVD, MP3)
• Loudspeakers & headphones
• Computers & gaming Music video games!
• Music synthesizers & effects boxes
• Digital radio (DAB) & TV broadcasting
• Cinema, home entertainment & car audio
• Hearing aids
• Professional audio
• Active noise control in headsets and cars
January 20, 2017Vesa Välimäki24
1/20/2017
13
Turning Points in Audio History
• Birth of electroacoustics (ca. 1925) Microphone, loudspeaker, amplifier, radio,
sound recording, talking movies, PA systems
• Solid state electronics (ca. 1955) Transistors and diodes replace tubes, audio
electronics becomes affordable and small
• Digital audio (ca. 1980) Sound becomes data CD, DAT, MP3
• Virtual audio technology (ca. 1992) Virtual audio reality, interaction, immersion,
simulations, spatial sound reproduction using loudspeakers or headphones
• Major changes in 15-30 years
January 20, 2017Vesa Välimäki25
Future of (Music) Listening
• Music like water Wireless internet with solved billing routine
• Integration of phone, music player & hearing aid Audio entertainment and information, binaural
telephony, aids for handicapped, or superman’s hearing
• Intelligent music companion Query device, automatic music identification,
personalization, interactive disco, social networks
• New reproduction methods Headphones with haptics, bonephones, augmented-
reality glasses, brain-computer interface
January 20, 2017Vesa Välimäki26
1/20/2017
14
Future of Audio Signal Processing
• Many devices will incorporate a processor, a microphone, and a loudspeaker
• More intelligent processing of audio signals Auditory scene analysis, source separation, etc.
• Digitization of audio technology will continue Digital (IP) loudspeakers and class-D amplifiers Software versions of all effects boxes and instruments
• New MPEG Interactive Music Applications Format (IMAF) standard, 2009 Enables personalized mixes of music
January 20, 2017Vesa Välimäki27
Acoustic Telepresence
• You need a stereo headset with mic, wireless full-duplex connection, and 3-D sound processing Requires a signal processor Pseudo-acoustic environment
= normal + transmitted sounds
• Many new applications Virtual tourist guide (requires GPS) Virtual and synthetic sound objects
(e.g., sound calendar, audio post-it notes) Hearing aid, hearing enhancer (hearwear) Hearing protection Voice control
Bidirectionalbinaural headset
Ref: Härmä et al., 2004
January 20, 2017Vesa Välimäki28
1/20/2017
15
Grand Challenges for Audio DSP
• Compare to grand challenges in other fields A chess program that beats humans – achieved The translating telephone – in the making
• Natural-sounding music synthesis Parametric modeling of musical instruments
• Computer imitation of famous singers For example Elvis
• Unmixing of audio files For example, reconstruct multi-track versions of Beatles songs
• A computer that understands music like humans
January 20, 2017Vesa Välimäki29
Bibliography
• D.A. Bohn, Operator adjustable equalizers: An overview. In Proceedings of the Audio Engineering Society 6th International Conference, pp. 369–381, 1988.
• Joel Chadabe, Electric Sound: The Past and Promise of Electronic Music, Prentice-Hall, 1997.
• Aki Härmä et al., “Augmented reality audio for mobile and wearable appliances,” Journal of the Audio Engineering Society, June 2004.
• Mark Kahrs (editor), “The past, present, and future of audio signal processing,” IEEE SP Magazine, Sept. 1997.
• Stanley P. Lipshitz, “Dawn of the digital age,” Journal of the Audio Engineering Society, Jan./Feb. 1998.
• Frederik Nebeker, Signal Processing: The Emergence of a Discipline, 1948 to 1998, IEEE History Center, 1998.
• Peter Noll, “MPEG digital audio coding,” IEEE Signal Processing Magazine,1997.• Curtis Roads, “Interview with Max Mathews,” Computer Music Journal, vol. 4, no. 4,
winter 1980. • Mark Vail, Vintage Synthesizers, Miller Freeman, 1993.
January 20, 2017Vesa Välimäki30
1/20/2017
16
Links
• History according to the Audio Engineering Societyhttp://www.aes.org/aeshc/docs/audio.history.timeline.html
• History of electronic musical instrumentshttp://120years.net/
• A. Hutchinson, “The Top 50 Inventions of the Past 50 Years,” Popular Mechanics, Dec. 2005. http://www.popularmechanics.com/technology/gadgets/news/2078467
• C.E. Shannon, “A mathematical theory of communication,” Bell System Technical Journal, 1948.http://onlinelibrary.wiley.com/doi/10.1002/j.1538-7305.1948.tb01338.x/abstract
January 20, 2017Vesa Välimäki31
0. General issues (Vesa & Fabian) 13.1.2017
1. History and future of audio DSP (Vesa) 20.1.2017
2. Digital filters in audio (Vesa) 27.1.2017
3. Audio filter design (Vesa) 3.2.2017
4. Analysis of audio signals (Vesa) 10.2.2017
No lecture (Evaluation week for Period III) 17.2.2017
5. Audio effects processing (Fabian) 24.2.2017
6. Synthesis of audio signals (Fabian) 3.3.2017
7. 3-D sound (Prof. Ville Pulkki) 10.3.2017
8. Physics-based sound synthesis (Vesa) 17.3.2017
9. Sampling rate conversion (Vesa) 24.3.2017
10. Audio coding (Vesa) 31.3.2017
Course Schedule in 2017 (Periods III, IV)
January 20, 201732
Vesa Välimäki
Recommended