"Dude, Where's My... Signals and Systems Textbook?"

• Joseph PiconeInst. for Signal and Info. ProcessingDept. Electrical and Computer Eng.Mississippi State University

• Contact Information:Box 9571Mississippi State UniversityMississippi State, Mississippi 39762Tel: 662-325-3149Fax: 662-325-2298Email: picone@isip.msstate.edu

IEEE: MSU STUDENT BRANCH

• URL:http://www.isip.msstate.edu/publications/seminars

INTRODUCTIONABSTRACT

A funny thing happened on the way to an IEEE section talk at IIT in 1980... I found a career. I'll relate the rest of this story, including a couple of nice demos during this talk (hint: her name is Julie). I'll review some of the early history of digital signal processing and describe some of the speech applications that transformed this field into one of the single largest semiconductor markets today. I will show how signal processing research elegantly combines mathematics, software, and computer hardware to produce innovative technology such as cell phones and computers enhanced with speech recognition. Oh yeah, and I'll try to explain why we choose to torture you with a class called Signals and Systems.

Joseph Picone is currently a Professor in the Department of Electrical and Computer Engineering at Mississippi State University, where he also directs the Institute for Signal and Information Processing. He has previously been employed by Texas Instruments and AT&T Bell Laboratories. Dr. Picone received his Ph.D. in Electrical Engineering from Illinois Institute of Technology in 1983. He is a Senior Member of the IEEE and a registered Professional Engineer.

CASE STUDIES

• Introduced at the Summer Consumer Electronics Show in Chicago

• First commercial speech synthesis consumer toy

• Based on linear prediction• Contained a proprietary speech

synthesis chip

SPEAK & SPELL™ (JUNE 1978)

• Left to right: Gene Frantz, Richard Wiggins, Paul Breedlove and Larry Brantingham (1978)

CASE STUDIES

• Yes / no / true / false recognizer

• Answer questions about history

• Variety of learning modules

• Speaker independent recognition

• Microphone + children???• Won several industry design awards

for the mechanical design

VOYAGER™ (JUNE 1988)

CASE STUDIES

• Worlds of Wonder approach TI in September of 1988.

• Can you put this toy on the market by Thanksgiving?

• 10-word speaker dependent isolated word recognizer

• 100 sentences for synthesis• “Transparent training”• First large-scale consumer toy

application for a DSP

JULIE (DECEMBER 1988)

CASE STUDIES

• Voice verification for calling card security

• First wide-spread deployment of recognition technology in the telephone network

• Stimulated interest in voice dialing and other user-programmable features

• Original application was obsolete before wide-scale deployment

WATSON (EARLY 1990’S)

CASE STUDIES

Jack Deller said:

• model voice with a digital all-pole filter• identify a person by their filter

coefficients• diagnose a disease by shifts in these

coefficients• recognize the words being spoken

using pattern matching techniques• maximum entropy, linear prediction,

and a few other big words I didn’t understand

... and I was hooked ...

WHY ATTEND IEEE TALKS?

CASE STUDIES

• Deller said you can replace this:

DIGITAL FILTERS

• With this:

• What are the advantages?

HUMAN LANGUAGE TECHNOLOGYSPEECH RECOGNITION RESEARCH?

• Why do we work on speech recognition?“Language is the preeminent trait of the human species.”“I never met someone who wasn’t interested in language.”“I decided to work on language because it seemed to bethe hardest problem to solve.”

• Why should we work on speech recognition?• Antiterrorism, homeland security, military applications• Telecommunications, mobile communications• Education, learning tools, educational toys, enrichment• Computing, intelligent systems, machine learning

• Commodity or liability? • Fragile technology that is error prone

HUMAN LANGUAGE TECHNOLOGYFUNDAMENTAL CHALLENGES

HUMAN LANGUAGE TECHNOLOGYSTATISTICAL APPROACHES

SPEECH RECOGNITIONBLOCK DIAGRAM OVERVIEW

Core components:

• transduction• feature extraction• acoustic modeling

(hidden Markov models)

• language modeling (statistical N-grams)

• search (Viterbi beam)

• knowledge sources

SPEECH RECOGNITIONFEATURE EXTRACTION

SPEECH RECOGNITIONACOUSTIC MODELING

SPEECH RECOGNITIONLANGUAGE MODELING

SPEECH RECOGNITIONVITERBI BEAM SEARCH

• breadth-first• time

synchronous• beam pruning

• supervision• word prediction• natural language

Traditional Output:

• best word sequence• time alignment of

information

Other Outputs:• word graphs• N-best sentences• confidence measures• metadata such as

speaker identity, accent, and prosody

SPEECH RECOGNITIONAPPLICATION OF INFORMATION RETRIEVAL

APPLICATIONSCOMMON EVALUATIONS FUEL RESEARCH

• Only elite participate• ISIP is competitive for

same level of technology

• More advanced systems use more heuristics and incremental technology

• Common evaluations:– common data/task– WER metric

• Performance improves over time (annually)

• Resource intensive

TECHNOLOGY

• Speech recognition– State of the art– Continuous speech– Large vocabulary– Speaker independent

• Goal: Accelerate research– Flexibility, Extensibility– Efficient (C++)– Easy to Use– Toolkits, GUIs

• Benefit: Technology– Standard benchmarks– Conversational speech

PUBLIC DOMAIN SOFTWARE

APPLICATIONSINFORMATION RETRIEVAL

• Metadata extraction from conversational speech

• Automatic gisting and intelligence gathering

• Speech to text is the core technology challenge

• Machines vs. humans

• Real-time audio indexing• Time-varying channel• Dynamic language model• Multilingual and cross-lingual

APPLICATIONS

• In-vehicle dialog systems improve information access.

• Advanced user interfaces enhance workforce training and increase manufacturing efficiency.

• Noise robustness in both environments to improve recognition performance

• Advanced statistical models and machine learning technology

• Multidisciplinary team (IE, ECE, CS).

CAVS: DIALOG SYSTEMS FOR THE CAR

SUMMARYTHREE THINGS TO THINK ABOUT

• Speech Recognition:• Application of machine learning technology• Requires multidisciplinary background• A grand challenge (long-term problem)

• Career Development: • Broaden your horizons!• Position yourself at the upper end of the job market

(e.g. graduate school)• Signals and Systems:

• Introduces you to a brave new world• Abstraction is essential in engineering• Competency in math and physics is critical