Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Bridging The Gap:Teaching and Assessing

Spoken English

Bridging The Gap:Teaching and Assessing

Spoken English

Carnegie SpeechCarnegie Speech

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Carnegie SpeechCarnegie Speech• Advanced products for testing & improving

speaking skills for nonnative speakers • Software company with proprietary pinpointing

technology licensed from Carnegie Mellon University and its Language Technologies Institute

• Outtake of > 20 years of language research• Marquee customers w/proven results • Installed globally in US, MX, El Salvador, India,

Poland, Romania, Costa Rica, Philippines, Korea, Singapore, and China

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Carnegie Speech’s ProductsCarnegie Speech’s Products• Applications that are effective:

– Self-paced

– With a trainer/assessor

– Windows LAN or desktop based, Web available 1Q 2008

• Carnegie Speech Assessment provides cost-effective, scalable solution for– Hiring Screens– Pre and Post-Training Assessments– QA Screens

• Native Accent provides cost-effective, scalable solutions for comprehensibility – Pronunciation Training – Refresher Training

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Customized for increased accuracy

Increases effectiveness of remediation and assessment

Identifies student’s exact errors

Adapts to individuals’ abilities and learning styles

Carnegie Speech’s Solution Carnegie Speech’s Solution

AssessmentAssessment

Tailors assessment to customer’s task

TutorTutor

Machine LearningMachine Learning

PinpointingPinpointing

ESLESL

Speech RecognitionSpeech Recognition

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Carnegie Speech AssessmentCarnegie Speech Assessment• Assessment

– 30 sentences balanced for 250 subskills of English.

– Tutorial explaining interface.– Applicant reads sentences.– Takes about 10 minutes.

• Reporting – individual and summary reports

Aggregated Assessment Report

Aggregated Assessment Report

Summary Assessment Report for All PhonesCreated on October 31, 2007

 

Name Date Articulation

Score Assessmnt

Results Comments

Annuradha March 22, 2007 27 / 38 Train Articulation - AA, EH, M, N, NG, S, T, UW, W, Y, Z

Sunil March 19, 2007 34 / 38 Pass Articulation - AA, AY, F, W

Anita March 22, 2007 32 / 38 Pass Articulation - AA, D, DH, P, T, Y

Prasad March 22, 2007 26 / 38 Train Articulation - AA, AE, AY, DH, OY, P, S, UH, UW, W, Y, Z

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Assessment Effectiveness AnalysisAssessment Effectiveness Analysis

• Carnegie Speech software and Company Linguist assessed interview candidates and employees

• Interview assessments - 94% agreement• Employee assessments - 91% agreement

Carnegie Speech Assessment Calibration Statistics

0

50

100

% of Software Classifications Matching Organization’s In-Person Classifications

CandidatesEmployees

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Native Accent TrainingNative Accent Training• Assessment

– Biased by native language– Initial assessment of student– Ongoing guidance to the curriculum manager

• Intelligent English pronunciation tutor– Phonetics– Duration– Pitch

• Multi-modal feedback– Aural– Written– Pictorial

• Reporting – individual and class

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Native Accent FeaturesNative Accent Features

• Phonemic-level assessment with 3-5 second feedback

• Detailed, comprehensible instructions for remediation, not “try again” not a waveform

• Customized curriculum for the individual for a 15-40 hour course

• Customized curriculum for native language of speaker. Supports 9 Indian languages.

• Gender-based model speakers• Objective quantitative skill measurement• Self-paced • With a trainer/assessor

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Pilot Results - IndiaPilot Results - India

• 13 hours of training on phonemes and prosody• Decreases trainer time by 64%• Delivers training at 50% of the cost

8

5

21

23

0

5

10

15

20

25

Articulation Pace and Pausing

Nu

mb

er o

f S

tud

ents

# of students whopassed pre-trainingassessment

# of students whopassed post-trainingassessment

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Native Accent for Teaching Iraqi Arabic to US Personnel

Native Accent for Teaching Iraqi Arabic to US Personnel

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Native Accent for Arabic (cont.)Native Accent for Arabic (cont.)

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Pronunciation – PhoneticsPronunciation – Phonetics

• Text is colored based on pinpointing

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

GrammarGrammar

• Multiple choice answers are crossed through when incorrect

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Sentence StressSentence Stress

• Pitch scale describes how to match pitch

Carnegie SpeechCarnegie SpeechCarnegie SpeechCarnegie Speech

Sentence Pause ReductionSentence Pause Reduction

• Word pairs appear one at a time, with the space between each decreasing each time

•Take Food

•Take Food

•Take Food