Audibility&

Speech Recognition

Chapter 6

Perry C. Hanavan, Au.D.

Speech Test Applications obtain information for counseling illustrate benefits of visual cues

determine hearing aid candidacy– or determine candidacy for CI or ALDs

predict hearing aid benefit determine when binaural aids might NOT be appropriate

– 80%, AD and 20% AS determine amplification characteristics and features

– whether high frequency beneficial demonstrate advantage of special hearing aid features demonstrate aided performance is better than undaided determine whether cognitive or APD exists

– older person may have cognitive problems, anyone may have APD demonstrate that understanding of speech is impaired

Audiologic Evaluation

Information helpful to providing AR Audibility Dynamic Range Frequency Resolution Temporal Resolution

Audiologic Evaluation

Critical consideration in helping to plan AR: Time of Onset of loss Degree of Loss Etiology Type of Loss And wealth of other factors

Audiologic Evaluation

Observations Interviews Questionnaires Otoscopic examination Pure tone results Speech recognition Immittance OAE Electrophysiologic (ECochG, ABR, MLR, LR, etc.)

Pure Tone Results

Degree of loss– normal, minimal, mild, moderate, moderately

severe, severe, profound

Type of loss– conductive– sensorineural– mixed

StimulusElements

ClinicalSignificance

ListenerVariables

Test-RestestVariability

StimulusFormat

ResponseFormat

StimulusMode

LearningEffects

SpeechTest

Factors

Within-Subject

StatisticalProcedure

Paired t-statistic

CognitiveAbility

LinguisticAbility

HearingLoss

Com-munication

Mode

Live

Recorded

SynthesizedSpeech

AlteredSpeech

AuditionOnly

VisionOnly

TactileOnly

AuditionPlus

Vision

AuditionPlus

Tactile

EquivalentLists

ManyItems

TestReliability

TestConditions

OpenSet

ClosedSet

AuditorySkill

StimulusUnits

StimulusContext

SNR

Distance/Intensity

StimulusFamiliarity

Stimulus Mode

auditory alone vision alone tactile alone auditory and vision auditory and tactile tactile and vision auditory, vision and tactile

AuditionOnly

VisionOnly

TactileOnly

AuditionPlus

Vision

AuditionPlus

Tactile

Auditory Alone

phoneme, syllable, word, phrase, sentence open, closed set high, low context cues quiet, noise – signal to noise ratio (SNR) recorded, live

Auditory Plus Vision

Speechreading enhancement– speechreading enhancement– speechreading enhancement ratio

• vision only / auditory plus vision = SE– Children –Craig Sentence and Craig Words and CHIVE

– Adults –Iowa Sentence Test and CUNY Sentences

– Erber (vision and hearing assessment)

Auditory Plus Visual Information

Sumby and Pollack (1954) demonstrated that the addition of visual speech information could significantly improve speech perception performance and that the importance of visual speech information increased as the listening situation became more difficult.

Auditory Plus Vision

Tye-Murray– CHIVE (adult)– CAVET (children)

– Audition plus vision

– Vision only

– Audition alone

Eber– Sent-Ident

Speechreading Enhancement/Visual Enhancement

Benefit from adding a visual signal to an auditory signal Refers to the benefit obtained from seeing and hearing a

speaker compared with auditory alone – Difference or ratio between speech recognition performance in an

vision-only condition and an audition plus-vision condition– Difference Formula (AV-V)– Normalized Score Formula (AV-V/100-V)

Patient A (V score=50% AV score = 75%)– 75-50=25% enhancement (difference score)– (75-50)/(100-50)=50% (normalized difference score)

Patient B (V score = 10% AV score = 55%)– 55-10=45% visual enhancement– (55-10)/(100-10)=50% (normalized difference score)

Auditory Enhancement

Benefit from adding an auditory signal to a visual only signal

Difference Formula (AV-A) Normalized Score Formula (AV-A/100-A) Patient A (A score=50% AV score = 75%)

– 75-50=25% enhancement (difference score)– (75-50)/(100-50)=50% (normalized difference score)

Patient B (A score = 10% AV score = 55%)– 55-10=45% auditory enhancement– (55-10)/(100-10)=50% (normalized difference score)

Integration Enhancement

Measure of ability to integrate auditory and visual information

AV-[100-(100-A)+(100-V)]/100-[100-(100-V)+(100-A)]

THE AUDITORY SANDWICH

Always put it back into Always put it back into hearinghearing!!

HEAR

HEAR-UNDERSTAND

SEE - SAYSEE - SAY

First, listen.

Then, if need be,watch or say it.

Then, listen again

(no visual cues)

Visual cues:• lip-reading• printed word• cued speech• signs

AUDITORY VISUAL

Back to “choices”

AuditorAuditory-y-

VerbalVerbal

AuditorAuditory-Oraly-Oral

Cued Cued SpeechSpeech

Total Total CommunicatiCommunicati

onon

ASLASL

LanguageLanguage

CommunicatiCommunicationon

CognitionCognition

FullyAuditory

Communicator

MostlyVisual

Communicator

MostlyAuditory

Communicator

FullyVisual

Communicator

AA AAvv AVAV VVAA VV

Choices reframed

FullyAuditory

Communicator

MostlyVisual

Communicator

MostlyAuditory

Communicator

FullyVisual

Communicator

AA AAvv AVAV VVAA VV

Flexibility is essential

Stimulus Elements

Units: phoneme, syllable, word, phrase, sentence, non-sense stimuli

Proximity: distant, near Intensity: soft, comfortable, loud Context: high, low context cues, SNR: quiet, background sounds--

signal to noise ratio Familiarity: high, low familiarity

with material, nonsense material

StimulusUnits

StimulusContext

SNR

Distance/Intensity

StimulusFamiliarity

Learning Effects Learning effect: familiarity with

items, procedures…not change in ability– Repeated Frame Test– Cinderella-Brahman Speech

Recognition Test Equivalent lists: lists that contain

items that are presumed to be equally difficult to recognize– PB, sentences– Repeated Frame Sentences Test

Numerous stimuli– Full lists rather than half lists

EquivalentLists

ManyItems

Stimulus Items Live vs. Recorded

– Voicing frequency– Intonation– Speech rate– Clarity of articulation– Physical characteristics

Synthesized speech Altered speech

– Time-compressed– Expanded– Filtered

Live

Recorded

SynthesizedSpeech

AlteredSpeech

Significance

Clinical significance When a small change in performance

is clinically significant When comparison between two test

results is clinically significant

Statistical design Paired t-statistic Within subject statistical procedure

Within-Subject

StatisticalProcedure

Paired t-statistic

Significant Differencefor NU-6 recorded speech tests

Listener Variables

Degree of hearing loss Cognitive abilities

– Thinking, reasoning, remembering, imagining, or learning words

Linguistic abilities– Knowledge of language

Communication mode used Multicultural

CognitiveAbility

LinguisticAbility

HearingLoss

Com-munication

Mode

Test Reliability Test reliability: the degree to which

a single test score approximates the true score

Test-retest variability: measure of consistency from one test presentation to the next

Test conditions: Variables affecting test-retest variability:– mode of presentation—live vs recorded– location—test booth vs classroom– talker—familiar vs unfamiliar, male vs

female– number times item repeated—once,

twice, etc. leads to better performance

TestReliability

TestConditions

Response Format

Closed or limited set Open set Auditory skill

– Detection– Discrimination– Identification– Comprehension

OpenSet

ClosedSet

AuditorySkill

Acclimatizationand

Brain Reorganization

Plasticity Physiological changes in the

CNS (and PNS – auditory nerve) that occurs from sensory experiences– Brain’s ability to reorganize

space– Benefit from HA, CI, HATs may

need to be measured at later date– Brain may continue to acclimate

for several years following HA, CI, HAT, therapy

Making Connections

A child is born with over 100 billion neurons or brain cells.

These neurons form connections, called synapses, which make up the wiring of the brain.

Brain Development

EARLY EXPERIENCES At age eight months an infant may have 1,000 trillion synapses.

By age 10 the number of synapses decrease to about 500 trillion.

The final number of synapses is largely determined by a child's early experiences, which can increase or decrease the number of synapses by as much as 25 percent.

Brain Development

"USE IT OR LOSE IT!" The brain operates on a "use it or lose it" principle: only those connections and pathways that are frequently activated are retained.

Other connections that are not consistently used will be pruned or discarded so the active connections can become stronger.

Brain Development

DEFINING LANGUAGE SKILLS When an infant is three months old, his brain can distinguish several hundred different spoken sounds.

Over the next several months, his brain will organize itself more efficiently so that it only recognizes those sounds that are part of the language he regularly hears.

During early childhood, the brain retains the ability to relearn sounds it has discarded, so young children typically learn new languages easily and without an accent.

Dendrites Dendrites are thin, branching

fibers lined with receptors at which the dendrite receives information from other neurons.

The greater the surface area, the greater the amount of information.

Some dendrites are covered with spines which greatly increase its surface area.

Plasticity, Adaptation, Acclimatization

Many researchers think that training techniques sometimes can help those with the reading disability, dyslexia, because they modify brain networks. The images above hint that this is the case. The top images show the brain activity (lit-up areas) of a 10 year-old boy while he completes a task that requires the ability to identify the sounds of words. His reading level equaled that of an eight-year-old child. The bottom images show his brain activity while he completes the same task after receiving eight weeks of a type of special training. Following the intervention training, his reading level increased by three years and the images indicate that his brain activity changed as well. Researchers are conducting a very large, ongoing study to confirm this one example.

Auditory AcclimatizationBack in the 1940s, wideband high-fidelity phonograph consoles were just becoming available. Because of his interest in high-quality audio, Harvey Fletcher bought one for his home. Harvey enjoyed listening to this new high-fidelity system, but unfortunately, the enjoyment was not shared by his wife. After listening to a an old 78 rpm record, with the surface noise made particularly prominent by the extended bandwidth of this new high-fidelity system, she said: "That sounds awful. I don't really like having that screechy sound in my home." Always the creative thinker, the next day, when his wife was out of the house, Harvey went into the living room and soldered twenty 1 uF capacitors across the loudspeaker terminals, rolling off the high frequencies. (Remember that amplifiers were high impedance back then, so the trick worked.) That evening, when the music played, his wife was now happy. One night each week, while his wife was sleeping, Harvey would sneak downstairs and clip one capacitor. After twenty weeks, when the music played, they were both happy.

AcclimatizationAdapting to a new environment (in this case, auditory) or as defined by Darwin, the process of inuring to a new climate, or the state of being so inured. This seems to be a reasonable term, as it also is used to describe how the human body acclimates to temperature, altitude, and other environmental conditions. From an auditory standpoint, Gatehouse was one of the first to use the term acclimatization, explaining the speech processing capabilities of a group of people aided monaurally. In later research Gatehouse used the term acclimatization to describe an improvement in speech recognition over time. Today, the term is used widely to explain adaptation to hearing aid use in general, and is not limited to the Gatehouse definition.

Adaptation

The process of adapting to something, such as environmental conditions (in this case, auditory); the responsive adjustment of a sense organ. This too is a reasonable term, as it has long been used in reference to the eye—e.g., adaptation to varying light conditions.

Sensory Reorganization

                                                                                             

                                                   

When nerve stimulation changes, as with amputation, the brain reorganizes. In one theory, signals from a finger and thumb of an uninjured person travel independently to separate regions in the brain's thalamus (left). After amputation, however, neurons that formerly responded to signals from the finger respond to signals from the thumb (right).

Auditory Reorganization/Plasticity

Cochlear dead regions

Brain reorganization will occur with damage to regions of the cochlea

Speech Recognition Tests

WIPI Word Intelligibility by

Picture Identification (WIPI)

closed-set picture-pointing (six pictures per plate)

appropriate for children whose language age is between 5 and 10-11

comprised of four 25-monosyllabic word lists

contains 26 color plates (one for practice), six pictures per page. (A, V, A-V)

NU-CHIPS NORTHWESTERN UNIVERSITY--

CHILDREN’S PERCEPTION OF SPEECH (NU-CHIPS)

closed set picture pointing word recognition test children whose language age is as low as three 50 words familiar to three-year-old children in

four randomizations includes two picture books with 50 monochrome

plates, four pictures per plate. Book A is used for forms A & B, book B for C & D.

There are two recordings, one male and one female talker.

MAC The Minimal Auditory Capabilities (MAC) (2nd Edition)

battery specifically targeted to CI patient Consists of a series of tasks which are graded in

difficulty Most of the MAC battery sub-tests assume patient’s

hearing loss has occurred post-lingual, but can be employed as a means of evaluating the hearing abilities of persons for whom traditional speech materials are too difficult

Second edition has been standardized. The recorded materials include gross sound

identification, inflection detection, contrast detection, accent discrimination, and word identification

14 sub-tests, 13 audio and one video

SERT SOUND EFFECTS RECOGNITION TEST

(SERT) developed for those instances where conventional word recognition measures are not appropriate, such as when language limitations due to hearing impairment

Certain children who are unable to recognize even simple speech can perceive correctly environmental sounds to which they are exposed in their daily lives

Under these circumstances, the SERT can provide valuable information about the integrity of the auditory system.

Closed set, picture-pointing tasks 10 sounds plus a practice sound

CID Every Day Sentences EVERYDAY SPEECH 10 sets of 10 sentences each with 50

"target" words in each set for word recognition assessment under contextual conditions

Can be employed in auditory training Sentences vary in length and are spoken

with minimal inflection Normative data on recording lacking

BKB Sentences

Bench, Koval, & Bamford (BKB) Open set sentences Appropriate for linguistic abilities of most 8-15 years of

age with hearing loss Lists of 16 simple sentences, including 50 key words were

devised to include vocabulary, grammar and sentence length for 8-15 year olds

The sentences are presented in an open-set format and the child imitates as much of the sentence as possible. Responses are recorded word-for-word and scored by percent of key words correctly repeated.

NU-6

NORTHWESTERN UNIVERSITY AUDITORY TEST NUMBER SIX (NU-6)

Phonetically balanced CNC monosyllabic open set word recognition test

Four lists of 50 words each recorded in four randomizations

Talker has a General American dialect Standardized

Auditory Numbers Test ANT Test helps identify tactile from auditory

listeners Word closed set recognition auditory alone

test Appropriate for 3-8 year old children with

severe to profound hearing loss Simple auditory alone test to measure

ability to perceive simple auditory cues– (Erber, 1980)

Iowa Consonant Confusion Test Closed set consonant (phoneme) recognition test test can also be analyzed in terms of the listener's

ability to identify phonetic features:– Chance performance for consonant voicing, manner, and

place of articulation identification is 50%, 33%, and 20% respectively

– Example: Mr. S achieved a total score of 79% correct, 96% on voicing, 94% on manner, and 85% on place

10 consonants presented 12 times in VCV context– p, t, k, b, d, g, v, z, n, m– Presentation examples: aba, ada, aga, etc– Each consonant presented 12 times in random order– Presented with carrier phrase “The next word is”

• (Tyler et al, 1983)

Example of Iowa

Consonant Confusion

Test

(example: auditory alone)

Response

p t k b d g v z n m

S

t

im

i

l

u

s

p 0 2 6 2 1 0 0 0 0 1

t 0 3 2 1 1 0 1 4 0 0

k 0 1 3 0 0 3 1 2 1 1

b 1 1 2 2 0 3 1 0 1 1

d 1 0 3 2 0 0 1 2 3 0

g 1 0 5 0 2 0 0 0 0 4

v 1 2 1 0 3 0 2 1 2 0

z 1 2 0 2 0 0 3 0 2 2

n 0 1 0 0 0 0 0 0 4 7

m 0 0 0 0 0 0 0 0 4 8

Children’s Auditory Test

CAT Auditory alone limited set test assessing

ability to perceive stress patterns and word recognition

Consists of 12 words– Monosyllabic words– Trochees– Spondees

IMSPAC

Imitative tests of Speech Pattern Contrast Perception

Developed by Arthur Boothroyd Syllable level, 4 lists randomized Choose odd one of 3 (forced choice) Pointing, button-press or verbal response Age 7 years and up

AB Short Word List

Isophonemic Word List Developed in 1968 by Arthur Boothroyd Speech recognition open set test Each list consists of ten words, and each word is

constructed as consonant - vowel – consonant 30 phonemes, 10 vowels and 20 consonants

present in each list CNC words Score is based on the phonemes correct out of 30

Larsen Recorded Test Auditory word discrimination test Pairs of phonemes in words

– few vs. chew

– bill vs. mill

– nice vs. vice

Lists represent an attempt to present the phone in the initial, medial and final position of a word

Limited choice—select one of two words by drawing line through printed word heard

CAVET

Children’s Audiovisual Enhancement Test Assesses speechreading enhancement in children within

the vocabulary level of 7-9 year olds with profound prelingual hearing loss

Designed to minimize ceiling and floor effects, eliminate syntactic factors, and minimize semantic factors

3 lists of 20 words each with half of words easy to recognize in a vision-only condition and half are less likely to be recognized in each list but presented in random order

Each list is designated for auditory alone, visual alone, or auditory-visual only mode

Test available in CD-ROM and VHS format– (Tye-Murray & Geers, 2002)

CUNY Sentences Test The City University of New York (CUNY) Topic Related Sentence Sets consist of 25

equivalent sets of 12 sentences of varying length Each sentence is related to one of 12 topics Open set sentence recognition test Listener is presented with sentences in three

listening conditions: A, V and AV The test is scored with the total number of words

correctly identified (Boothroyd, Hannin, & Hnath, 1985)

Lexical Neighborhood Test (LNT) Lexical neighbors--words that sound similar to a target item. Often defined as words that differ by a single phoneme from

target word Open-set test that requires child to imitate stimulus words

immediately after they are presented Each list consists of 50 monosyllabic words, 25 of which are

high-frequency words with few lexical neighbors (easy) and 25 of which are lower frequency words with many lexical neighbors (hard)

Words were selected to be familiar to children with limited vocabularies.– Easy Words: juice, good, drive, time, hard, gray, foot, orange, count– Hard Words: thumb, pie, wet, fight, toe, cut, pink, hi, song, fun, use, mine,

Alternate version of test, the Multisyllabic Lexical Neighborhood Test (MLNT) consists of 50 words with two to three syllables. – (Kirk, Pisoni, & Osberger, 1995)

QuickSIN Provides a one-minute estimate of SNR loss for adults A quick method to quantify patient’s ability to hear in noise Can determine if extended high frequency emphasis improves or

degrades understanding of speech in noise Assist in choosing appropriate amplification and/or other hearing

assistance technologies Demonstrates directional microphones may improve speech

intelligibility in noise Open set test recognition test Consists of list of six sentences with five key words per sentence

presented in four-talker babble noise.  Sentences are presented at pre-recorded signal-to-noise ratios which

decrease in 5dB steps from 25 (very easy) to 0 (extremely difficult).  The SNR's used are 25, 20, 15, 10, 5, 0, encompassing normal to

severely impaired performance in noise.

BKB - SIN

Similar to Quick-SIN but can be utilize with children

Sentence recognition using BKB sentence material

Open set sentence recognition test BKB SIN test Test score sheets

HINT The HINT is a prerecorded test that measures

sentence speech recognition abilities in quiet or in noise accurately, reliably and efficiently

The HINT demonstrated the substantial role that binaural, directional hearing plays in a normal hearing individual's ability to communicate in noise

Any degree of hearing impairment, therefore, reduces the benefits of directional hearing in noise and increases communication inadequacy

Used to evaluate functional hearing capabilities of HI users, CI users and those in listening-critical jobs

Copyrighted by the House Institute and is available for purchase on a compact disc recording to hearing clinics and research laboratories nationwide.

A children's version of the HINT is also available

DICHOTIC SENTENCE IDENTIFICATION (DSI)

Developed in an attempt to conceive a dichotic listening task that would be only minimally affected by peripheral hearing loss

Sentences selected from the Synthetic Sentence Identification (SSI) test presented dichotically

Onsets and offsets of the sentences are aligned with an accuracy of 100 microseconds.

Developers claim DSI less susceptible to hearing loss than SSW test

Viable test of central auditory function with hearing loss Test applicable for auditory assessment of impaired ears through

pure tone averages (PTAs) up to 50 dB 2 sets of 30 pairs of sentences (closed set identification test) Normative data are available

Ling-6

Phoneme level detection and recognition test

m, s, sh, e, a, u Procedure Detection and identification of phonemes

Further Auditory Evaluations

Mark C. Flynn—Evaluation of Individuals with hearing loss

Assignment

If you accept the challenge, assign each test discussed in this section on the following grid, thus,

Be able to correctly indicate on the following template, each of the auditory tests discussed earlier

Speech ParadigmDetection Discrimination Recognition Comprehension

Non-Speech Sounds

Phonemes

Syllables

Suprasegmentals

Words

Phrases

Sentences

Connected Speech