Kalyuga 2000 Expert Reversal Effect

Embed Size (px)

Citation preview

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    1/11

    Journal of Educational Psychology Copyright 2000 by the American PsychologicalAssociation Inc.

    2000 Vol. 92 No. 1 126-136 0022-0663100155.00 DOI : 10.1037//0022-0663.92.1.126

    Incorporating Learner Ex perien ce Into the esig n

    of M ult im edia Instruction

    Slava Kalyuga Paul Chandler and John Swel ler

    U n i v e r s it y o f N e w S o u t h W a l e s

    In Experiment 1, inexperienced trade apprentices w ere presented with one o f four al ternative

    instructional designs: a diagram with visual text, a diagram with auditory text, a diagram with

    both visual and auditory text , or the diagram only. An auditory presentation of text proved

    superior to a visual-only presentation but not w hen the text was p resented in both auditory and

    visual forms. The diagram-only format was the least intel l igible to inexperienced learners.

    When part icipants became more experienced in the domain after two specifically designed

    training sessions, the advantage of a visual diagram -aud itory text forma t disappeared. In

    Experiment 2, the diagram-only group w as com pared with the audio-text group after an

    addit ional training session. The results were the reverse of those of Experiment 1: The

    diagram-only group outperformed the audio-text group. Su ggestions are mad e for mult imedia

    instruction that takes learner experience into consideration.

    T h e e x p e r i m e n t s r e p o r t e d i n t h is a r t ic l e w e r e d e s i g n e d t o

    t e s t s o m e h y p o t h e s e s g e n e r a t e d o n t h e b a s i s o f c o g n i t iv e

    l o a d t h e o r y ( s e e S w e l l e r , 1 9 9 9 , a n d S w e l l e r, v a n M e r r i e n -

    b o e r , & P a a s , 1 9 9 8 , f o r a r e c e n t r e v i e w o f t h e t h e o r e ti c a l

    m o d e l ) . C e n t r a l t o t h e t h e o r y i s t h e n o t i o n t h a t w o r k i n g

    m e m o r y l i m i t a t i o n s s h o u l d b e a m a j o r c o n s i d e r a t i o n w h e n

    d e s i g n i n g i n s tr u c t io n . A l t h o u g h w o r k i n g m e m o r y c o n s i d e r -

    a t i o n s a r e o f t e n o v e r l o o k e d , s t o r a g e a n d p r o c e s s i n g l i m i t a -

    t i o n s h a v e b e e n w e l l k n o w n f o r s o m e t i m e ( e . g . , M i l l e r ,

    1 9 5 6 ), a n d t h e s e a f f e c t o n e ' s c a p a b i l i t y t o le a r n f r o m

    i n s tr u c t io n . O n l y a f e w e l e m e n t s o f i n f o r m a t i o n c a n b e

    p r o c e s s e d i n w o r k i n g m e m o r y a t a n y t i m e . T o o m a n y

    e l e m e n t s m a y o v e r b u r d e n w o r k i n g m e m o r y , d e c re a s in g t h e

    e f f e c t i v e n e s s o f p r o c e s s i n g . O n t h e o t h e r h a n d , a n u n l i m i t e d

    n u m b e r o f e l e m e n t s c a n b e h e l d i n l o n g - te r m m e m o r y i n t h e

    f o r m o f h i e ra r c h ic a l ly o r g a n i z e d s c h e m a s - - d o m a i n - s p e c i f i c

    k n o w l e d g e s t r u c t u r e s t h a t a l l o w p e o p l e t o c a t e g o r i z e m u l -

    t i p le e l e m e n t s o f i n f o r m a t i o n a s a s i n g l e e l e m e n t . T h u s ,

    b e c a u s e s c h e m a s a l l o w o n e t o t r e a t m u l t i p l e s u b e l e m e n t s o f

    i n f o r m a t i o n a s a s i n g le e l e m e n t i n w o r k i n g m e m o r y , th e y

    r e d u c e t h e b u r d e n o n w o r k i n g m e m o r y .

    C o g n i t i v e l o a d t h e o r y h a s b e e n u s e d t o g e n e r a t e a n d

    exp l a i n t he sp l i t - a t t en t i on e f f ec t: I f lea r ne r s a r e r equ i r ed t o

    s p l it t h e i r a t te n t i o n b e t w e e n , a n d m e n t a l l y i n te g r a t e, t w o o r

    m o r e r e l a t e d s o u r c e s o f i n f o r m a t i o n ( e . g ., t e x t a n d d i a -

    g r a m s ) , t h i s p r o c e s s m a y p l a c e a n u n n e c e s s a r y s t r a i n o n

    Slava Kalyuga, Paul Chandler, and John Sweller, School of

    Education, Universi ty of New South Wales, Sydney, New South

    Wa les, Australia.

    The research reported in this art icle was supported by grants

    from the Austral ian Research C ouncil . We thank ou r collaborating

    partners, Emall Ltd. and The Broken Hill Proprietary Company

    Limited (BHP ), for their ongoing support and, in part icular, Bryan

    Jones and Richard Winter of Email Ltd. and Eddy Go sek of BHP.

    Correspondence concerning this art icle should be addressed to

    Slava Kalyuga, School of Education, Universi ty of New South

    Wales, Sydney, New South Wales, 2052, Australia. Electronic mall

    ma y be sent to [email protected].

    l i m i t e d w o r k i n g m e m o r y a n d h i n d e r l e a r n i n g ( C h a n d l e r &

    S wel l e r , 1991) .

    P h y s i c a l l y i n t e g r a t e d f o r m a t s a r e a n a l t e r n a t i v e t o t h e s e

    c o n v e n t i o n a l s p l i t - s o u r c e i n s tr u c ti o n s . W i t h a n i n t e g r a te d

    f o r m a t , s e c t i o n s o f t e x t a r e d ir e c t ly e m b e d d e d o n t o t h e

    d i a g r a m i n c l o s e p r o x i m i t y t o c o r r es p o n d i n g c o m p o n e n t s o f

    t h e d i a g r a m . P h y s i c a l i n t e g r a t i o n o f r e l a te d e l e m e n t s o f

    d i a g r a m s a n d t e x t r e d u c e s t h e s e a r c h - a n d - m a t c h p r o c e s s

    r equ i r ed o f con ven t i on a l i ns t r uc ti ons , t hus l e s sen i ng t he

    b u r d e n o n w o r k i n g m e m o r y . T h e s u p e r i o r i t y o f p h y s i c a l l y

    i n t e g r a t e d i n s t r u c t i o n s o v e r c o n v e n t i o n a l f o r m a t s h a s b e e n

    l a b e l e d t h e

    s p l i t a t t e n t i o n e f f e c t

    a n d h a s b e e n d e m o n s t r a t ed

    i n m a n y d i v e r s e s t u d i e s t h a t h a v e e x a m i n e d a r a n g e o f

    l ea r n i ng a r eas ( Chand l e r & S wel l e r , 1991 , 1992 , 1996 ;

    M ayer & Ga l l i n i , 1990 ; S we l l e r , Chand l e r , T i e r ney , &

    Cooper , 1990 ; T ann i z i & S wel l e r , 1988 ; W ar d & S wel l e r ,

    1 9 9 0 ). W h a t M a y e r a n d c o l l e a g u e s c a l l t h e

    c o n t i g u i t y e f f e c t

    ( e .g . , M a y e r & A n d e r s o n , 1 9 9 1 , 1 9 9 2 ) i s a f o r m o f t e m p o r a l

    spl i t a t tent ion.

    T h e r e m a y b e a l t e r n a t i v e w a y s o f d e a l i n g w i t h s p l i t

    a t te n t io n . C u r r e n t th e o r i e s o f w o r k i n g m e m o r y c o n s i d e r

    c a p a c i t i e s t o b e d i s t r ib u t e d o v e r s e v e r a l p a r t l y i n d e p e n d e n t

    s u b s y s t e m s . F o r e x a m p l e , d u a l- p r o c e s s in g m o d e l s o f m e m o r y

    s u g g e s t t h a t t h e r e a r e s e p a r a t e a u d i t o r y a n d v i s u a l c h a n n e l s

    ( Badde l ey , 1992 ; P a i v i o , 1990 ; P enney , 1989 ; S chne i de r &

    D e t w e i l e r , 1 9 8 7 ) . T h u s , t h e u s e o f s o m e c o m b i n a t i o n s o f

    b o t h a u d i t o r y a n d v i s u a l i n s tr u c t io n m a y a c t a s a s u b s t i t u te

    f o r phys i ca l i n t egr a t i on wh en dea l i ng w i t h sp l i t a t t en t ion . I f

    o n e s o u r c e o f i n f o r m a t i o n i s p r e s e n t e d i n a u d i t o r y f o r m a n d

    t h e o t h e r i n v i s u a l f o r m , m e n t a l i n t e g ra t i o n o f t h e a u d i o a n d

    v i s u a l i n f o r m a t i o n i s s t i ll r e q u i re d , b u t i t m a y n o t o v e r l o a d

    w o r k i n g m e m o r y c a p a c i ty i f w o r k i n g m e m o r y i s e n h a n c e d

    b y a d u a l - m o d e p r e s e n t a ti o n . T h e a m o u n t o f i n f o r m a t i o n

    t h a t c a n b e p r o c e s s e d u s i n g b o t h a u d i t o r y a n d v i s u a l

    c h a n n e l s m a y b e c o n s i d e r a b l y l a r g e r t h a n t h a t u s i n g o n l y a

    s i n g le c h a n n e l. T h u s , l im i t e d w o r k i n g m e m o r y m a y b e

    e f f e c t i v e l y e x p a n d e d b y u s i n g m o r e t h a n o n e s e n s o r y

    m o d a l i t y , a n d i n s t ru c t i o n a l m a t e r i a l s w i t h d u a l - m o d e p r e s e n -

    126

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    2/11

    LEARNER EXPERIENCEAND MULTIMEDIA NSTRUCTION 12 7

    ta t ion ( e . g . , a v i sua l d iagr am accom panied by an aud i to r y

    t e x t ) m a y b e m o r e e f f ic i e n t t h a n e q u i v a l e n t s i n g l e -m o d a l i t y

    f or m a ts .

    Mayer and h i s a s soc ia te s ( e . g . , Mayer , 1997) have con-

    d u c t e d a n u m b e r o f e x p e r i m e n t s d e m o n s t r a t i n g t h e s u p e r i o r-

    i t y o f c o m b i n e d a u d i o a n d v i s u a l i n s t r u c t i o n s . M a y e r a n d

    M o r e n o ( 1 9 9 8 ) o b s e r v e d t h a t s t u d e n t s w h o r e c e i v e d a u d i -

    t o r y e x p l a n a t io n s s i m u l t a n e o u s l y w i t h a c o m p u t e r - g e n e r a t e d

    a n i m a t i o n o u t p e r f o r m e d s t u d en t s w h o i n s t e a d r e c e i v e d c o n -

    c u r r e n t o n - s c re e n v i s u a l t e x t i n v o l v i n g t h e s a m e w o r d s a n d

    pr esen ta t ion t im ing . Mul t im edia l ea r ne r s can in tegr a te wor ds

    a n d p i c t u r e s m o r e e a s i l y w h e n t h e w o r d s a r e p r e s e n t e d

    audi to r i ly r a the r than v i sua l ly .

    I n a s e r ie s o f expe r im ents us ing geom et r y ins t r uc t iona l

    m a te r ia l , Mousav i , Low, and S wel le r ( 1995) f ound tha t a

    v i s u a l ly p r e s e n t e d g e o m e t r y d i a g r am , c o m b i n e d w i t h a u d i t o -

    r i l y p r e s e n t e d s t a t e m e n t s , e n h a n c e d l e a r n i n g c o m p a r e d t o

    convent iona l , v i sua l - on ly p r esen ta t ions ( the instructional

    modality effect . Tinda l l - F or d , Chandle r , and S wel le r ( 1997)

    a l so inves t iga ted th i s e f f ec t us ing e le m en ta r y e lec t r i ca l

    eng inee r ing ins t r uc t ions and dem ons t r a ted tha t an aud io t ex t

    a n d v i s u a l d i a g r a m f o r m a t w a s s u p e r i o r t o p u r e l y v i s u a l ly

    based ins t r uc t ions . J eung , Chandle r , and S wel le r ( 1997)

    d e m o n s t r a t e d t h a t d u a l - m o d e p r e s e n t a t io n s e n h a n c e d l e a rn -

    i n g o n l y b y e l i m i n a t i n g a n e x t e n s i v e v i s u a l s e a r c h w h e n

    coor d ina t ing aud i to r y and v i sua l in f or m a t ion .

    C o m b i n e d a u d i o a n d v i s u a l p r e s e n t a ti o n m a y p r o v i d e a n

    a l te r na t ive way of dea l ing wi th sp l i t - a t t en t ion ins t r uc t iona l

    f o r m a ts . Al though lea r ne r s a r e s t i l l r equ i r ed to coor d ina te

    in f or m a t ion f r om d i spa r a te aud io and v i sua l sour ces , they

    h a v e m o r e e f f e c t iv e w o r k i n g m e m o r y a v a i l a b l e to d e a l w i t h

    i n f o r m a t i o n i n te g r a ti o n t h a n t h e y w o u l d w h e n d e a l i n g w i t h

    an equ iva len t v i sua l - on ly f o r m a t , thus dem ons t r a t ing a

    modality effect. H o w e v e r , p r e s e n t i n g s e v e r a l s o u r c e s o f

    i n f o r m a t i o n s i m u l t a n e o u s l y , e v e n i n a n i n t e g r a t e d f o r m a t

    ( e . g . , phys ica l ly in tegr a ted d iagr am s and tex t ) , m ay no t be

    a lways e f f ec t ive , pa r t i cu la r ly i f som e of the in f or m a t ion to

    b e p r o c e s s e d i s u n n e c e s s a r y o r r e d u n d a n t . C h a n d l e r a n d

    S wel le r ( 1991) ; Bobis , S we l le r , and Cooper ( 1993) ; and

    S w e l l e r a n d C h a n d l e r ( 1 9 9 4 ) d e m o n s t r a t e d t h a t th e e l i m i n a -

    t i o n r a t h e r t h a n i n t e g r a t io n o f r e d u n d a n t s o u r c e s o f i n f o r m a -

    t ion m igh t be bene f ic ia l f o r l ea r n ing . I f the r edundant

    in f or m a t ion i s in tegr a ted wi th e s sen t ia l in f or m a t ion , the r e i s

    no cho ice bu t to p r oces s i t ( e . g . , an unneces sa r y t ex t

    accom panying a f u l ly s e l f - con ta ined and in te l l ig ib le d ia -

    g r a m ) . R e d u n d a n c y i m p o s e s a n a d d i t i o n a l c o g n i t i v e l o a d

    tha t in te r f e r es wi th the l ea r n ing pr oces s an d has been labe led

    the redundancy effect.

    T h e r e d u n d a n c y e f f e c t g e n e r a l l y o c c u r s u n d e r c o n d i t i o n s

    in which d i f f e r en t sour ces o f in f or m a t ion a r e in te l l ig ib le in

    i s o l at i o n a n d i n w h i c h e a c h s o u r c e p r o v i d e s s i m i la r i n f o r m a -

    t ion bu t in a d i f f e r en t f o r m . At tend ing to unneces sa r y

    i n f o r m a t i o n r e q u i r es c o g n i t i v e r e s o u r ce s t h a t c o n s e q u e n t l y

    a r e u n a v a i l a b l e f o r t h e c o n s t r u c t i o n a n d r e f i n e m e n t o f

    s c h e m a s . R e d u c i n g c o g n i t i v e lo a d c a n b e a c c o m p l i s h e d b y

    e l im ina t ing r edundant in f or m a t ion and thus f r ee ing r e -

    sour ces f o r l ea r n ing . I f , f o r exam ple , a d iagr am i s su f f i -

    c ien t ly s e l f - con ta ined and in te l l ig ib le in i so la t ion , then any

    a c c o m p a n y i n g t e x t e x p l a i n i n g t h e d i a g ra m t h a t p ro v i d e s n o

    a d d i t i o n a l i n f o r m a t i o n m a y b e r e d u n d a n t a n d s h o u l d b e

    om i t ted .

    I t does no t f o l low tha t the e l im ina t ion of a d iagr am in a

    r e d u n d a n t d i a g r a m - a n d -t e x t c o m b i n a t i o n w i l l i m p r o v e l e a r n -

    i n g . D e p e n d i n g o n t h e c o n t e x t , d i a g r a m s m a y b e m u c h

    eas ie r to l ea r n f r om than tex t . A d i s t inc t ion a l so needs to be

    m a d e b e t w e e n r e d u n d a n c y a n d r e v i s io n o f p r e v i o u s l y le a r n e d

    m ate r ia l . Rev is ion i s no t a r edu nda nt ac t iv i ty tha t wi l l

    in te r f e r e wi th l ea r n ing , because r ev i s ion wi l l no t inc r ease

    w o r k i n g - m e m o r y l o a d . R e d u n d a n c y o c c u r s w h e n l e a r n e r s

    m us t un neces sa r i ly t rans la te and coor d ina te m ul t ip le sour ces

    of in f or m a t ion ( such a s a d iagr am and tex t tha t r edesc r ibes

    the in f or m a t ion in the d iagr am ) . Tha t ac t iv i ty i s m enta l ly

    d e m a n d i n g a n d , f o r le a r n e rs w h o c a n f u l l y u n d e rs t a n d o n e

    s o u r c e o f i n f o r m a t i o n , p r e se n t i n g t h e m w i t h o t h e r s o u r c e s

    gene r a te s an ex t r ane ous cogni t ive load .

    We ( Ka lyuga , Chandle r , & S wel le r , 1999) p r ov ided

    a n o t h e r e x a m p l e o f t h e r e d u n d a n c y e f f e c t . W e c o m p a r e d

    thr ee ins t r uc t iona l f o r m a ts us ing ins t r uc t iona l m a te r ia l s

    about so lde r ing in a t r a in ing env i r onm ent . Thr ee g r oups o f

    pa r t i c ipan ts wer e v i sua l ly p r esen ted wi th d iagr am m at ic

    i n f o rm a t i o n . T h e f ir st g r o u p a l s o r e c e i v e d t h e a c c o m p a n y i n g

    tex t v i sua l ly ( v i sua l - t ex t g r oup) , a s econ d gr oup r ece ived the

    tex t aud i to r i ly ( aud i to r y- tex t g r oup) , and a f ina l g r oup

    r ece ived the t ex t aud i to r i ly and v i sua l ly ( v i sua l - and- audi to r y-

    t e x t g r o u p ). C o m b i n e d a u d i o a n d v i s u a l p r es e n t a ti o n s o v e r -

    cam e sp l i t - a t t en t ion pr ob lem s ( i . e. , the aud i to r y - tex t g r oup

    o u t p e r f o r m e d t h e v i s u a l -t e x t g r o u p ), c o n f i r m i n g a m o d a l i t y

    e f f ec t . The r e a l so was a r edun danc y e f f ec t , wi th the aud i to r y-

    tex t g r oup c lea r ly ou tpe r f or m ing the aud i to r y- and- v i sua l -

    t ex t g r oup . S im ul taneou s inc lus ion of v i sua l ly p r esen ted t ex t

    w i t h i d e n t i c a l a u d i t o r y t e x t , w h i c h i s c o m m o n i n m a n y

    s tandar d m ul t im edia packages , im p osed an add i t iona l unnec -

    es sa r y cogni t ive load tha t in te r f e r ed wi th l ea r n ing . E l im i -

    n a t i o n o f r e d u n d a n t v i s u a l t e x t u a l e x p l a n a t i o n s p r o v e d

    benef ic ia l .

    Whe the r a sour ce o f in f or m a t ion i s in te l l ig ib le in i so la -

    t ion , and add i t iona l in f or m a t ion the r e f or e redundan t , in

    m a n y c a s e s d e p e n d s o n t h e l e v e l o f e x p e r i e n c e o f t h e l e a r ne r .

    F o r e x a m p l e , r e g a r d i n g t h e i n f o r m a t i o n i n F i g u r e 1 , i f

    l ea r ne r s have suf f ic ien t knowledge to under s tand the d ia -

    gr am , then the t ex t ( in a v i sua l f o r m , aud i to r y f o r m , o r bo th)

    m i g h t b e r e d u n d a n t f o r t h e s e l e a rn e r s. T h e y m a y p r e f e r to

    ignor e the t ex t bu t m a y hav e d i f f i cu l ty do ing so , re su l t ing in

    a h ighe r cogni t ive load . I n th i s s i tua t ion , the bes t ins t r uc -

    t i o n al f o r m a t w i t h t h e l o w e s t u n n e c e s s a r y c o g n i t iv e l o a d f o r

    t h e s e l e a rn e r s m a y b e a d i a g r a m - o n l y f o r m a t . O n t h e o t h e r

    hand , i f the d iagr am i s no t in te l lig ib le in i so la t ion because o f

    a lower l eve l o f l ea r ne r expe r ience , then add i t iona l t ex t

    w o u l d b e e s s e n t i al a n d t h e r e f o r e sh o u l d b e p r e s e n t e d i n a n

    audi to r y r a the r than a v i sua l f o r m a t .

    We ( K a lyuga , Chandle r , & S w el le r , 1998) a l so f ound tha t

    low- knowledge e lec t r i ca l t r a inees bene f i t ed f r om addi t iona l

    tex t - based in f or m a t ion wi th d iagr am s of e lec t r i ca l c i r cu i t s

    ( e spec ia l ly when tex tua l exp lana t ions wer e in tegr a ted in to

    the d iagr am s , thus r educ ing sp l i t a t t en t ion) . High- knowl-

    edge e lec t r i ca l t r a inees showed a p r e f e r ence f o r an ins t r uc -

    t iona l packag e tha t cons i s ted o f the e lec t r ica l c i r cu i t d iagr am

    only . The sp l i t - a t t en t ion e f f ec t was r ep laced by the r edun-

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    3/11

    128 KALYUGA, CHANDLER,AND SWELLER

    igure 1. A section of the computer screen presentation for instruction on the cutting speed

    nomogram (diagram-with-visual-text and diagram-with-visual-plus-audio-text instructional formats

    used in Stage 1 of Experiment 1).

    dancy effect as the tra inees acquired experience. Specif i-

    cally, for low-kn owledge learners the most effective instruc-

    tional package was an integrated package that was more

    eff ic ient than both a conventional separate text-and-diagram

    form at (thus indicating a split-attention effect) and a diagram-

    only format, which was largely unintell igible . As the tra in-

    ees acquired exp erience, the text, whether presented sepa-

    rate ly or integrated, became redundant, and the most eff ic ient

    instructional format becam e the diagram-only format. Thus,

    the most eff ic ient method of information delivery may be

    partly determined by the level of exp erience of the learner.

    In this ar t icle , we fur ther examine the role of exp erience

    with respec t to instructional design, specifically with respect

    to dual-mod e instruction. Our experiments were designed to

    investigate whether the level of learner experience would

    rela te the modali ty effect to the redund ancy effect , in a way

    similar to the re la tions between spli t a t tention and redun-

    dancy in our 1998 studies. Specifically, with respect to

    dual-mode instruction, we hypothesized that for inexperi-

    enced learners a diagram-with-auditory- text instructional

    format would be m ore effective than a diagram-with-visual-

    text format (modali ty effect) . We predicted that a diagram-

    only format, with no text, would be the least eff icient format.

    We expected that , as the learners ' level of experience was

    raised through tra ining, the diagram-only format would

    becom e the most effective and would be signif icantly more

    eff ic ient than both the diagram-with-auditory- text and the

    diagram-with-visual-text formats, thus providing a dem on-

    stra tion o f redund ancy effects.

    The prediction that , with increased expertise , two m odes

    of presentation ( the addit ion of w ords to pic tures) can have a

    deleter ious ra ther than merely a neutra l effect may appear

    improbable and so requires fur ther explanation. The counter-

    intuit ive nature o f this prediction mirrors the counter intui-

    t ive nature of the redundancy effect in general . For anyone,

    once he or she has acquired suff ic ient knowledge to fully

    understand one source of information, addit ional sources

    that require integration and coordination will have deleteri-

    ous effects unless he or she can ignore the secondary sources

    of information. Although this prediction follows from cog ni-

    t ive load theory, an intuit ive feel for the suggestion might be

    obtained by consider ing the following. Imagine a new traff ic

    system o f one-way streets is to be insti tuted on your campus.

    A coordinate diagram of the campus is provided, with

    arrows ind icating the traffic flow. In addition, a d escription,

    in e ither auditory or writ ten form, is provided. That descr ip-

    t ion includes sta tements such as the road between the

    Smith Building (Coordinate E3) and the Brown Building

    (Coordinate B7) will be one-way from a southwest to a

    northeast direction. I f you (an expert on the layout of you r

    campus) fo llow this descr iption c losely, the cognitive load

    will be greater than just looking at the m ap with i ts arrows.

    The descr iption is redundant, imposes a heavy cognitive

    load, and is likely to slow learning. In contrast, a newcomer

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    4/11

    LE RNER EXPERIENCE ND MULTIMEDI INSTRUCTION 29

    t o t h e c a m p u s , w h o m a y h a v e n o i d e a w h e r e t h e S m i t h o r

    B r o w n B u i l d i n g s a r e l o c a t e d , m a y n e e d t h e v e r b a l d e s c r ip -

    t i o n a n d m a y u n d e r s t a n d m o r e q u i c k l y i f it i s g i v e n in

    a u d i t o r y r a t h e r t h a n w r i t t e n f o r m .

    E x p e r i m e n t 1

    A s e c t io n o f a c o m p u t e r s c r e e n p r e s e n t a t io n f o r i n s t r u c -

    t i o n o n t h e c u t t i n g s p e e d n o m o g r a m i s p r e s e n t e d in F i g u r e 1 .

    B e f o r e m a c h i n e d r i l l i n g c a n c o m m e n c e , a n a p p r o p r i a t e

    n u m b e r o f r p m m u s t b e e s t a b li s h e d o n t h e c u t ti n g m a c h i n e s

    s p e e d c o n t r o l . T h e r p m d e p e n d s o n t h e d i a m e t e r o f t h e d r il l

    a n d t h e r e c o m m e n d e d c u t t i n g s p e e d ( w h i c h i s t h e s u r f a c e

    speed o f the work a s i t pas ses the cu t t ing too l ) . The cu t t ing

    s p e e d i s u s u a l l y m e a s u r e d i n m e t e r s p e r m i n u t e . T h e

    r e c o m m e n d e d c u t t i n g s p e e d v a r i e s w i t h t h e t y p e o f m a t e r i al

    be ing cu t .

    T h e c o n v e r s i o n f r o m c u tt in g s p e e d t o r p m m a y b e d o n e

    us ing fo rm ulas , re fe rence t ab le s , o r cha r t s . A cu t t ing speed

    n o m o g r a m i s a q u i c k a n d r e li a b l e m e t h o d o f d e t e r m i n i n g t h e

    r p m t o r u n a d r i l l o f a g i v e n d i a m e t e r ( i n m m ) a t a g i v e n

    cu t t ing speed ( in m e te rs pe r m inu te ) .

    A s s u m e t h e m a t e r i al o f F i g u r e 1 ( an e x p l a n a t i o n o f h o w t o

    u s e t h e n o m o g r a m t o d e t e r m i n e t h e a p p r o p r i a t e r p m t o d r i l l a

    2 5 - m m h o l e i n a b r o n z e w o r k p i e c e ) i s p r e se n t e d t o l e a rn e r s

    w h o h a v e j u s t a s s i m i l at e d t h e b a s i c c o n c e p t s r e q u i r e d t o

    unders tand the in form a t ion (cu t t ing speed , rpm , e tc . ) bu t do

    n o t k n o w h o w t o a p p l y t h i s n o m o g r a m i n s p e c i f i c c i r c u m -

    s tances . F or these re la t ive ly inexper ienced l ea rne rs , ne i the r

    the d iagram wi th the t ab le nor the t ex t a re unders tandab le in

    i so la t ion . To unders tand the in form a t ion , the d iagram , the

    t a b le , a n d t h e t e x t m u s t b e m e n t a l l y i n te g r a te d . E l e m e n t s o f

    i n f o r m a t i o n i n c l u d e d i n t h e t e x t m u s t b e m a t c h e d w i t h

    c o r r e s p o n d i n g r e l a t e d e le m e n t s i n t h e d i a g r a m a n d t a b le , a n d

    i n t e ra c t i o n s b e t w e e n t h e m t r a ce d . B e c a u s e m a n y i n t e r ac t i n g

    e l e m e n t s o f i n f o r m a t i o n a r e i n v o l v e d i n s u c h s e a r c h a n d

    t rac ing (e . g . , m a te r ia l and d iam ete r va lue in the t ex t ,

    m a te r ia l and cor re sponding cu t t ing speed range in the t ab le ,

    v e r t i c a l a n d d i a g o n a l l i n e s c o r r e s p o n d i n g t o d i a m e t e r a n d

    cu t t ing speed va lue s in the d iagram , e tc . ) , a heavy cog ni t ive

    l o a d d u e t o s p l it a t t e n ti o n w a s e x p e c t e d . T h i s c o m b i n a t i o n o f

    lea rne rs and ins t ruc t iona l m a te r ia l s represen t s an exam ple o f

    spl i t a t tent ion.

    Lea rn ing m a y be fac i l i t a t ed in th i s si tua t ion by inc reas ing

    e f f e c t i v e w o r k i n g m e m o r y c a p a c i t y u s i n g d u a l - m o d e p r e s e n -

    ta t ion t echn iques , thus reduc ing the cogni t ive load a s soc i -

    a ted wi th m enta l in tegra t ion . F or exam ple , v i sua l t ex tua l

    e x p l a n a t i o n s c a n b e r e p l a c e d w i t h e q u i v a l e n t a u d i t o r y

    e x p l a n a ti o n s . T h e d i a g r a m , t a b l e , a n d t h e a u d i t o r y e x p l a n a -

    t ion s t il l m u s t be m enta l ly in tegra ted , wi th m any in te rac t ing

    a u d i o a n d v i s u a l e l e m e n t s o f i n f o r m a t i o n i n v o l v e d i n t h e

    i n t eg r a t io n . N e v e r t h e l e ss , b e c a u s e o f t h e e f f e c t i v e e x p a n s i o n

    o f w o r k i n g m e m o r y , d u e t o t h e c o m b i n e d u s e o f v i su a l a n d

    a u d i t o r y p r o c e s s o r s , t h e m a t e r i a l m a y b e m o r e r e a d i l y

    h a n d l e d t h a n w h e n p r e s e n t e d s o l e l y i n a v i s u a l f o r m .

    Now, a s sum e tha t the ins t ruc t iona l p resen ta t ion in F igure

    1 , inc lud ing the v i sua l ly based t ex t , i s used s im ul taneous ly

    wi th an aud i to ry n a r ra t ion o f the t ex t . In th is s i tua t ion , e i the r

    the aud i to ry t ex t o r v i sua l t ex t i s redundant . Al l the

    i n t e ra c t i n g e l e m e n t s o f t h e d i a g r a m a n d v i s u a l a n d a u d i t o r y

    e x p l a n a t io n s m u s t b e m e n t a l l y i n te g r a te d . W o r k i n g m e m o r y

    c a p a c it y , e v e n i f e x p a n d e d b e c a u s e o f t h e u s e o f b o t h a u d i o

    a n d v i s u a l p r o c e s s o r s, s h o u l d b e m o r e r e a d i l y a b l e t o h a n d l e

    the m a te r ia l wi thou t the v i sua l t ex t , which i s e s sen t i a l ly

    r e d u n d a n t . E s t a b li s h i n g c o n n e c t i o n s b e t w e e n c o r r e s p o n d i n g

    e l e m e n t s o f v i s u a l a n d a u d i t o r y c o m p o n e n t s o f w o r k i n g

    m e m o r y m a y u n n e c e s s a r i l y c o n s u m e a d d i t i o n a l c o g n i t i v e

    resources to those used in a d iagram -plus -aud i to ry- tex t -

    a lone fo rm a t . To read and l i s t en to ve rba l m a te r ia l , lea rne rs

    m u s t p r e c i s e l y c o o r d i n a t e t h e t w o s e n s o r y m o d e s . T h e

    s l igh tes t los s o f coord ina t ion wi l l re su l t in chao t ic inpu t .

    W o r k i n g m e m o r y r e s o u r c e s r e q u i r e d f o r c o o r d i n a t i o n w i l l

    b e u n a v a i la b l e f o r l e a r n in g . T h u s , f r o m t h e p o i n t o f v i e w o f

    c o g n i t i v e l o a d t h e o r y , a d u p l i c a t i o n o f i n f o r m a t i o n u s i n g

    d i f fe ren t m odes o f p resen ta t ion inc reases the r i sk o f ove r load-

    i n g w o r k i n g m e m o r y c a p a c i t y a n d m a y h a v e n e g a t i v e

    l e a r n i n g c o n s e q u e n c e s . O n e o f t h e d u p l i c a t e d s o u r c e s o f

    in form a t ion (v i sua l o r aud i to ry exp lana t ions ) i s c lea r ly

    r e d u n d a n t . I t d o e s n o t p r o v i d e a n y n e w i n f o r m a t i o n f o r t h e

    lea rne rs in add i t ion to tha t a l ready prov ided . In th i s s i tua -

    t i o n, e l i m i n a t i o n o f t h e r e d u n d a n t v i s u a l t e x t -b a s e d i n f o r m a -

    t ion m igh t b e bene f ic ia l fo r l ea rn ing .

    F ina l ly , a s sum e tha t the ins t ruc t iona l p resen ta t ion in

    F igure 1 con ta ins on ly the d iagram w i th t ab le and tha t a l l the

    tex tua l exp lana t ions , bo th in aud i to ry and v i sua l fo rm s , a re

    e x c l u d e d . I n t h i s s i t u a ti o n m a n y i n e x p e r i e n c e d l e ar n e r s m a y

    not be ab le to unders tand the d iagram and re la ted t ab le .

    T e x t u a l e x p l a n at i o n s m a y b e e s s e n t i a l s i m p l y t o u n d e r s t a n d

    t h e d i a g r a m m a t i c a n d t a b l e -b a s e d i n f o r m a t i o n , b u t t h e s a m e

    d i a g r a m - o n l y i n s t r u c t i o n , p r e s e n t e d t o m o r e e x p e r i e n c e d

    lea rne rs , m ay be bo th in te l l ig ib le in i so la t ion and a l so

    inform a t ive in tha t l ea rne rs can acqu i re knowledge f rom i t .

    A d d i t i o n a l i n f o r m a t i o n is n o t n e e d e d b e c a u s e l e a r n e r s m a y

    h a v e a c q u i r e d k n o w l e d g e t h a t e a s i l y a ll o w s i n f e r e n c e s t o b e

    m ade . I f such add i t iona l in form a t ion i s s t i l l p rov ided fo r

    m o r e e x p e r i e n c e d l e a r n e r s , i t i s r e d u n d a n t a n d m i g h t b e

    ignored by t ra inees . I f i t i s no t pos s ib le to ignore t ex tua l

    e x p l a n a t i o n s , p r o c e s s i n g t h e m m a y u n n e c e s s a r i l y i n c r e a s e

    cogni t ive load . These l ea rne rs m ay be a s s i s t ed by the

    e l im ina t ion o f the add i t iona l m a te r ia l .

    T h u s , E x p e r i m e n t 1 w a s d e s i g n e d f i r s t t o c o m p a r e f o u r

    d i f fe ren t ins t ruc t iona l fo rm a ts (d iagram wi th v i sua l t ex t ,

    d iagram wi th aud io t ex t , d iagram wi th v i sua l p lus aud io t ex t ,

    and d iagram on ly) wi th re la t ive ly inexper ienced t ra inees .

    T h e t r a i n e e s w e r e p r o v i d e d d i r e c t t r a i n i n g i n t h e d o m a i n ,

    a n d w e c o m p a r e d t h e s a m e f o u r i n s t ru c t i o n a l fo r m a t s a f t e r

    the t ra in ing . In accord wi th cogni t ive load theory , we

    p r e d i c t e d t h a t t h e i n e x p e r i e n c e d t r a i n e e s w o u l d b e n e f i t

    r e l a t iv e l y m o r e f r o m a d i a g r a m - w i t h - a u d i o - t e x t f o r m a t t h a n

    a d iagram -plus -v i sua l - t ex t fo rm a t , rep l i ca t ing the m oda l i ty

    e f fec t o f p rev ious s tud ies (Ka ly uga e t a l. , 1999; Mou sav i e t

    a l ., 1995; T inda l l -F ord e t a l ., 1997) . F o r the d iagram -w i th-

    v i s u a l -p l u s - a u d io - t e x t f o r m a t , w e e x p e c t e d t h a t i n c l u s i o n o f

    the v i sua l t ex t s im ul taneous ly wi th iden t i ca l aud i to ry t ex t

    w o u l d i m p o s e a n a d d i ti o n a l c o g n i t iv e l o a d o n i n e x p e r i e n c e d

    lea rne rs who proces s i t , r e su l t ing in a nega t ive e f fec t

    (add i t iona l cogni t ive re sources a re needed fo r m enta l in tegra -

    t ion o f the v i sua l t ex t wi th the d iagram an d aud io t ex t ) . Thus ,

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    5/11

    13 KALYUGA, CHANDLER, AND SWELLER

    a d i a g r a m - w i t h - a u d i o - t e x t f o r m a t s h o u l d b e s u p e r i o r t o t h e

    d i a g r a m - w i t h - v i s u a l - p l u s - a u d i o - t e x t f o r m a t , d e m o n s t r a t i n g

    a r e d u n d a n c y e f f e c t ( K a l y u g a e t a l. , 1 9 9 9) . A d i a g r a m - o n l y

    f o r m a t s h o u l d b e t h e w o r s t p r e s e n t a t i o n f o r m a t f o r t h e

    i n e x p e r i e n c e d l e a r n e r s , a s t e x t - b a s e d i n f o r m a t i o n m a y b e

    e s s e n t i a l to u n d e r s t a n d t h e d i a g r a m .

    F o r m o r e e x p e r i e n c e d t r a i n e e s t h e a d v a n t a g e o f a d i a g r a m -

    w i t h - a u d i o t e x t f o r m a t s h o u l d d i s a p p e a r , a n d a s e l f-

    c o n t a i n ed d i a g r a m - o n l y f o r m a t m a y b e m o r e a d v a n t a g e o u s.

    T h e r e l a t i v e s ta n d i n g o f t h e d i a g r a m - w i t h - v i s u a l - t e x t f o r m a t

    a n d t h e d i a g r a m - w i t h - v i s u a l - p l u s - a u d i o t e x t fo r m a t d e p e n d s

    o n w h e t h e r ( a ) p r o c e s s i n g t h e t e x t u al e x p l a n a t i o n s i m p o s e s

    a n a d d i t i o n a l c o g n i t i v e l o a d h e a v y e n o u g h t o o v e r l o a d

    l e a r n e r s ' w o r k i n g m e m o r y c a p a c i t y o r ( b ) t h e s e e x p l a n a t i o n s

    a r e m o s t l y i g n o r e d b y l e a r n e r s .

    M e t h o d

    P a r t i c i p a n t s

    S i x t y t r a d e a p p r e n t i c e s f r o m t w o m a j o r A u s t r a l i a n m a n u f a c tu r -

    ing com panies pa r t i c ipa te d in th i s exper iment . A l l pa r t i c ipan t s had

    c o m p l e t e d a t l e a s t Y e a r 1 0 o f h i g h s c h o o l a n d h a d c o m p l e t e d a b o u t

    3 m onths o f the i r f i r s t -yea r t r ade course . Accord ing to the t echn ica l

    t r a in ing superv i so rs f rom the companies , none o f the pa r t i c ipan t s

    had p rob lems wi th genera l r ead ing ab i l i ty . F rom the i r r egu la r

    t ra in ing courses and p rac t i ca l work dur ing the f i r s t 3 months o f

    the i r appren t icesh ip , a l l pa r t i c ipan t s had ga ined su f f ic ien t knowl-

    edge o f t echn ica l t e rmino logy and concep ts ( cu t t ing speed , rpm,

    e tc . ) , cu t t ing p r inc ip les , and equ ipment to unders tand the ins t ruc -

    t iona l ma te r ia l s . Because they ha d no t bee n sp ec i f i ca l ly t augh t

    c u t t i n g s p e e d n o m o g r a m s a s a w a y o f c a l c u l a ti n g a p p r o p r ia t e r p m

    (formulas o r t ab les had been used fo r th i s purpose dur ing p rac t i ca l

    work) , none o f the pa r t i c ipan t s had p rev ious exposure to the

    ins t ruc t iona l ma te r ia l s used in the s tudy .

    M a t e r i a l s a n d P r o c e d u r e

    P a r t ic i p a n t s w e r e r a n d o m l y a l l o c a te d t o f o u r g r o u p s ( d i a g r a m

    wi th v i sua l t ex t , d iagram wi th v i su a l p lus aud io t ex t , d iagra m wi th

    aud io t ex t , and d iagram on ly ) o f 15 lea rne rs each cor respond ing to

    the four ins t ruc t iona l fo rmats . One pa r t i c ipan t f rom the d iagram-

    wi th -and io - tex t g roup d roppe d ou t a f t e r the f i r s t s tage , so tha t

    g roup inc lude d on ly 14 l ea rne rs who com ple ted a l l the s tages o f the

    exper iment . A l l ins t ruc t ions and t r a in ing fo r the s tudy were

    d e l i v e r e d t h r o u g h a n A p p l e P o w e r M a c i n t o sh c o m p u t e r ; S l a v a

    K a l y u g a d e s i g n e d a l l t h e c o m p u t e r - b a s e d tr a i n in g p a c k a g e s w i t h

    Authorware P rofessional

    (1993).

    T h e d i a g r a m - w i t h - v i s u a l -p h i s - a u d i o - te x t f o r m a t c o n t a i n e d a

    c u t t i n g s p e e d n o m o g r a m a n d w r i t t e n e x p l a n a t i o n s o f s e q u e n t i a l

    s teps in how to use the nomo gram to de te rmine a pa r t i cu la r rpm. A

    t e x t u a l p r o b l e m s t a te m e n t (e . g ., A s s u m e y o u w i s h to d e t e r m i n e

    the approp r ia te R .P .M. to d r i l l a 25 m m h o le in a b ronze

    w o r k p i e c e ) w a s p r e s e n t e d v i s u a l ly n e x t to t h e n o m o g r a m . T h e

    c u t t in g s p e e d n o m o g r a m a n d e x p l a n a t o r y n o t es f o r t h i s n o m o g r a m

    were p resen ted on the same sc reen ( see F igure 1 ) . Pa r t i c ipan t s

    cou ld no t conc lude the l ea rn ing sess ion be fore they a t t ended to a l l

    the pa ragraphs a t l eas t once . Al l the pa ragraph s tha t were s t i l l to be

    read were ind ica ted by g rey rec tang u la r shapes enc los ing those

    paragraphs . Before read ing a pa ragraph , a t r a inee had to c l i ck on

    the pa ragraph . Then , sc reen-b ased h igh l igh t s o f the approp r ia te

    e l e m e n t s o f t h e n o m o g r a m ( t h e m a t e r i a l n a m e , c u t ti n g s p e e d r a n g e ,

    l ines , in te r sec t ion po in ts , e tc . ) appeared on the sc reen . Som e s im ple

    an imat ions a l so were use d in the p resen ta t ion . For examp le , i f the

    lea rner c l i cked on the sen tence By fo l lowing the hor izon ta l line to

    the le f t , we can read o f f the appropr ia te R .P .M. , an a r row would

    move a long a h igh l igh ted hor izon ta l l ine to the l e f t toward the

    cor respond ing rpm. I t i s impor tan t to no te tha t an imat ion was

    common to a l l ins t ruc t iona l g roups (excep t the d iagram-on ly

    group) . Aud i to ry -bas ed ex t was iden t ica l to the v i sua l - t ex t in fo rma-

    t ion and was de l ive red s imul taneous ly wi th v i sua l t ex t th rough

    headpho nes wh i le lea rne rs s tud ied the cor respond ing v i sua l sc reen

    a n i m a t i o n s o f t h e d ia g r a m m a t i c i n f o rm a t i o n . A h e l p b u t t on w a s

    prov id ed fo r l ea rne rs who s t i l l were no t su re wha t to do .

    The d iagram -wi th -v i su a l - t ex t fo rmat was v i sua l ly iden t ica l to

    the d iagram-wi th -v i sua l -p lus -aud io - tex t o rmat wi th a l l the aud i -

    to ry exp lana t ions e l imina ted ( the headphones were removed) . The

    d i a g r a m - w i t h - a u d i o - te x t f o r m a t d i f f e re d f r o m t h e d i a g r a m - w i t h -

    v i sua l -p lus -aud io - tex t fo rmat in tha t the fo rm er d id no t inc lude

    v isua l exp lana t ions o f the d iagram ( the headphone s were p rov id ed

    for the aud io s ta tements ) . For the d iagra m-w i th -aud io - tex t fo rmat ,

    on ly the head ings o f the sequen t ia l s t eps (e .g . , S tep 1 : Se lec t the

    cu t t ing speed , S tep 2 : Se lec t the d iagon a l l ine ) were d i sp layed to be

    c l icked on by the l ea rne rs , w i th a l l the v i sua l t ex t -based exp lana-

    t i o ns r e p l a c e d b y e q u i v a l e n t a u d i to r y c o m m e n t a r i e s . T h e a u d i t o r y

    i n f o r m a t i o n w a s c o o r d i n a t e d w i t h s c r e e n - b a s e d a n i m a t i o n s a n d

    h i g h l i g h t s o f t h e a p p r o p r i a t e e l e m e n t s o f t h e n o m o g r a m . T h e

    d i a g r a m - o n l y f o rm a t c o n t a i n e d t h e n o m o g r a m w i t h o u t t e x tu a l

    exp lana t ions and s ta tements . No h igh l igh t s o f e lements o f the

    n o m o g r a m o r a n i m a t i o n s w e r e u s e d i n t h i s f o r m a t.

    The s tudy was cond uc ted in two s tages .

    S tage 1

    Experimental training Tra inees s tud ied the cu t t ing speed

    nom ogram ins t ruc t ion ( see F igure 1 ) in the i r r e spec t ive exper im en-

    ta l g roups (a l l four com pute r -based p resen ta t ions were se l f -paced ,

    and the re w as no t ime l im i t fo r th i s ins t ruc t ion phase ) .

    Subjective ratings

    Afte r pa r t i c ipan t s s tud ied the ins t ruc t ions ,

    they co mple te d sub jec t ive ra t ings o f t a sk d i f f i cu l ty . Se l f - ra t ings o f

    t a s k d i f f i c u lt y r e p r e s e n t o n e v i a b l e m e t h o d o f m e a s u r in g c o g n i t i v e

    l o a d a n d h a v e b e e n u s e d p r e v i o u s l y fo r t h i s p u r p o s e ( s e e P a a s &

    van Me r r ienboer , 1993 , 1994) . We chose to use sub jec t ive ra t ings

    o f t a s k d i f f i c u lt y b e c a u s e t h e y a r e e a s y t o i m p l e m e n t a n d d o n o t

    in t rude on p r imary t a sk pe r fo rmance as do o the r methods , such as

    secondary t a sks . Sub jec t ive measures o f menta l load a re re l i ab le

    and cor re la te h igh ly ( .8 - .99) w i th ob jec t ive measures (M oray ,

    1982 ; O 'Donne l l & Eggemeie r , 1986) . These ra t ings o f menta l

    e f fo r t were co l lec ted e lec t ron ica l ly on the com pute r . A 7 -po in t sca le

    was used ; pa r t i c ipan t s were asked Ho w easy o r d i f f i cu lt was th i s

    n o m o g r a m t o u n d e r s t an d ? C l i c k y o u r a n s w e r . T h e p a r t i ci p a n t s

    s e l e c t e d o n e o f t h e s e v e n o p t i o n s: e x t r e m e l y e a s y , . . . v e r y e a s y ,

    easy , ne i the r easy nor d i f f i cu l t , d i f f i cu l t , ve ry d i f f i cu l t ,

    and ex t rem ely d i f fi cu l t . A menta l load ra ting rang ing f rom 1 to 7

    was the re fo re co l lec ted f rom each pa r t i c ipan t .

    Performance tes t Par t i c ipan t s then comple ted a se r ie s o f 10

    mul t ip le -ch o ice ques t ions . The ques t ions asked pa r t i c ipan t s to

    s o l v e f o r a n u n k n o w n v a r i a b l e f r o m g i v e n v a r i a b l e s u s i n g t h e

    c u t t i n g s p e e d n o m o g r a m . F o u r n u m e r i c a l o r v e r b a l a l t e r n a t i v e s

    were p rov ided fo r each mu l t ip le -cho ice ques tion . The f i r st ques t ion

    w a s W h a t R .E M . s h o u l d b e u s e d t o d r il l a 4 0 m m d i a m e t e r h o l e i n

    the a lumin ium work p iece? The a l te rna t ive responses were 1000 ,

    500 , 300 , and 100 , wi th the cor rec t answer in i t a l i c s . Th is

    ques t ion was s imi la r to the p rob lem s ta tement p resen ted dur ing the

    exper imenta l t r a in ing phase . The remain ing 9 ques t ions invo lved

    se t s and sequences o f p rocedura l s t eps tha t were d i f fe ren t f rom

    those p resen ted dur ing the exper imenta l t r a in ing phase . The

    c h a n g e s w e r e a f f e c t e d b y a l t e r i n g t h e p r o b l e m ' s k n o w n s

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    6/11

    LEARNER EXPERIENCE AND MULTIMEDIA INSTRUCTION

    3

    and unknowns. For example, the question Wh at material from

    those listed below can be drilled with R.P.M. of 200, if the required

    work diameter is 50 mm? (cast iron, aluminium, brass, copper)

    involved first selecting a horizontal l ine (200 rpm) and a vertical

    line (50 ram), locating their intersection point, selecting a diagonal

    line running through the intersection point, determining a numeri-

    cal value of cutting speed corresponding to that l ine (32 rn/min)

    and, finally, selecting materials from the table that included that

    value in the cutting speed range. Some questions, such as Assum e

    you ha ve a drilling machine with a ran ge of R.P.M. from 16 to 500.

    Which work diameters from those listed below cannot be drilled in

    an aluminium workpiece? (500, 100, 50,

    30),

    included given

    parameters determined as a whole range rather than a specific value

    (e.g., 16-500 rpm). These questions required participants to

    consider several l ines for each parameter. Up to 1.5 min were

    allowed for each of the 10 questions. The responses to each

    multiple-choice item were electronically recorded and judged as

    either correct or incorrect, resulting in a score out of 10 for each

    participant.

    First training session.

    After the test , all participants were

    provided with training in using the cutting speed nomogram. The

    training session began with a worked exam ple that contained visual

    textual explanations of the nomogram indicating how to determine

    the appropriate rpm to drill a 25-mm diameter hole in a bronze

    workpiece. The participants were allowed to study these instruc-

    tions as long as they needed. Next, all participants were presented

    with a series of 50 multiple-choice questions similar to those used

    during the testing phase. These questions were self-paced and

    included automatic feedback

    (correct

    or

    incorrect)

    immediately

    after each response (trainees reattempted questions until they

    generated correct answers). This training period lasted 1-1.5 hr.

    Stage 2

    Stage 2 too k place 1 week after the first stage.

    Second training session.

    This stage started with another train-

    ing session that consisted of more exercises (multiple-choice

    questions) similar to those used during the first training session.

    The training session (which lasted about an hour) was followed by

    the experimental training and testing.

    Experimental training.

    This time, a subs tantially different

    representation of the cutting speed nomogram was used during the

    instructional phase (the nomogram used for calculating rpm for

    turning operations). The materials used a completely different

    layout and were m ore complicated than those used during the first

    stage test and training sessions (see Figure 2). The new nomogram

    used a more com plicated table (with separate columns for rough

    turning, finish turning, and drilling operations), which was inte-

    grated with the diagram. These materials were new to the students,

    and the subsequent test problems could not be solved by a direct

    application of previously acquired procedures. For exam ple, search-

    ing for given m aterial in the table now required taking into account

    what kind of operation was involved (rough turning, finish turning,

    or dril l ing). On the other hand, if the required materials were

    located, the corresponding diagonal l ines could be selected directly

    without remembering the cutting speed range values. The instruc-

    tional materials were presented in the same four formats, and

    participants remain ed in the same groups as in the first stage.

    Figure

    2. A section of the computer screen presentation for instruction on the cutting speed

    nomogram (diagram-with-visual-text and diagram -with-visual-plus-audio-text nstructional formats

    used in Stage 2 of Exp eriment 1).

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    7/11

      32

    KALYUGA, CHANDLER, AND SWELLER

    Subjective ratings. Afte r pa r t i c ipan t s s tud ied the ins t ruc t ions ,

    t h e y m a d e s u b j e c t i v e ra t i n g s o f m e n t a l l o a d u s i n g t h e s a m e 7 - p o i n t

    sca le used in S tage 1 .

    Performance test .

    The pe r fo rmance t e s t inc luded a se r ie s o f 10

    mul t ip le -c ho ice ques t ions tha t r equ i red pa r t i c ipan t s to use the

    c u t t i n g s p e e d n o m o g r a m v e r s i o n p r e s e n t e d d u r i n g e x p e r i m e n t a l

    t r a in ing to so lve fo r an unknow n var iab le f rom g iven va r iab les .

    S im i la r to S tage 1 , a l l ques t ions excep t the f i r st invo lved se t s and

    sequences o f p rocedura l s t eps tha t were d i f fe ren t f rom those

    presen ted dur ing the exper im enta l t r a in ing phase because d i f fe ren t

    k n o w n s a n d u n k n o w n s w e r e u s e d ( e. g . , A s s u m e y o u h a v e a la t h e

    wi th a range o f R .P.M. f rom 40 to 500 . Which w ork d iamete r s f rom

    those l i s t ed be low canno t b e f in i shed wi th [an] a l loy s tee l

    work p iece? 6 , 20, 50 , 100 ) . The ques t ions dea l t w i th mate r ia l s ,

    opera t ions , and se t s o f pa ramete r va lues tha t were represen ted in

    t h e n e w v e r s i o n o f t h e n o m o g r a m .

    A l l c o m p u t e r - b a s e d t r a in i n g p r o g r a m s w e r e d e s i g n e d t o d i s p l a y

    ins t ruc tiona l ma te r ia l s a s we l l a s au tomat ica l ly record a l l exper imen-

    ta l da ta , inc lud ing lea rne rs ' r e sponses to d i f fe ren t t e s t i t ems ,

    response t imes , and sub jec t ive ra t ings o f men ta l load . We ca lcu-

    la ted ins t ruc t iona l e f f i c iency measures us ing Paas an d van M er r ien-

    boer ' s (1993 , 1994) p rocedure . Th is approach combines measures

    of cogn i t ive load wi th measures o f t e s t pe r fo ianance to de r ive

    in fo rmat ion on the re la t ive e f f i c iency o f ins t ruc t ion . I t a s sumes tha t

    ins t ruc t iona l p resen ta t ions a re e f f i c ien t i f they p roduce be t t e r

    pe r fo rmance resu l t s wi th l e ss cogn i t ive load requ i red . We ca lcu-

    la ted e f f i c iency va lues by conver t ing cogn i t ive load and pe r fo r -

    mance measures in to z scores ( s tandard iz ing those measures ac ross

    c o n d i t i o n s) a n d c o m b i n i n g z s c o r e s u s i n g t h e f o r m u l a E =

    (P - R) / V '2 , where E = e f f i c iency , P = pe r fo rmanc e z score , and

    R = ra t ing sca le z score ( the S /2 in the denom ina to r is used to make

    t h e g r a p h i c a l i n te r p r e ta t i o n o f t h e f o r m u l a m o r e s t r a i g h t f o r w a r d - -

    see Paas and van Mer r ienboer [1993] fo r de ta i l s ) . Us ing th i s

    fo rmula , i f pe r fo rman ce and ra t ing z scores a re equa l (P = R) ,

    e f f i c iency i s ze ro (E = 0 ) . I f the pe r fo rm ance z score i s h igher than

    the ra t ing z sc ore (P > R) , ins t ruc t iona l e f fi c iency is pos i t ive

    (E > 0 ) . On the o the r hand , i f the pe r fo rman ce z score i s lower than

    the ra t ing z score (P < R) , ins t ruc t iona l e f f i c iency is nega t ive

    (E < 0 ) . Us ing the e f f i c iency measure se rves as a sa feguard aga ins t

    the poss ib i l i ty tha t sub jec t ive ra t ings measu re som e o the r sub jec -

    t ive charac te r i s t i c s unre la ted to ins t ruc t iona l ou tcom es (e .g . , se l f -

    conf idence) ra the r than cogn i t ive load . I f l ea rne rs f ind someth ing

    easy to l ea rn b u t pe r fo rm be t te r on t e s t s (h igh e f f i c iency) , the

    l ike l ihood tha t they a re ra t ing cogn i t ive load ra the r than some o the r

    fac to r i s inc reased .

    R e s u l t s a n d D i s c u s s i o n

    W e c o n d u c t e d a s e r i e s o f 4 ( i n s t r u c t i o n a l f o r m a t ) × 2

    ( S t a g e 1 o r 2 ) a n a l y s e s o f v a r i a n c e ( A N O V A s ) w i t h r e p e a t e d

    m e a s u r e s o n t h e s e c o n d f a c t o r , u s i n g t h e d a t a f r o m E x p e r i -

    m e n t 1 . T h e d e p e n d e n t v a r i a b l e s w e r e i n s t ru c t i o n t im e ( t i m e

    e a c h l e a r n e r s p e n t s t u d y i n g i n s t r u c t io n s d u r i n g e x p e r i m e n t a l

    t r a i n i n g ) , s u b j e c t i v e r a t i n g s o f m e n t a l e f f o r t , te s t p e r f o r -

    m a n c e s c o r e s o n t h e m u l t i p l e - c h o i c e i te m s , a n d i n s t r u c t i o n a l

    e f f i c ie n c y m e a s u r e s . M e a n s a n d s t a n d a r d d e v i a t i o n s a r e

    d i s p l a y e d i n T a b l e 1 .

    T h e a n a l y s i s o f o v e r a l l g r o u p m a i n e f f e c ts r e v e a l e d

    s i g n i f ic a n t d i ff e r e n c e s b e t w e e n i n s t r u c ti o n a l f o r m a t s , F ( 3 ,

    5 5 ) = 1 2 .2 7 , M S E = 4 4 5 8 . 1 5 , p < . 0 0 0 1 , f o r t h e i n s t r u c t i o n

    t i m e ( o v e r a l l i n s t r u c t i o n t i m e f o r t h e d i a g r a m - o n l y g r o u p

    w a s l o w e r t h an f o r o t h e r g r o u p s ) ; F ( 3 , 5 5 ) = 5 . 5 1 , M S E =

    1 . 6 9, p < . 0 0 5 , f o r t h e s u b j e c t i v e r a t i n g d a t a ( o v e r a l l

    d i f f ic u l t y r a t i n g f o r t h e d i a g r a m - w i t h - a u d i o - t e x t g r o u p w a s

    l o w e r t h a n f o r o t h e r g r o u p s ) ; a n d F ( 3 , 5 5 ) = 3 . 8 7,

    M S E =

    1 . 8 2 , p < . 0 5 , f o r i n s t r u c t i o n a l e f f i c i e n c y ( o v e r a l l i n s t r u c -

    t i o n a l e f fi c i e n c y f o r t h e d i a g r a m - w i t h - a u d i o - t e x t g r o u p w a s

    h i g h e r t h a n f o r o t h e r g r o u p s ) . N o s i g n i f ic a n t o v e r a ll m a i n

    e f f e ct o f t h e m u l t i p l e - c h o i c e i t e m s w a s f o u n d , F ( 3 , 5 5 ) =

    1 .85 , M S E = 6 . 5 5 ( b u t n o t e t h a t b e c a u s e w e p r e d i c t e d t h a t

    t h e r e l a t i v e e f f e c t iv e n e s s o f t h e i n s t r u c t i o n a l p r o c e d u r e s

    w o u l d a l t e r w i t h e x p e r i e n c e , a n o n s i g n i f i c a n t m a i n e f f e c t i s

    t o b e e x p e c te d ) . T h e a n a l y s i s o f S t a g e 1 - S t a g e 2 m a i n

    T a b l e 1

    M ean Ins t ruc t i on T i mes , R a t i ngs , Scores , and L eve l s o f I ns t ruc t i ona l E f f i c i ency by

    E xp er i me nt a l Cond i t ion an d S t age o f T est ing , E x per i m ent I

    Exper im enta l cond i t ion and s tage o f t e s ting

    D i a g r a m w i t h

    v i sua l t ex t

    D i a g r a m w i t h

    a u d i o p l u s D i a g r a m w i t h

    v i s u a l t e x t a u d i o t e x t D i a g r a m o n l y

    Variab le 1 2 1 2 1 2 1 2

    Ins t ruc t ion t ime(s )

    M 163 75 155 100

    SD

    94 48 64 66

    Rat ing sca lea

    M 3.0 3.8 2.9 3.3

    SD

    1.1 0.9 1.1 1.2

    Ques t ion scoreb

    M 5.8 5.5 5.6 5.0

    SD

    1.9 2.4 2.1 2.4

    Ins t ruc t iona l e f f i c iency

    M 0 .0 3 - 0 . 1 7 - 0 . 0 4 - 0 . 0 3

    SD

    1.12 0.94 1.16 1.30

    128 69 44 22

    37 20 21 16

    2.1 2.5 3.7 3.5

    1.0 1.2 1.0 0.8

    7.1 6.4 5.1 6.2

    1.4 2.0 1.7 2.3

    0 .91 0 .88 -0 .5 9 0 .23

    0.73 1.04 0.95 1.05

    aRat ings were made on 7 -po in t sca les (1 =

    extremely easy, 7 = extremely difficult ,

    b M a x i m u m

    ques t ion score = 10 . CActua l r ange fo r ins t ruc t iona l e f f i c iency was f rom - 2 . 34 to 2 .20 (poss ib le

    range i s -oo , +oo).

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    8/11

    LEARNER EXPERIENCEAND MULTIMEDIA NSTRUCTION 13 3

    ef fec t s revea led s ign i f i can t d i f fe rences fo r ins t ruc t ion t im e ,

    F (1 , 55) = 79 . 98 , M S = 1,154.58, p < .0001, and

    subjec t ive m enta l ra t ings , F (1 , 55) = 7 . 93 ,

    M S

    = 0.47, p <

    . 01 . N o s i g n if i c an t e f f e c t w a s f o u n d o f t h e m u l t i p l e - c h o i c e

    i t em s and ins t ruc t iona l e f f i c iency on S tage 1 -S ta ge 2 . Thus ,

    wi th t ra in ing , pa r t i c ipan t s spen t s ign i f i can t ly l e s s t im e

    s tudying ins t ruc t ions . Never the le s s , they ra ted the ins t ruc -

    t iona l m a te r ia l s used in the S tage 2 t e s t s ign i f i can t ly h ighe r

    i n d i f f ic u l ty . T h e s e m a t e r ia l s w e r e m o r e c o m p l i c a t e d t h a n

    a n d d i f f e r e n t f r o m t h o s e u s e d d u r i n g t h e S t a g e 1 te s t a n d

    t ra in ing s es s ions .

    B e c a u s e o n e o f t h e m a i n p u r p o s e s o f t h is e x p e r i m e n t w a s

    t o s t u d y t h e d y n a m i c s o f c h a n g e w i t h e x p e r i e n c e , w e w e r e

    p r i m a r i l y i n t e r e s t e d i n t h e i n t e r a ct i o n e f f e c t b e t w e e n g r o u p s

    and s tages . We expec ted tha t d i rec t t ra in ing wi th cu t t ing

    s p e e d n o m o g r a m s w o u l d c o n t r i b u t e t o t h e d e v e l o p m e n t o f

    r e l a t i v e e x p e r i e n c e i n t h i s s p e c i f i c d o m a i n a n d p r o d u c e a

    s w i t c h o f t h e m o s t e f f e c t i v e m e t h o d o f i n s t r u c t i o n . W e

    p r e d i c t e d t h a t t h e d i a g r a m - a n d - t e x t g r o u p s ( d i a g r a m - w i t h -

    v i sua l - t ex t , d iagram -wi th-v i sua l -p lus -aud io- tex t , and d ia -

    g r a m - w i t h - a u d i o - t e x t g r o u p s ) w o u l d p e r f o r m r e l a t i v e l y b e t-

    t e r o n t h e S t a g e 1 t e st t h a n t h e d i a g r a m - o n l y g r o u p w o u l d .

    W e e x p e c t e d t h a t , b e c a u s e o f a m o d a l i t y e f f e c t , t h e d i a g r a m -

    w i t h - a u d i o -t e x t g r o u p w o u l d p e r f o r m b e t t e r t h a n t h e d i a g ra m -

    wi th-v i sua l - t ex t g roup . In con t ra s t , on the S tage 2 t e s t , the

    i n c r e a s e i n e x p e r i e n c e d u e t o t r a i n i n g s h o u l d r e n d e r t h e

    tex tua l based m a te r ia l redundant , re su l t ing in a re la t ive ly

    i m p r o v e d p e r f o r m a n c e b y t h e d i a g r a m - o n l y g r o u p . I n t e r a c-

    t i o n e f f e c ts s h o u l d b e a c o n s e q u e n c e o f t h is p a t t e r n o f

    resul ts .

    In accordance wi th th i s p red ic t ion , the in te rac t ion da ta o f

    a l l the 4 × 2 AN OV As w ere o f m a jor in te re s t in th i s s tudy .

    There were s ign i f i can t in te rac t ions fo r the ins t ruc t ion t im e ,

    F (3 , 55) = 4 . 71 , M S = 1 , 154 .58 , p < . 01 ; fo r the m ul -

    t ip le -cho ice i t em s , F (3 , 55) = 2 . 83 , M S = 1.92, p < .05;

    and fo r the ins t ruc t iona l e f f i c iency m easures , F (3 , 55) =

    4.03, M S = 0 . 38 , p < . 05 . A m arg ina l ly s ign i f i can t

    i n t e r a c ti o n w a s o b s e r v e d f o r t h e s u b j e c t iv e r a ti n g s o f m e n t a l

    load , F (3 , 55) = 2 . 72 ,

    M S

    = 0 . 47 , p = . 053 . Th e

    s ign i f i can t in te rac t ions sugges t tha t the m os t e f f i c ien t m o de

    o f i n s t r u c t i o n d e p e n d s o n t h e l e v e l o f e x p e r i e n c e o f l e a r ne r s .

    T h e r e w e r e s i g n if i c an t d i f f er e n c e s b e t w e e n g r o u p s i n t h e

    t im e they took to p roce s s the ins t ruc tions , F (3 , 56) = 12 . 05 ,

    M S = 3 , 677 . 94 , p < . 0001 , fo r S tage 1 , and F (3 , 55) =

    8.48, M S = 1 , 878 . 05 , p = . 0001 , fo r S tage 2 . Th e ins t ruc -

    t i o n t i m e f o r t h e d i a g r a m - o n l y g r o u p w a s s i g n i f i ca n t l y lo w e r

    t h a n t h a t o f a n y o f t h e t h r e e o t h e r g r o u p s w i t h n o o t h e r

    s i g n i f i c a n t d i f f e r e n c e s ( N e w m a n - K e u l s t e s t ; s e e W i n e r ,

    1971) . F or sub jec t ive ra t ings o f m enta l load , the re were

    s ign i f i can t s im ple e f fec t s : F (3 , 56) = 6 . 67 ,

    M S

    = 1.06, p <

    . 001 , fo r S tage 1 and F (3 , 55 ) = 3 . 96 ,

    M S

    = 1 . 10 , p < . 05 ,

    f o r S t a g e 2 . I n S t a g e 1 th e d i a g r a m - w i t h - a u d i o - t e x t f o r m a t

    w a s p e r c e i v e d t o b e s i g n i f i ca n t l y l o w e r i n m e n t a l l o a d t h a n

    a n y o f t h e t h r e e o t h e r f o r m a t s , w i t h n o o t h e r s i g n i fi c a n t

    d i f f e re n c e s ( N e w m a n - K e u l s ) . F o r S t a g e 2 , t h e d i a g r a m - w i t h -

    a u d i o - t e x t f o r m a t w a s p e r c e i v e d t o b e s i g n if i c an t l y l o w e r i n

    m e n t a l l o a d t h a n t h e d i a g r a m - w i t h - v i s u a l - te x t a n d d i a g r a m -

    only fo rm a ts , wi th no o the r s ign i f i can t d i f fe rences . F or the

    m ul t ip le -cho ice i t em s , the re was a s ign i f i can t s im ple e f fec t

    fo r S tage 1 on ly , F (3 , 56 ) = 3 . 92 ,

    M S

    = 3 . 1 7 , p < . 05 . T h e

    d i a g r a m - w i t h - a u d i o - t e x t g r o u p p e r f o r m e d s i g n if i c an t l y b e t -

    t e r t h a n a n y o f t h e t h r e e o t h e r g r o u p s w i t h n o o t h e r

    s i g n if i c an t d i f f e r e n c e s ( N e w m a n - K e u l s ) . F o r i n s tr u c t io n a l

    e f f i c iency , the re was a s ign i f i can t s im ple e f fec t o f S tage 1

    on ly , F (3 , 56) = 6 . 21 ,

    M S

    = 1.01, p = .001. Th e

    d iagram -wi th-aud io- tex t g roup was s ign i f i can t ly m ore e f f i -

    c i e n t t h a n a n y o f t h e t h r e e o t h e r g r o u p s , w i t h n o o t h e r

    s i g n if i c a n t d if f e r e n c e s ( N e w m a n - K e u l s ) .

    T h e i n e x p e r i e n c e d t r a i n e e s c l e a r l y b e n e f i t e d m o s t f r o m

    t h e d i a g r a m - w i t h - a u d i o - t e x t f o r m a t . T h e y p e r f o r m e d s i g n if i-

    can t ly be t t e r , wi th s ign i f i can t ly lower m enta l load , than

    t ra inees who s tud ied the d iagram -wi th-v i sua l - t ex t fo rm a t .

    T h i s r e s u l t i s c o n s i s t en t w i t h t h e m o d a l i t y e f f e c t o f p r e v i o u s

    s tud ies (Ka lyuga e t a l ., 1999; May er , 1997; M aye r M o-

    reno , 1998; M ousavi e t a l ., 1995; Tmdall-Ford e t al ., 1997). The

    d i a g r a m - w i t h - a u d i o - t ex t g r o u p a l s o p e r f o r m e d s i g n i f i ca n t l y

    be t t e r , wi th s ign i f i can t ly lower m en ta l load , than the d iagram -

    wi th-v i sua l -p lus -aud io- tex t g roup . Menta l in tegra t ion o f the

    v i sua l t ex t wi th the d iagram and aud io t ex t requ i red add i -

    t i o n a l c o g n i t i v e r e s o u r c e s a n d o v e r l o a d e d p a r t i c i p a n ts ' w o r k -

    i n g m e m o r y c a p a c i ty . T h e r e w a s a r e d u n d a n c y e f f e c t s i m i la r

    to tha t in our 1999 s tud y (Ka lyug a e t a l ., 1999) .

    A f t e r p a r t i c ip a n t s w e r e e x p o s e d t o t r a in i n g m a t e r ia l s l o n g

    e n o u g h t o b e c o m e m o r e e x p e r i e n c e d i n t h e d o m a i n , t h e

    d i s t i n c t a d v a n t a g e o f t h e d i a g r a m - w i t h - a u d i o - t e x t f o r m a t

    o v e r t h e o t h e r f o r m a t s d i s a p p e ar e d . T h e c o n t r a s t b e t w e e n t h e

    pre - and pos t t ra in ing s tages i s e spec ia l ly d i s t inc t ive when

    c o m p a r i n g t h e d i a g r a m - o n l y a n d t h e d i a g r a m - w i t h - a u d i o -

    t e x t f o r m a t s . T h e s e t w o f o r m a t s r e p r e s e n t e d , c o r r e s p o n d -

    ing ly , the l eas t and m os t e f fec t ive fo rm a ts in i t ia l ly , wi th

    s ta t i st i ca l ly s ign i f i can t d i f fe rences on a l l the pa ram ete rs , b u t

    t h e d i f f e r e n c e s r e d u c e d s u b s t a n t i a l l y - - a n d , i n t h e c a s e o f t h e

    te s t re su l t s , e s sen t i a l ly d i s appea red- - in S tage 2 . As the

    p a r t i c ip a n t s ' l e v e l o f e x p e r i e n c e w a s r a i se d , t h e p e r f o r m a n c e

    o f t h e d i a g r a m - o n l y g r o u p i m p r o v e d m o r e t h a n t h e p e r f o r -

    m a n c e o f a n y o t h e r g r o u p . I n i t i a ll y m o s t d i s a d v a n t a g e d , t h i s

    g r o u p i m p r o v e d m o r e r a p i d l y th a n a n y o t h e r g r o u p . T h u s , i n

    a c c o r d a n c e w i t h o u r p r e d i c ti o n s , a s e x p e r i e n c e i n c r ea s e d ,

    t h e r e l a t i v e i m p r o v e m e n t i n p e r f o r m a n c e o f t h e d i a g r a m -

    o n l y g r o u p w a s s u p e r i o r to t h e o t h e r t h r e e d i a g r a m - w i t h - t e x t

    groups . N ever the le s s , a redun dancy e f fec t wi th a s t a t i s t i ca lly

    s i g n if i c an t s u p e r i o r p e r f o r m a n c e b y t h e d i a g r a m - o n l y g r o u p

    was no t ob ta ined . In te rac t ion e f fec t s obse r ved in th i s expe r i -

    m e n t m a y h a v e o c c u r r e d n o t b e c a u s e o f c o g n i t i v e l o a d

    f a c t o r s b u t s i m p l y b e c a u s e t h e d i a g r a m - o n l y g r o u p h a d m o r e

    r o o m f o r i m p r o v e m e n t . O b t a i n i n g a r e d u n d a n c y e f f e c t w a s

    es sen t i a l fo r t e s t ing cogni t ive load pred ic t ions . W e expec te d

    t h a t a r e d u n d a n c y e f f e c t m i g h t o c c u r w i t h m o r e t r a i n i n g

    prov ided to pa r t i c ipan t s .

    B e f o r e m o v i n g o n w i t h a d d i t i o n a l t r a i n i n g , w e r e -

    e x a m i n e d a d d i t i o n a l e x p e r i m e n t a l co n d i t i o n s u n d e r w h i c h a

    redun dancy e f fec t m igh t occur . In Exper im ent 1 , a l l th ree

    d iagram -and- tex t ins t ruc t iona l fo rm a ts (d iagram wi th v i sua l

    t ex t , d iagram wi th aud io t ex t , and d iagram wi th aud io p lus

    v i sua l t ex t s ) a l lowed lea rne rs to ignore v i sua l and aud i to ry

    tex tua l exp lana t ions i f they wished . Visua l exp lana t ions

    w e r e s e p a r a t e d f r o m t h e d i a g r a m a n d c o u l d b e e a s i l y

    sk ipped by c l i ck ing on them wi tho u t read ing . S im i la r ly , the

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    9/11

      34

    KALYUGA, CHANDLER, AND SWELLER

    a u d i t o r y e x p l a n a t i o n s c o u l d b e i g n o r e d o r t h e d i a g r a m s

    i n s p e c t e d a t l e i s u r e b e f o r e a n d a f t e r t h e a u d i t o r y e x p l a n a t i o n

    h a d b e e n h e a r d . T h u s , a l l t h e d i a g r a m - a n d - t e x t fo r m a t s i n

    E x p e r i m e n t 1 a l l o w e d u n i n t e r r u p t e d i n s p e c t i o n o f t h e d i a -

    g r a m w i t h o u t i n c r e a s e d c o g n i t i v e l o a d f o r l o n g i n t e r v a l s o f

    t i m e . D u r i n g t h a t t i m e t h e d i a g r a m - a n d - t e x t f o r m a t s w e r e

    e f f e c t i v e l y e q u i v a l e n t t o t h e d i a g r a m - o n l y f o r m a t f o r l e a r n -

    e r s w h o i g n o r e d t h e t e x t u a l e x p la n a t i o n s . W e c a n h y p o t h -

    e s i z e t h at t h e r e d u n d a n c y e f f e c t m i g h t b e o b t a i n a b l e w i t h t h e

    d i a g r a m - a n d - t e x t i n s tr u c t i o n a l f o r m a t s u s i n g e x p e r i e n c e d

    l e a r n e r s i f e x p e r i m e n t a l c o n d i t i o n s r e d u c e t h e p o s s i b i l i t y o f

    u n i n t e r r u p t e d f r e e i n s p e c t i o n o f th e d i a g r a m a l o n e . W e

    t e s t e d th i s a s s u m p t i o n i n E x p e r i m e n t 2 .

    E x p e r i m e n t 2

    T h i s e x p e r i m e n t w a s d e s i g n e d t o e x p l o r e w h e t h e r a

    r e d u n d a n c y e f f e c t w o u l d b e o b t a i n a b l e w i t h e x p e r i e n c e d

    l e a r n e r s u n d e r c o n d i t i o n s i n w h i c h l e a r n e rs i n t h e d i a g r a m -

    a n d - t e x t g r o u p s c a n n o t i n s p e c t a d i a g r a m w i t h o u t s i m u l t a -

    n e o u s l y a t t e n d i n g t o t h e a u d i t o r y e x p l a n a t i o n s . B y t h e t i m e

    t h i s e x p e r i m e n t t o o k p l a c e ( 2 ½ m o n t h s a f t e r E x p e r i m e n t 1 ),

    o n l y 3 8 a p p r e n t i c e s a n d t r a in e e s f r o m t h o s e w h o p a r t i c i p a t e d

    i n E x p e r i m e n t 1 w e r e a v a i l a b l e f o r te s t in g . W e c o m p a r e d

    o n l y t w o i n s t r u c ti o n a l fo r m a t s : t h e d i a g r a m - o n l y f o r m a t a n d

    t h e d i a g r a m - w i t h - a u d i o - t e x t f o r m a t . T h i s t i m e , th e d i a g r a m -

    w i t h - a n d i o - t e x t g r o u p h a d t o c o n t i n u o u s l y a t t e n d t o t h e

    a u d i t o r y e x p l a n a t i o n s w i t h o u t t h e p o s s i b i l i t y o f f re e i n s p e c -

    t i o n o f t h e d i a g r a m a l o n e .

    I n a c c o r d w i t h c o g n i t i v e l o a d t h e o r y , w e e x p e c t e d t h a t t h e

    t r a i n ee s , w h o n o w w e r e c o n s i d e r a b l y m o r e e x p e r i e n c e d t h a n

    w h e n t h e y p a r t i c i p a t e d i n E x p e r i m e n t 1 , w o u l d b e n e f i t m o r e

    f r o m a s e l f - c o n t a i n e d , d i a g r a m - o n l y f o r m a t t h a n f r o m a

    d i a g r a m - w i t h - a u d i o - t e x t f o r m a t . A t t e n d i n g t o th e r e d u n d a n t

    a u d i t o r y e x p l a n a t i o n s s i m u l t a n e o u s l y w i t h i n s p e c t i o n o f th e

    d i a g r a m s h o u l d i m p o s e a n a d d i t i o n a l c o g n i t i v e l o a d a n d t h u s

    d e c r e a s e t h e p e r f o r m a n c e a n d e f f i c i e n c y o f th e i n s t r u c t i o n a l

    p r e s e n t a t i o n .

    M e ~ o d

    P a r t i c i p a n t s

    Thi r ty -e igh t appren t ices and t r a inees who had pa r t i c ipa ted in

    Exper im ent 1 took pa r t in th i s s tudy . Dur ing the 2~/2 months tha t

    had passe d s ince Exper imen t 1 , a l l the pa r t i c ipan t s had con t inu ed

    the i r normal , schedu led , fu l l - t ime t ra in ing a t the i r company ' s

    t ra in ing cen te r . Tha t t r a in ing inc luded p rac t i ce in pe r fo rming the

    dr i l l ing-and- tu rn ing opera t ions used in th i s exper im ent , a l though

    the p rocedures used wh en se lec t ing appropr ia te rpm d i f fe red f rom

    t h e c u t t in g s p e e d n o m o g r a m m e t h o d s .

    M a t e r i a l s a n d P r o c e d u re

    Par t i c ipan t s were randomly a l loca ted to two g roups (19 l ea rne rs

    in each g roup , r egard less o f the i r g roup ing in Exper iment 1 )

    c o r r e s p o n d i n g t o t h e d i a g r a m - o n l y a n d d i a g r a m - w i t h - a u d i o - t e x t

    fo rmats o f ins t ruc t iona l ma te r ia l s . A l l the ins t ruc t iona l and te s t

    mate r ia l s were compute r based and ag a in cons t ruc ted wi th Author

    ware Professional

    for M acinto sh. T he computer-based training pro-

    gram was d es igned to d i sp la y ins t ruc t iona l ma te r ia l s a s we l l a s

    au tom at ica l ly record a l l exper im enta l da ta , inc lud ing lea rn e rs ' r e -

    sponses to d i f fe ren t t e s t i tems a nd sub jec t ive ra tings o f men ta l load .

    A br ie f ( abou t 20-2 5 min) t r a in ing sess ion p reced ed the exper i -

    menta l ins t ruc t ion and te s t ing to remind lea rne rs abou t mate r ia l s

    t h e y h a d s t u d i e d p r e v i o u s ly a n d t o p r o v i d e s o m e a d d i t io n a l t ra i n i n g

    to a l low suf f ic ien t exper ience to render the aud i to ry t ex tua l

    mate r ia l r edundan t . The t r a in ing mate r ia l s inc luded s ix worked

    examples (wi th t ex tua l exp lana t ions in tegra ted d i rec t ly in to the

    d i a g r a m ) . T h r e e w o r k e d e x a m p l e s w e r e b a s e d o n t h e n o m o g r a m

    represen ta t ion used dur ing the S tage 1 t e s t in Exper iment 1 , and the

    o t h e r t h r e e w o r k e d e x a m p l e s w e r e b a s e d o n t h e n o m o g r a m

    represen ta t ion used dur ing the S tage 2 t e s t . Each worked example

    w a s f o l l o w e d b y a m u l t i p le - c h o i c e i t e m w i t h a s i m i l a r p r o b l e m a n d

    w i t h im m e d i a t e fe e d b a c k ( c o r r e c t o r i n c o r r e c t ) . B e f o r e

    read ing a pa ragraph , a t r a inee had to c l i ck on i t ( l ea rne rs were

    ins t ruc ted to do th i s a t the beg inn ing o f the t r a in ing sess ion and

    were con t inuous ly reminded dur ing the t r a in ing) . Then , sc reen-

    b a s e d h i g h l i g h ts o f t h e a p p r o p r i a te e l em e n t s o f t h e n o m o g r a m ( t h e

    mate r ia l name , cu t t ing speed rang e , l ines , in te r sec t ion po in t s , e tc . )

    appeared on the sc reen .

    The exper imenta l ins t ruc t ions were based on a ve r s ion o f the

    n o m o g r a m t h a t w a s d i f f e r e n t f r o m t h o s e u s e d i n E x p e r i m e n t 1 ,

    though i t inc luded e lements o f the ve rs ions used dur ing bo th

    prev ious s tages . S im i la r to S tage 1 , Exper im ent 1 , l inea r sca les

    w e r e u s e d f o r b o t h h o r i z o n ta l a n d v e r t i c a l ax e s , w i t h n u m b e r s o f

    rpm in c reas ing in the top- to -bo t tom d i rec t ion . Numer ica l va lues o f

    paramete r s were s imi la r to those represen ted in the S tage 1 ,

    Exper iment 1 , ve r s ion o f the nomogram (a l though the range o f

    d iamete r s was shor te r , ex tend ing f rom 5 mm to on ly 125 mm) . A

    l i s t o f ma te r ia l s was t aken f ro m the t ab le used in the ins t ruc t ions o f

    S tage 1 , Exper iment 1 , bu t the t ab le was in tegra ted wi th the

    n o m o g r a m i n a w a y u s e d i n t h e i n s t ru c t io n s o f S t a g e 2 , E x p e r i m e n t

    1 . These m ate r ia l s were re la t ive ly new to s tuden ts , and the

    subsequen t t e s t p rob lems cou ld no t be so lved by a d i rec t app l ica -

    t ion o f rou t ines iden t ica l to those used dur ing e i the r S tage 1 o r

    S tage 2 o f Exper iment 1 . By the t ime pa r t i c ipan t s encoun te red

    t h e s e m a t e r i a ls , t h e y s h o u l d h a v e p o s s e s s e d k n o w l e d g e a b o u t u s i n g

    n o m o g r a m s i n v a r i o u s ty p e s o f t a s k s . T h e y w e r e e x p e c t e d t o b e

    exper ienc ed in us ing nom ograms , b u t no t th i s pa r t i cu la r ve r s ion o f

    the nomogram . As a consequence , they s t i l l r equ i red add i t iona l

    lea rn ing to become fami l i a r wi th th i s nove l ve r s ion o f the

    nom ogram (e .g . , to f ind ou t where and ho w d i f fe ren t pa rame te r s a re

    represen ted in the new ou t l ine , e tc .) .

    The d iagram -wi th -aud io - tex t fo rmat inc luded a wr i t t en t ex tua l

    p r o b l e m s t a te m e n t n e x t t o t h e n o m o g r a m : F i g u r e o u t t h e st e p s t o

    de te rmine the appro pr ia te R .P .M. to d r i l l a 25 m m h o le in a b ronze

    w o r k p i e c e . A u d i t o r y e x p l a n a ti o n s o f p r o c e d u r a l st e p s i n u si n g t h e

    n o m o g r a m t o s o l v e t h e s t a t ed p r o b l e m s t a rt e d i m m e d i a t e l y a f t e r th e

    ins t ruc t ion was d i sp layed . Consecu t ive s teps fo l lowed each o the r

    wi thou t in te r rup t ions . The d iagram -wi th -aud io - tex t fo rmat was

    d isp lay ed fo r the 45 s tha t were necessa ry to a r t i cu la te a loud a l l the

    tex tua l exp lana t ions and te rmina ted a f te r the l a s t exp lana t ion had

    b e e n v e r b a l i z e d . T h e d i a g r a m - o n l y f o rm a t w a s a l s o d i s p l a y e d f o r

    45 s and inc luded the same tex tua l p rob lem s ta tement nex t to the

    n o m o g r a m . N o e l e m e n t s o f t h e n o m o g r a m w e r e h i g h l i g h t e d i n

    e i the r fo rmat . Thus , the p resence o f aud i to ry exp lana t ions repre -

    sen ted the on ly d i f fe rence be tween the two expe r imenta l fo rmats .

    Af te r 45 s o f s tudy ing the ins t ruc t ions , sub jec t ive measures o f

    menta l load w ere co l lec ted f rom a l l pa r t i c ipan t s . Response s were

    made on a 7 -po in t sca le and were co l lec ted e lec t ron ica l ly on the

    compute r . A se r ies o f 10 mu l t ip le -cho ice ques t ions fo l lowed .

    Par t i c ipan t s were a l lowed up to 60 s to answer each ques t ion . The

    responses to each m ul t ip le -cho ice i tem w ere e lec t ron ica l ly re -

    corded and judge d as e i the r co r rec t o r incor rec t , p rov id in g a score

    of up to 10 fo r each pa r t i c ipan t .

  • 8/16/2019 Kalyuga 2000 Expert Reversal Effect

    10/11

    LEARNER EXPERIENCEAND MULTIMEDIA NSTRUCTION 35

    R e s u l t s a n d D i s c u s s i o n

    W e c o n d u c t e d a s e r ie s o f s i m p l e f a c t o r ia l A N O V A s u s i n g

    t h e d a t a f r o m E x p e r i m e n t 2 . T h e d e p e n d e n t v a r i a b le s w e r e

    s u b j e c t i v e r a t i n g s o f m e n t a l e f f o r t a n d t e s t p e r f o r m a n c e

    s c o r e s o n m u l t i p l e -c h o i c e i te m s . T h e i n d e p e n d e n t v a r i a b l e

    w a s t h e i n s t r u c ti o n a l f o r m a t ( d i a g r a m - o n l y f o r m a t v s . d i a-

    g r a m - w i t h - a u d i o - t e x t f o r m a t ) . M e a n s a n d s t a n d a r d d e v i a -

    t ions a re d i sp lay ed in Tab le 2 .

    T h e r e w e r e s i g n if i c a n t d i f f e r e n c e s b e t w e e n g r o u p s i n

    rega rd to m ul t ip le -cho ice it em s , F (1 , 36) = 4 . 19 ,

    M S E =

    6 . 04 , p < . 05 , and ins t ruc t iona l e f f i c iency , F (1 , 36) = 6 . 29 ,

    M S E

    = 1 . 14 , p < . 05 . F or sub jec t ive ra t ings o f m enta l load ,

    the re was a m arg ina l ly s ign i f i can t d i f fe rence , F (1 , 36) =

    3.52, M S E = 1 . 91 , p = . 069 .

    T h u s , t h e d i a g r a m - w i t h - a u d i o t e x t g r o u p p e r f o r m e d s i g -

    n i f i can t ly worse than the d iagra m -on ly g roup on the m ul t ip le -

    c h o i c e t e s t. A t t h e s a m e t i m e , t h e d i a g r a m - w i t h - a u d i o - t e x t

    f o r m a t w a s r a t e d m o r e d i f f ic u l t t h a n t h e d i a g r a m - o n l y

    f o r m a t ; t h a t is , a r e d u n d a n c y e f f e c t w a s o b t a i n e d u n d e r

    condi t ions tha t requ i red l ea rne rs in the d iagram -wi th-aud io-

    t e x t g r o u p t o i n s p e c t t h e d i a g r a m a n d s i m u l t a n e o u s l y l i s te n

    t o a u d i t o r y e x p l a n a ti o n s o f h o w t o u s e t h i s d i a g ra m t o s o l v e

    a s p e c if i c p r o b l e m . A t t e n d i n g t o r e d u n d a n t a u d i t o r y e x p l a n a -

    t ions im posed an add i t iona l cogni t ive load , thus dec reas ing

    p a r t i c ip a n t s p e r f o r m a n c e a n d t h e e f f i c i e n c y o f t h e d i a g r am -

    wi th-a ud io- tex t ins t ruc t iona l p resen ta t ion .

    G e n e r a l D i s c u s si o n

    A c c o r d i n g t o c o g n i t i v e l o a d t h e o r y , u n d e r s p l i t -a t t en t i o n

    c o n d i t i o n s l i m i t e d w o r k i n g m e m o r y m a y b e e f f e c t i v e l y

    e x p a n d e d b y u s i n g m o r e t h a n o n e s e n s o r y m o d a l i t y , a n d

    i n s tr u c t io n a l f o r m a t s i n w h i c h s e p a r a t e s o u r c e s o f i n f o r m a -

    t ion a re p resen ted in a l t e rna te , aud i to ry o r v i sua l , fo rm s

    m i g h t b e m o r e e f f i ci e n t t h a n e q u i v a l e n t s i n g l e - m o d a l i t y

    f o r m a t s . S u c h d u a l - m o d a l i t y p r e s e n t a t i o n t e c h n i q u e s a r e

    som et im es used in t rad i t iona l ins t ruc t iona l p rac t i ce . F or

    exam ple , s tuden t s m ay pre fe r l i s t en ing to o ra l exp lana t ions

    Table 2

    M e a n R a t i n g s , S c o r e s , a n d L e v e l s o f l n s tr u c t i o n a l

    E f f i c ie n c y b y E x p e r i m e n t a l C o n d i t io n , E x p e r i m e n t 2

    Experimental condition

    Diagram

    Variable with audio text Diagram only

    Rating scalea

    M 3.5 2.7

    SD 1.5 1.2

    Question scoresb

    M 5.3 6.9

    SD 2.6 2.4

    Instructional efficiency~

    M -0 . 4 3 0 .43

    SD

    1.10 1.00

    aRatings were m ade on 7-point scales (1 =

    extremely easy, 7 = ex-

    tremely difficult , bM axim um question score = 10. CActual

    range for instructional efficiency was from - 2. 6 2 to 2.16 (possible

    range is -oo, +~ ).

    o f n e w , c o m p l e x , d i a g r a m - b a s e d m a t e r i a l s ( e . g . , w h e n

    s t u d y i n g g e o m e t r y o r e n g i n e e r i n g ) r a t h e r t h a n r e a d i n g s u c h

    e x p l a n a t i o n s i n t e x t b o o k s . I n p r a c t i c e , h o w e v e r , m a n y

    s tanda rd m ul t im e dia ins t ruc t iona l p resen ta t ions use aud i to ry

    explana t ions s im ul taneous ly wi th the s am e v i sua l ly p re -

    sen ted t ex t . In o the r words , dua l -m ode presen ta t ions a re

    u s e d u n d e r r e d u n d a n c y r a t h e r t h a n s p l i t - a tt e n t io n c o n d i t i o n s .

    F r o m t h e p o i n t o f v i e w o f c o g n i t i v e l o a d t h e o r y , s u c h

    d u p l i c a t io n o f t h