Effect of Instructional Media -ARCS

Int'l J of Instructional Media Vol. 32(4), 2005

THE EFFECT OF INSTRUCTIONALMEDIA ON LEARNER MOTIVATION

DAVID L. RODGERSMarshall University Graduate CollegeBEVERLY J. WITHROW-THORTONCharleston Area Medical Center Institute

ABSTRACTWith a variety of instructional media available to educators, selecting the appropriate

instructional format is a critical decision to stimulate learner motivation. This study evalu-ated learners' perceptions of motivation in a comparison of three instructional formats:lecture, video, and interactive computer-based instruction. Ninety-six participants wereassigned to one of three groups. Each group received training on the same subject by oneof the three instructional methods. A modified version of the Instructional Materials Moti-vation Survey (IMMS) was used to determine learner attitude and motivation. Computer-based instruction generated a higher degree of motivation than either video or lecture (p =.000). There was no significant difference between video and lecture in the degree ofmotivation each of these produced (p = .574). This study presented new information intwo areas. First, it targeted adult learners as opposed to the more commonly studiedK - 12 or college student. Second, it represented an expanded scope for the survey instru-ment.

INTRODUCTIONMotivation is an important element required for learning. Educators have a

variety of instructional media and teaching formats available to present informa-tion. Selecting a medium that motivates learners is an important consideration.This study compares the effect of different media on learners' motivation tolearn. Through the use of a survey instrument - the Instructional Materials Moti-vation Survey (IMMS) - overall motivation to learn was evaluated. This studyprovides new information in two areas. First, most applications of the IMMS andother motivational scoring scales on learner motivation have been conducted inacademic arenas and included K - 12 or college students. This study shifted thefocus to adult learners in the workplace. This area has been much less explored.Second, the IMMS was modified to increase its scope to evaluate instructionalmedia and format, not just instructional materials.

The Instructional Materials Motivation Survey was designed to evaluate how

333

334 / David L. Rodgers & Beverly J. Withrow-Thorton

instructional materials affect motivation to learn. Designed with the premise thatcertain conditions must be present to stimulate the desire to learn, the IMMS [1]was developed around Keller's [2, 3] ARCS model of motivational design. In themodel, Keller defined four basic human characteristics that need to be influencedfor people to learn: attention, relevance, confidence, and satisfaction. The IMMS isdesigned with four sub-scales, one each to evaluate the effect of the four character-istics. The four subscales combine to create a total motivational score.

Because the IMMS is used to evaluate motivation in conjunction with instruc-tional materials, it only evaluates state motivation as it relates to that instruc-tional material. State motivation is a temporary condition that affects the level ofconcentration and attention toward an assigned task. The IMMS does not evalu-ate trait motivation that is the underlying tendency to achieve higher degrees oflearning with little or no influence by the situation.

In describing the ARCS Model, Keller [3] offers these insights into the fourareas.

1. Attention - The learner's attention must be acquired and maintained. Theinstructional material must have attention-getting strategies to direct thelearner to the material. Once that attention is achieved there must be atten-tion-sustaining strategies employed to keep the learner's attention on task.Coupled with this is the ability of the material to stimulate the curiosity ofthe learner to learn more.

2. Relevance - Unless the learner perceives a need for the information, moti-vation to learn will be impaired. Instructional materials must be designed toshow that the information is important to the learner and relates to theirwork or other personal goals or motives. Through the use of relevance-pro-ducing strategies, such as goal orientation or motive matching, learners mustbe able to see how the subject matter will relate to their own experiences.

3. Confidence- A serious distracter to motivation is the inability of thelearner to maintain confidence in his or her ability to learn the material.Frustrations with the complexity of the subject may decrease motivation.Conversely, an over abundance of confidence may also produce a detrimen-tal effect as the learner may skip over details after assuming to alreadyknow the material.

4. Satisfaction - The learner must be satisfied with the learning experience inorder to maintain motivation. This satisfaction can manifest itself in bothextrinsic and intrinsic ways. Extrinsic rewards such as achieving a highgrade, course completion certificates, or other material awards help providemotivation. However, intrinsic rewards are equally important. Receivingpositive feedback through interaction, improving self-esteem by learningmore about a subject, and the ability to control their own learning areexamples of intrinsic rewards.

The ARCS model gains its support from expectancy-value theory. Wherethere is value attached to the task and the learner believes success is probable,motivation increases as the expectation to succeed and the value of the taskincrease [4, 5]. By improving confidence towards probability of success and

Media Motivation / 335

increasing the value of the task by creating a higher degree of relevance,expectancy-value theory would predict higher degrees of motivation.

The IMMS tests the effectiveness of instructional material on motivation tolearn by evaluating how the material or presentation stimulates the learnersresponse to the four areas of the ARCS Model. The IMMS is a 36-item, self-administered measure that uses a five-point Likert-type scale. The 36 items arebroken into four subscales based on the ARCS Model. Specifically there are 12items for attention, 9 items for relevance, 9 items for confidence, and 6 items forsatisfaction. The total score can be used as an overall score to judge state moti-vation regarding instructional media or materials.

The purpose of this study was to identify how different instructional mediaaffected learner motivation in a workplace training situation. Three formats wereselected for evaluation: linear video that consisted of video taped presentationswith demonstrations of techniques and principals discussed, lecture presenta-tions by experts on the subjects that included use of overhead projector as wellas demonstration of skills, and computer-based instruction that utilized an inter-active multimedia format. These three instructional methods are representativeof methods frequently used in the workplace [6].

Based on earlier research that indicated computer-based instruction promoteda more positive learning experience and could lead to a higher degree of motiva-tion for the user, [7-91 it was hypothesized that computer-based instructionwould provide a more positive learning experience that should be reflected withhigher degrees of motivation to learn.

METHODS

SubjectsThis study was experimental, comparing the motivation of three groups after

being assigned to receive training by one of three methods of instruction. Thedependent variable was motivation, ascertained with the IMMS, while the inde-pendent variable was method of instruction. The IMMS was administered to 96subjects. The group consisted of new employees of a large medical center whowere undergoing safety training. Since the training is an annual requirement ofemployees, new employees were used as the sample group because they had notbeen previously exposed to the material. There were 76 females in the group and20 males. Mean age was 33.

Procedure

Using a quota sample based on gender, age, and education, subjects were ran-domly assigned to one of three groups. Distribution of the quota characteristicswas equal allowing for the creation of three homogeneous groups. The groupswere then randomly assigned to receive the safety education in one of the threeformats. Each format provided the same information with the only difference


being the medium.The video consisted of a series of professionally produced programs that uti-

lized combinations of on-camera narrators, live action, graphics, sound effects,and demonstrations. Lectures were conducted by hospital employees consideredexperts on the topics with each speaker having presented the program manytimes in the past with good student evaluations. Lecturers utilized supportingmaterials such as overhead projector and slides as well as demonstrating someprocedures with props such as a fire extinguisher. Lecture is the usual formatused by the hospital to conduct new employee safety training. Computer-basedinstruction was delivered in a computer lab equipped with a computer worksta-tion for each employee in the group. The program was produced by the hospitaland featured a multi-media format with video, sound, and interactions such assimulations, game playing, and quizzes. The subjects were scheduled to receivethe training as part of a regular new-employee orientation program. After receiv-ing the training, subjects were asked to complete the IMMS.

MeasureThe IMMS wording in its original form was not adequate for participants to

relate to all types of materials or media. In its unmodified form, the IMMS had abias towards written media, using expressions such as "the writing" and "wordson the page." Since non-written media was used in this study, some minor modi-fications of the questions were made to allow them to be applied to all of theinstructional methods in the study.

Alpha coefficients for the four subscales collected in this study indicatedmoderate to high reliability. The total scale had an alpha coefficient of .89,showing a marked relationship. Keller [10] reported a slightly higher reliabilityfor the IMMS in its original form. Reported coefficients for other application ofthe scale ranged between .81 and .96 [11]. The individual subscale and totalscale coefficients for this study is compared against Kellers [10] reported coeffi-cients as shown in Table 1. Using the same sample set, a split-half reliability testwas conducted. In a split across the four subscales, a reliability of .80 for thetotal scale was found.

TABLE 1. ALPHA COEFFICIENT SCORES FOR IMMS

Subscale N of Items Alpha Level Keller [10]Attention 12 .76 .89Confidence 9 .81 .90Relevance 9 .66 .81Satisfaction 6 .85 .92Total Scale 36 .89 .96


RESULTSA one-way analysis of variance was conducted with motivation, as reported

on the IMMS, as the dependent variable and instructional media as the indepen-dent variable that resulted in a significant finding [F (2, 93) = 17.515, p = .000]that indicated that at least one of the media had a significant effect on motiva-tion. In a post hoc test, computer-based instruction was found to have significantdifferences when compared with video and lecture instruction on the total scale.The LSD post hoc test revealed a mean difference between computer-basedinstruction (CBI) and video of 27.3852, SE = 5.027, p = .000. It also showed amean difference between CBI and lecture of 24.5484, SE = 5.142, p =.000.These findings support the hypothesis.

In each of the individual subscales - attention, relevance, confidence, and sat-isfaction - significant findings favored CBI as the favored medium (see Table 2).For attention, relevance, and satisfaction, p = .000. For confidence, p = .020. TheLSD post hoc test indicated no significant difference between video and lectureinstruction in the degree of motivation each produced with a mean difference of2.8368, SE = 5.142, p = .574. Findings for the total scale and each subscale forthe LSD post hoc test are shown in Table 2.

TABLE 2 POST HOC TEST (LSD) ON IMMS

Mean Differ StandardSubscale Media (A) Media (B) (A-B) Error SignificanceAttention Video CBI -12.1205 2.176 .000*

Lecture -1.088 2.176 .618CBI Video 12.1205 2.176 .000*

Lecture 11.0323 2.226 .000*Lecture Video 1.0882 2.176 .618

CBI -11.0323 2.226 .000*Relevance Video CBI -5.3083 1.339 .000*

Lecture .7562 1.339 .575CBI Video 5.3083 1.339 .000*

Lecture 6.0645 1.369 .000*Lecture Video -7.562 1.369 .574

CBI -6.0645 1.369 .000*Confidence Video CBI -3.6869 1.551 .020*

Lecture -1.3321 1.551 .393CBI Video 3.6869 1.551 .020*

Lecture 2.3548 1.587 .141Lecture Video 1.3321 1.551 .393

CBI -2.3548 1.587 .141


Satisfaction Video CBI -6.2694 1.240 .000*Lecture -1.1082 1.240 .374

CBI Video 6.2694 1.240 .000*Lecture 5.1613 1.268 .000*

Lecture Video 1.1082 1.240 .374CBI -5.1613 1.268 .000*

Total Scale Video CBI -27.3852 5.027 .000*Lecture -2.8368 5.027 .574

CBI Video 27.3852 5.027 .000*Lecture 24.5484 5.142 .000*

Lecture Video 2.8368 5.027 .574CBI -24.5484 5.142 .000*

p < .025

DISCUSSIONThe results of the overall score were highly significant in rating computer-

based instruction as a more motivating medium than either video or lecture. Atthe subscale level, findings that attention, confidence, and satisfaction weregreater for CB[ were consistent with much of the literature judging these atti-tudes against other media. An interesting result was the strength of the findingon the relevance subscale. Despite all groups receiving the same information andhaving the same potential opportunities to apply the information in their work-place, the CBI group felt the knowledge and skills were more relevant thaneither the video or lecture groups. The CBI program was able to involve thelearners with the content and used techniques such as simulated scenarios thatrequired immediate learner input. These interactions and their instant feedbackprovided opportunities for the learners to become more cognitively engaged and,as Caincross and Mannion explained, "causes them to think about the materialthat is presented, what it means, its relevance, how it can be applied and in whatcontexts" [12, p. 161). In video and lecture groups, the instructional format pro-vided information but did not provide practice opportunities. Through the use ofsimulation and game playing, CBI provided opportunities for learners to demon-strate new knowledge and be given immediate feedback. This explanation forimproved motivation has been forwarded in other studies [ 131.

When evaluating these results against the theoretical basis of the ARCSmodel, elements of expectancy-value theory are evident. In a recent review ofmotivation literature, Eccles and Wigfield [ 14] summarized modem expectancy-value theory development. As they noted, important influences on the expectan-cies and values included learners' feelings about their level of competence inperforming the task and their view of the difficulty of the task. Because of theinteractive nature of the CBI program, immediate feedback permitted learners tobuild individual outcome histories more readily than learners in the other two


groups. This ongoing interaction as opposed to the more limited interaction fromlinear video and lecture created a history of success that generated a strongerdegree of confidence in the material. Yildirim, Ozden, and Aksu [15] suggestedthat learner-controlled instructional media causes increased motivation that leadsto feelings of increased confidence that then lead to even higher degrees of moti-vation. Other authors have cited CBI's learner-directed capabilities as an advan-tage over other methods of instruction [ 161.

There are other theoretical explanations for the success of CBI in generating ahigher degree of motivation. From the constructivist's viewpoint, CBI leads to amore student-centered, self-directed approach that allows each learner to man-age his or her own learning process. By being more engaged in the process, alearner is able to construct a reality that has a higher degree of personal rele-vance [17]. This engaging process creates a more interesting learning environ-ment that stimulates intrinsic motivation.

As for limitations, while the quality of each instructional method was thoughtto be relatively equal, some variations may have existed that could have haddetrimental effects on motivation. This was managed by using professionallyproduced programs for the video segments and CBI, and experienced expert pre-senters for the lecture sessions. This study looked at only three instructionalmedia. There are a number of other instructional methods in use in business andindustry including role playing, case study, and conference [6]. Additional com-parisons with other methods would be valuable.

Although the IMMS has had frequent appearances in the literature [5], othertested measures of evaluating motivation exist such as the Student MotivationScale [ 18]. However, Keller argues that while other measures are similar in theirintent, the IMMS is more specific with regard to the materials, not just the sub-ject matter [101.

The original IMMS did require some modification to make it applicable tothese presentation formats. The application of the IMMS in this study was anexpansion of the scope of use for this measurement. It was felt the changes wereslight and the effect on the measurement's validity was not adverse. This wassupported by the relatively strong alpha coefficients reported in Table 1. Apply-ing a modified form of the IMMS to this broader definition of instructionalmaterials was effective and indicates the expanded scope of use to includeinstructional formats, and not just instructional materials, to be practical.

The IMMS has been modified by other authors with success. For example,Small and Ferreira [19] revised the scale to create a measurement aimed at gaug-ing motivation in reference to specific information resources. Dempsey andJohnson [5] developed the ARCS Gaming Scale that focused on motivationrelated to computer games. Eom and Reiser [161 modified the IMMS in theirstudy on the influence of student's self-regulated learning strategies on outcomesincluding motivation and the level of learner- versus program-controlled CBI.Others have used individual subscales of the IMMS with success [201. There areother measurement instruments based on the ARCS model [21, 22], but theIMMS remains the most well tested and reliable ARCS instrument to evaluate


the effect of instructional material on motivation.The ARCS model was developed primarily using K- 12 students [231. As

Bohlin and Milheim [23] suggested, there may be different motivational factorsto consider when using the model to stimulate motivation in adults. With agreater life experience and the potential to have learned attitudes that may reflectnegatively on one instructional medium over another, there are considerations tobe taken into account with adult learners. However, the instructional media uti-lized in this study were specifically designed for adults. Still, Bohlin and Mil-heim made a critical point in the importance of looking at adult implications ofthe ARCS model, especially considering the accelerated rate of change and newknowledge many adults in the work force face.

While this study shows a higher degree of learner motivation for computer-based instruction over video or lecture, this must be linked to learning and recall.Without a successful link between these, curriculum and program designers mayhave a difficult time justifying a change to computer-based instruction basedsolely on motivation or satisfaction. While several studies have shown higherdegrees of learning [7, 8, 15, 24], other studies have proved inconclusive [9, 25,26]. Continued research into this area is warranted.Direct Reprint Requests to:David L. RodgersCharleston Area Medical CenterHealth Education and Research Institute501 Morris StreetLife Support Training Center - 5 EastCharleston, WV 25301Email: dave.rodgers @ camc.org


REFERENCES1. Keller, J. M., Instructional Materials Motivation Survey. Unpublished manuscript,

Florida State University, 1993.

2. Keller, J. M. The systematic process of motivational design. Performance and Instruc-tion, 26(9), p. 1-8, 1987.

3. Keller, J. M. Strategies for stimulating the motivation to learn. Performance andInstruction, 26(8), p. 1-7, 1987.

4. Small, R. Motivation in instructional design. Teacher Librarian, 27(5), p. 29-31, 2000.5. Dempsey, J. V. and Johnson, R. B. The development of an ARCS gaming scale. Jour-

nal of Instructional Psychology, 25, p. 215-221, 1998.6. Shoenfelt, E. L., The relative effectiveness of training methods for attaining training

objectives: Current opinion of training practitioners. (ERIC Document ReproductionServices No. ED 329 879), 1991.

7. Yang, Y. C. The effects of media on motivation and content recall: Comparison of com-puter- and print-based instruction. Journal of Educational Technology Systems, 20,p. 95-105, 1992.

8. Cronin, M. W., Grice, G. L., and Olsen, R. K., Jr. The effects of interactive video instruc-tion in coping with speech fright. Communication Education, 43, p. 42-53, 1994.

9. Ivers, K. S. and Barron, A. E., Teaching telecommunications: A comparison betweenvideo and computer-based instruction. (ERIC Document Reproduction Services No.ED 378 963), 1994.

10. Keller, J. M., Manual for the Instructional Materials Motivation Survey (IMMS).Unpublished manuscript, Florida State University, 1990.

II. Song, S. H. and Keller, J. M. The ARCS model for developing motivationally-adaptivecomputer-assisted instruction, in Proceedings of Selected Paper Presentations at theConvention of the Association for Educational Communications and Technology.Houston, TX: (ERIC Document Reproduction Services No ED436181),1999.

12. Caincross, S. and Mannion, M. Interactive multimedia and learning: Realizing the bene-fit. Innovations in Education and Teaching International, 38(2), p. 156-164, 2001.

13. Klein, J. D., Freitag, E., and Wolf, B. Providing practice using instructional gaming:A motivating alternative, in Proceedings of Selected Paper Presentations at the Con-vention of the Association for Educational Communications and Technology. Ana-heim, CA: (ERIC Document Reproduction Services No ED323926), 1990.

14. Eccles, J. S. and Wigfield, A. Motivational beliefs, values, and goals. Annuals Reviewof Psychology, 53, p. 109-132,2002.

15. Yildirim, Z., Ozden, M. Y., and Aksu, M. Comparison of hypermedia learning andtraditional instruction on knowledge acquisition and retention. Journal of Educa-tional Research, 94, p. 207-214, 2001.

16. Eom, W. and Reiser, R. The effects of self-regulation and instructional control onperformance and motivation in computer-based instruction. Journal of InstructionalMedia, 27(3), p. 247-260, 2000.


17. Bailey, D. H. Constructivism and multimedia: Theory and application; innovation andtransformation. International Journal of Instructional Media, 23(2), p. 161-165, 1996.

18. Beatty, M. J., Student motivation scale, in Communication Research Measures, R.B.Rubin, P. Palmgreen, and H.E. Sypher, (Eds.). Guilford: New York. p. 343-346, 1994.

19. Small, R. V. and Ferreira, S. M. Information location and use, motivation, and learn-ing patterns when using print or multimedia information resources. Journal of Edu-cational Multimedia and Hypermedia, 3, p. 251 -271, 1994.

20. Price, C. B. Affective and cognitive influences of textual display in printed instruction.in Proceedings of Selected Paper Presentations at the Convention of the Associationfor Educational Communications and Technology. Anaheim, CA: (ERIC DocumentReproduction Services No ED323941),1990.

21. Small, R., Motivation in instructional design (Report No. EDO-IR-97-06). EricDigest: Syracuse, NY, 1997.

22. Bohlin, R. M., Milheim, W. D., and Viechnicki, K. J. Factor analysis of the instruc-tional motivation needs of adult reamers. in Proceedings of Selected Research andDevelopment Presentations at the 1994 National Convention of the Association forEducational Communications and Technology Sponsored by the Research and The-ory Division. New Orleans: LA: (ERIC Document Reproduction Services No ED362152), 1993.

23. Bohlin, R. M. and Milheim, W. D. Applications of an adult motivational instructionaldesign model, in Proceedings of Selected Research and Development Presentationsat the 1994 National Convention of the Association for Educational Communica-tions and Technology Sponsored by the Research and Theory Division. Nashville,TN: (ERIC Document Reproduction Services No ED373704), 1994.

24. Chang, C.Y. Does computer-assisted instruction + problem solving improved scienceoutcomes? A pioneer study. Journal of Educational Research, 95(3), p. 143-150, 2002.

25. Crain, L. A. Effects of instructional media on immediate and long-term recall. Inter-personal computing and technology, 2(2), p. 19-27, 1994.

26. Maul, G. P. and Spotts, D. S. A comparison of computer-based instruction and class-room instruction. Industrial Engineering, 25(2), p. 25-27, 1993.

AUTHOR NOTEDavid L. Rodgers is a candidate in Curriculum and Instruction at Marshall

University Graduate College (South Charleston, W. Va.,USA).Beverly J. Withrow-Thorton is a Health Communications Specialist with the

Charleston Area Medical Center Health Education and Research Institute(Charleston, W.Va., USA)

COPYRIGHT INFORMATION

TITLE: The Effect of Instructional Media on Learner MotivationSOURCE: International Journal of Instructional Media 32 no4 2005PAGE(S): 333-42

WN: 0500402715006

The magazine publisher is the copyright holder of this article and itis reproduced with permission. Further reproduction of this article inviolation of the copyright is prohibited. To contact the publisher:http://www.adprima.com/ijim.htm

Copyright 1982-2006 The H.W. Wilson Company. All rights reserved.

Documents

Effect of Instructional Media -ARCS