In the Classroom

574 Journal of Chemical Education • Vol. 75 No. 5 May 1998 • JChemEd.chem.wisc.edu

Using Interactive Anonymous Quizzesin Large General Chemistry Lecture Courses

Thomas HolmeDepartment of Chemistry, University of Wisconsin–Milwaukee, P.O. Box 413, Milwaukee, WI 53201

The scrutiny of learning in large lecture settings over thepast decade has resulted in some important and interestingsuggestions. Introductory chemistry courses are often taughtin large lecture sections, and several recent papers describeways to enhance the learning in this situation (1–6 ). Onecommon concern voiced by students about this manner ofpresenting a course is the lack (or perceived lack) of two-waycommunication with their instructor. From this perspective,a class with 60 or more students constitutes a large lecture,because students in classes of that size do not generally com-municate with the instructor on a regular basis. Methods thatimprove communication typically result in enhanced out-comes for students—more learning, better grades, and higheroverall satisfaction with the course.

This paper will describe one way to enhance student–instructor communication through regular student feedbackabout key concepts. Examinations also serve this purpose, butthey are seldom frequent enough to provide more than a snap-shot of student learning. The feedback method described herealso provides an opportunity for interactive student discus-sions in the large-lecture setting. We will refer to this tech-nique hereafter as Interactive Anonymous Quizzes (IAQs).The technique is not particularly time consuming and it pro-vides a means for introducing interactive exercises into acourse without substantial loss of coverage.

The Format of Interactive Anonymous Quizzes

The basic idea of anonymous quizzes is not complicated.The format has been used for some time by Mazur in physics(7 ) and no doubt by others who have heard of his efforts.What turns this concept into a feedback device is the col-lecting of responses from students and the brief analysis ofthose responses before the subsequent lecture.

In addition to asking a multiple-choice question, wesurvey student attendance and review work in our IAQs. Atthe top of each IAQ we ask students if they (i) attended thelecture in which the topic for the subsequent question waspresented and (ii) reviewed the material since that lecture.We have used this technique for several years in two differ-ent university settings, with a variety of questions. For thepurpose of discussion here we will focus on four questionsthat are representative of our usage in several ways, includ-ing that they (i) show the difficulty level we address with thistechnique, (ii) provide examples of questions that do not ex-pressly evaluate some numerical algorithm, and (iii) are takenfrom a variety of content areas in the typical second semester ofgeneral chemistry. Records were not retained of our early usageof the method at a different university, so we cannot performany comparison between students in different locations orfrom different environments. The examples we will discussare taken from a course with an enrollment of 83 students.It was the second semester of a two-semester general chemistry

sequence. Of these students, 17% were pre-engineering stu-dents, 37% biology majors, 35% pre-health-sciences majors,and 11% “other” science majors (including, in this case, nostudents intending to be chemistry majors). We have usedthis technique in sections of more than 250 students. Thefour IAQs we will discuss are as follows (the “title” by whichthe IAQ will be tabulated appears in parentheses):

I. (Hydrolysis) What is the pH of a sodium cyanide(NaCN) solution? (a) less than 7 (b) 7 (c) more than 7

II. (Oxidation #) What is the oxidation number of sulfurin the compound SO2Cl2? Electronegativities, S = 2.5,O = 3.5, Cl = 3.0 (a) 2 (b) {2 (c) 6

III. (Electrochemistry) You have a solution containing Ni2+

that you wish to plate out as solid Ni. Which of thefollowing solids could you place in the solution toaccomplish your objective (the activity series is shownon the overhead)? (a) Cr (s) (b) Sn(s) (c) Cu(s)

IV. (Nuclear) One unstable isotope of fluorine, 17F, has ahalf-life of roughly one minute. What type of decaywould you predict 17F undergoes? (a) positron emis-sion (b) electron emission (c) alpha emission

Students are required not only to answer the questionwith (a), (b), or (c) but also to provide an indication of theconfidence they have in that answer (1 = very confident, 2 =somewhat confident, and 3 = just guessing). This confidencemeasure is patterned after Mazur (7 ). Having provided twominutes for individual students to answer the question, wetell the students to convince those who are seated near themthat they are correct. After about 3 minutes for discussionstudents again provide the answer and confidence level. TheIAQs are administered at the start of the class but collected atthe end. Brief discussion of the question provides the correctanswer.

The use of multiple-choice questions imposes some limi-tation on the process, but we have tried free-format questionswith limited success and the use of multiple-choice serves thepurpose of rapid determination of student understanding.While the examples chosen for discussion in this paper havefewer than 100 responses, in early uses sections of more than250 students proved too large to allow rapid reading of free-format responses. Our intent in the choice of multiple-choicequestions is not to include detailed arithmetic manipulation.The questions focus on concepts in chemistry, although withwidespread interest in conceptual questions (8) our usage ofthat term may not match that of others.

The Value of the Technique

While this technique is indeed simple and by no meanshighly novel, there are benefits derived from viewing themethod as a form of feedback. It takes roughly two minutesto scan a group of 100 or so quizzes to determine if the classis on track on the quiz subject. With more time it is possible

In the Classroom

JChemEd.chem.wisc.edu • Vol. 75 No. 5 May 1998 • Journal of Chemical Education 575

to derive even more information, as reported below. In ourexperience, if the performance on anonymous quizzes hasbeen poor the subject needs to be treated again, or studentswill not pick up the material on their own. For example, inthe four questions noted earlier only the question on nuclearchemistry showed enough weakness in the class to warrantadditional coverage of this material. Success rates on the ex-amination for similar questions were markedly better thanthose on the quiz. This improvement is not solely attributableto the expanded coverage, but there is little argument thatrepeated exposure to new ideas is useful to students. By usingregular feedback, it is possible to determine when suchrepeated exposure is needed and therefore likely to be worththe time investment.

The types of additional information that can be gainedfrom this type of quiz pertain to the efficacy of an interactiveapproach to teaching. The format of the assessment devicebuilds in time for students to discuss the material with eachother and “re-answer” the question. For the four questionsnoted earlier, therefore, we may observe how the student–student interactions affect students’ demonstrated contentknowledge and confidence in the knowledge. This analysisis summarized in Table 1 for each of the four questions.

While Table 1 has a considerable amount of informa-tion there are a few points worth noting in general.

1. In each case, the percentage of students that have thecorrect answer increases after students have discussedthe issue with each other.

2. There is an increase in confidence among studentsabout their understanding as a result of the interac-tive exercise.

3. Relatively few students report their confidence levelas “just guessing”. There may be dynamics associatedwith the group work that limit the extent to whichstudents will self report that they are guessing.

This overall summary of the IAQs does not encompassall of the information about the utility of interactive work ina large lecture setting. Table 2 shows how students’ answerschanged as a result of the discussion time they undertook.There are four possible outcomes of these discussions. Stu-dents will (i) have the right answer before and after (ii) havethe wrong answer before and after (iii) switch from right towrong or (iv) switch from wrong to right. In any case theymay also have increased confidence, the same confidence, orlesser confidence. Investigating Table 2 shows several addi-tional key features about the interactive work.

The first and most important observation is that thereare very few instances where students, by virtue of their dis-cussion, gravitate from the correct answer to an incorrect one.In only four instances did this decidedly negative outcomeoccur. Table 2 summarizes more than 200 responses, so thissmall number should cause little concern that interactive tech-niques in large lecture may cause unacceptable harm to studentunderstanding. Again, our early usage of this technique didnot include permanent record keeping, but our impressionwas positive from the outset. We have never noticed an IAQfor which more than one student changed from the correctto an incorrect answer. The second important observationdemonstrated by this table is that students regularly haveincreased confidence in their correct answer, even if they werecorrect at the outset. A third observation is that in most cases,

ziuQnistfihSleveLecnedifnoC'stnedutS.2elbaTnoissucsiDpuorGllamSretfasrewsnA

noitseuQziuQtfihSleveLecnedifnoC

erofeb(rewsnA / )noissucsidretfa a

/.rroc.rroc

/.rrocni.rroc

/.rrocni.rrocni

/.rroc.rrocni

sisylordyH.Itnedifnocerom 62 8 3 0ecnedifnocemas 91 4 2 0

tnedifnocssel 2 1 0 0rebmunnoitadixO.II

tnedifnocerom 41 9 4 0ecnedifnocemas 51 3 2 1

tnedifnocssel 1 0 0 0yrtsimehcortcelE.IIItnedifnocerom 31 3 0 1ecnedifnocemas 43 1 3 1

tnedifnocssel 0 0 0 0yrtsimehCraelcuN.VI

tnedifnocerom 3 9 3 1ecnedifnocemas 8 4 21 0

tnedifnocssel 0 2 2 0aData are numbers of students showing indicated shift in confidence

level. Answers: corr. = correct, incorr. = incorrect.

ruoFotsesnopseRtnedutSfoyrammuS.1elbaTsezziuQsuomynonAevitatneserpeR

noitseuQziuQleveLecnedifnoC b

esnopseR a

tcerroC tcerrocnIerofeB retfA erofeB retfA

sisylordyH.Itnedifnocyrev 8 63 2 1

tnedifnoctahwemos 43 22 11 5gnisseugtsuj 6 2 6 0

…latoT 54 06 91 6rebmunnoitadixO.II

tnedifnocyrev 71 43 4 2tnedifnoctahwemos 51 21 6 5

gnisseugtsuj 6 3 8 0…latoT 83 94 81 7

yrtsimehcortcelE.IIItnedifnocyrev 52 63 0 0

tnedifnoctahwemos 02 61 4 4gnisseugtsuj 7 2 3 1

…latoT 25 45 7 5yrtsimehcraelcuN.VI

tnedifnocyrev 1 6 0 1tnedifnoctahwemos 6 31 11 7

gnisseugtsuj 9 8 12 01…latoT 61 72 23 81

aData represent number of students. “Before” and “after” refer toresponses before and after small group discussions.

b”Very confident” is rated 1; “somewhat confident”, 2; and “justguessing”, 3.

In the Classroom

576 Journal of Chemical Education • Vol. 75 No. 5 May 1998 • JChemEd.chem.wisc.edu

a sizable percentage of students (20%, 24%, 7%, and 34%of students in the examples discussed) switch from the wronganswer to the correct one. The exception was the questionon electrochemistry, which most students answered correctly atthe outset. If one considers as positive outcomes a switch toa correct answer or an increase in confidence in the rightanswer, 95 responses were positive. Negative outcomes wouldinclude a switch from right to wrong or a decrease in confi-dence in a correct answer, and they number only 7. Therecould be additional stratification of the remaining answers,but they are all more neutral than those defined here. Thepreponderance of positive outcomes due to student discussionsis evident regardless of the precise definition of positive andnegative.

Because we are arguing that these numbers help to justifythe use of IAQs, it is important that we note how they comparewith previous, related work. This technique and its analysisare not formal chemistry education research, but rather anattempt to assess how a specific classroom technique hasworked. Nonetheless, the results do relate to more formal re-search. For example, our use of IAQs describes a short, well-focused time for student–student interactions. The successof the technique may be related to the finding that studentsin science classes tend to be very task oriented in their dis-cussions with each other (9, 10). This method, therefore, playsto the observed inclination of science students. Moreover,Lemke (11) has suggested that dialogue between students isfundamentally valuable to them and tends to improve learning.In light of these studies, the success of students after discus-sion is not particularly surprising. The value of providing thatdiscussion without tremendous expenditure of time is theimportant point here.

Another aspect of this type of student–student interactionis that it provides “wait time” for students. In many largelecture settings, questions asked by the instructor are per-ceived by students to be rhetorical. Many formal studies havefound that simply waiting for answers gives students time tointernalize material (12). When students first work on ananswer themselves and then discuss with their neighbors, theconcept of wait time is built into the process. There are otherways to insure wait time, including some we have discussedin previous papers (5, 6 ), but this way requires relativelymodest modification of a standard lecture teaching style.

Summary

The utility of IAQs lies in two distinct regimes. It rep-resents a method for simultaneously incorporating improved

communication and feedback in a large lecture with a rela-tively nonintrusive inclusion of interactive learning. Both feed-back and interactive learning have been found useful in avariety of settings (13, 14), so a means for introducing themnaturally in large lecture settings is a useful development.Relatively little investment is required in terms of resources(photocopy expenses are all that are encountered) and time(useful feedback can be garnered quickly by the lecturer uponcollection of the quizzes).

The second noteworthy aspect of this technique is thatit allows for an investigation of the utility of interactive meth-ods in large lecture settings. Analysis of four representativeanonymous quizzes shows that time spent allowing studentsto discuss the quizzes with each other leads to a general quan-titative improvement in the content knowledge of the class.There is also an evident increase in confidence about under-standing for a significant number of students. Finally, thereis very little negative impact in terms of students convincingeach other of incorrect information. The only arguably nega-tive impact therefore is the time spent carrying out the exercise.With the types of positive outcomes established through theanalysis here, one might reasonably argue that the time is wellspent and worth consideration in the teaching of large lecturecourses in chemistry.

Literature Cited

1. Strauss, M.; Levine, S. H. J. Chem. Educ. 1985, 62, 317.2. Strauss, M.; Fulwiler, T. J. Coll Sci. Teach. 1990, 19, 158.3. Geske, J. J. Coll. Teach. 1992, 40, 151.4. Holmgren, P. J. Coll. Sci. Teach. 1992, 21, 193.5. Holme, T. A. J. Chem. Educ. 1992, 69, 974.6. Holme, T. A. J. Chem. Educ. 1993, 70, 933.7. Mazur, E. Peer Instruction: A Users Manual; Prentice-Hall: Saddle

River, NJ, 1997.8. ACS Division of Chemical Education Examination General

Chemistry (Conceptual), Form 1996.9. Richmond, G.; Striley, J. J. Res. Sci. Teach. 1996, 33, 839.

10. Kempa, R. F.; Ayob, A. Int. J. Sci. Educ. 1991, 13, 341.11. Lemke, J. Talking Science: Language, Learning and Values; Ablex:

Norwood, NJ, 1990.12. Rowe, M. B. J. Res. Sci. Teach. 1974, 11, 81. Tobin, K. G. J. Res.

Sci. Teach. 1980, 17, 469. Tobin, K.G. Rev. Educ. Res. 1987, 57,69. Duell, O. K. Am. Educ. Res. J. 1994, 31, 397. Jegede, O. J.;Olajide, J. O. Sci. Educ. 1995, 79, 233.

13. Angelo, T. A.; Cross, K. P. Classroom Assessment Techniques, 2nded.; Jossey-Bass: San Francisco, 1988.

14. Johnson, D. W.; Johnson R. T.; Smith, K. A. Active Learning:Cooperation in the College Classroom; Interaction Book Co.: Edina,MN, 1991.