Early Access ExamsCREATING REALISTIC TESTS: GETTING STUDENTS TO THINK LIKE GEOSCIENTISTSJ. Nicholls and L.F. ReidDepartment of Geoscience,University of CalgaryCalgary, AB T2L 1A2Canada

Early Access ExamsCREATING REALISTIC TESTS: GETTING STUDENTS TO THINK LIKE GEOSCIENTISTSJ. Nicholls and L.F. Reid

2

AbstractFinal exam papers were handed to students on the first day of class in senior level igneous petrology courses at the University of Calgary. For students, these early access exams showed that the course content would be used and applied; they further indicated the extent of knowledge expected. For the instructor, the early access exams defined the learning objectives; the course had to focus on the material needed to write the exams.

During the scheduled, two-hour, examination period at the end of the course, students wrote essays about a real suite of volcanic rocks. The examination paper contained rock analyses, rock descriptions similar to those the students had made in laboratory exercises, and literature references. Students could bring figures and factual captions to the exam but no other material. The figures and captions were submitted with the essays.

In the essays, students were asked to characterize the suite, to provide rock names, to describe the chemical variations of the suite in terms of process, and to discuss how the magmas and consequent rocks formed. They were to infer pressures and temperatures at which magmatic processes operated and to include inferences about connections between magma chambers.

The suites changed from year-to-year – Rocks from Lunar Crater, Nevada and several recent eruption cycles from Kilauea Volcano, Hawai’i provided essay topics.

Igneous petrology seldom made the highlight reel in students’ programs, located as we are in Calgary, the centre of Canada’s petroleum industry. The idea was to motivate stu-dents. Amazingly, early access worked: Third and fourth year students wrote essays that were better than acceptable with respect to content – some were excellent. A by-product of early access exams is easier teaching. Students actively participate in learning a topic when it is introduced with an explanation of its role in deciphering volcanic processes.

The writing was not the best because the essays were written under pressure. Conse-quently, the essays took time to read. Numerous, small, guided writing exercises during the course would likely have improved the presentations.

Student evaluations changed from ‘this course sucks’ to ‘I can’t believe I wrote an essay on magmatic history.’ Some students even made a special effort to express their apprecia-tion.

Early Access Exams

3

In the BeginningFor years, when the leaves turned yellow, Nicholls would announce: “I’m sure not looking forward to teaching this term.” Although the announcement became a standing joke, igneous petrology was, in fact, not fun to teach. The students hated the course and the most common question was the dreaded, “Will this be on the final?” In contrast, the same students and the same instructor were together in field courses. Field courses were fun to teach and students liked them. They, the students, found preparing their reports, both oral and written, a challenge and a learning experience. They focused on the geology, the mapping, and the solving of real problems. Early access exams originated as a possible way to get students to bring their attitudes, their desire to learn, and their focus from the field into the classroom and into the petrology course. Giving students early access to the final exam, usually in the first week of classes, and requiring them to write an essay during the exam period at the end of the course, brought this change.

Early access to the exam, combined with a requirement that the essay be written without notes during the exam period, minimized the possibility of plagiarism and inappropriate inclusion of work by others, bad things that can go wrong with take home exams. By-the-way, the question – “Will this be on the final?” – was heard no more.

Final Exam PaperPreambleA copy of this final exam paper will be handed to you when you enter the examination room at the end of the course. For the final exam you will infer a magmatic history for a suite of rocks from a volcanic field in southern Ne-vada. The field is located near the proposed nuclear-waste repository atYucca

Early Access Exams

4

Mountain. Lunar Crater Volcanic Field is one of a near dozen or so basaltic lava fields in southern Nevada and southern California. All of them erupted in the last several thousand or tens of thousand years.

Rock analyses and brief petrographic descriptions of rocks from the field are included in this exam paper. These data for the Lunar Crater Volcanic Field come from Scott and Trask (1971). Thin sections and hand samples of specimens collected from the lava field will be available in the lab during the term. Use these data and materials, along with information that you might be able to discover in the literature, to write an essay on the magmatic history of these rocks.

The essay should provide rock names, describe the chemical characteristics of the suite, and discuss how the rocks formed. What processes operated to produce the chemical variations? What were the pressures and temperatures at which these processes operated? What inferences can you draw about the connections between magma chambers and magma batches?

What You Can Bring to the ExamYou can bring into the place of writing the Figures you need to illustrate your essay and any Tables that you consider absolutely necessary. Each and every Figure and Table that you bring must be turned in with your essay and you must write a multiple sentence paragraph discussing the significance of each and every Figure and Table. Although Figures and Tables must have clearly labeled axes, columns, rows, titles and legends, they cannot have captions and annotations.

Why You Get the Exam Paper EarlyIf you have mastered the content of this course you will have the ability to infer the magmatic history of a suite of rocks. Developing a magmatic his-tory, using the methods, tools, and theory you learn in this course, takes time. Early access to the exam and the required data gives you that time. Early access should also reduce final exam stress. You will have the opportunity to collaborate and discuss aspects of the problem with each other, much like you would study together for a regular exam. The content you create for your

Early Access Exams

5

essay, however, must be your own and must come from your brain at the time of writing, which is the reason the ideas you include in your essay cannot be brought into the exam room as captions and annotations on the Tables and Figures. Consequently, make sure the information on the Tables and Figures is fact, not interpretation. The legitimacy and integrity of the exam can be maintained only if the substance of your essay comes from your knowledge and not from someone else.

Suggested Outline for Your EssayAn effective essay will have a logical and orderly structure. If you don’t have an outline for your essay, you might consider the following:

IntroductionEvery good story needs a beginning. You might, for example, begin with a brief description of the geologic setting, similar to the one in the preamble to this exam. Or you might want to start with a description of what constitutes a magmatic history. Or you might want to contrast your results with a history for another rock suite from another volcanic center such as from Kilauea.

Nature of the rock suiteSuite Identification and rock names: Tholeiite Basalt Suite, Olivine Basalt Suite, or Alkali Olivine Basalt Suite.

Chemical characteristics of the suite.Are the mineral assemblages found in the rocks consistent with the rock chemistry?

Processes that contributed to the variations between rock types in the suite.

Inferences based on mass balance constraints.

Inferences based on thermodynamic modeling.

What were the P-T conditions at which the processes occurred?

Inferences about magma chamber connections and magma batches.

These inferences come from the geologic setting and the constraints

Early Access Exams

6

implied by the nature of processes that you discovered while working through the sections on mass balance constraints and thermodynamic modeling. What do people know about the dates of the eruptions at Lunar Crater Lava Field? How long was it between eruptions? How do the time periods between eruptions in continental lava fields compare to those in Hawai’i, for example?

Describe any inconsistencies between the data, the models, and the inferences.

Summary and DiscussionEvery good story needs an ending. What were the most important things you learned about the rocks of the Lunar Crater Volcanic Field?

Petrographic Notes (Scott and Trask, 1971)The numbers in parentheses are sample numbers from the original paper.

LCrat-1: Older basalt, subalkaline (LC-51); lowermost flow in Lunar Crater, collected from north wall, 600 ft south of cairn. Light gray, with phenocrysts of plagioclase (up to 6 mm) and olivine (avg 1 mm) in groundmass of plagioclase, olivine, and opaque minerals.

LCrat-2: Older basalt, subalkaline (LC-101); 30 ft above contact with Tertiary ignimbrite in canyon 2 miles south of Lunar Crater. Light gray, with abundant plagioclase phenocrysts (up to 3 mm) and phenocrysts of olivine (avg 0.6 mm) in groundmass of plagioclase, olivine, opaque minerals, and glass.

LCrat-3: Older basalt, subalkaline (LC-95); from middle of sequence in Lunar Crater, collected from northwest wall, 1,000 ft southwest of cairn. Medium gray, vuggy, with phenocrysts of olivine (up to 2 mm) and plagioclase (up to 3 mm), rare xenocrysts (?) of clinopyroxene (up to 2 mm) in groundmass of plagioclase, olivine and opaque minerals.

LCrat-4: Older basalt, alkalic (LC-109); from vegetated flow, 1,700 ft north of fresh black basalt (younger basanitoid) and 2.5 miles north of U.S. Highway 6. Dark gray with phenocrysts of plagioclase, olivine, and clinopyroxene (avg 0.6 mm) in groundmass of plagioclase, clinopyroxene, opaque minerals, and glass.

LCrat-5: Basalt of probable intermediate age, subalkaline (LC-99) 2.6 miles

Early Access Exams

7

south of Lunar Crater. Light purplish gray, with phenocrysts of plagioclase, olivine and clinopyroxene (avg 2 mm) in groundmass of plagioclase, clinopyroxene, olivine, and opaque minerals.

LCrat-6: Basanitoid of intermediate age (LC-54); near top of sequence in Lunar Crater, collected in gully at southwest corner. Dark gray, with phenocrysts of olivine and clinopyroxene (avg 1.0 mm) and microphenocrysts of plagioclase in groundmass of plagioclase, clinopyroxene, olivine, opaque minerals, and glass.

LCrat-7: Basanitoid of intermediate age (LC-61); lightly vegetated flows 1.6 miles northeast of Lunar Crater. Black, highly vesicular, with xenocrysts of clinopyroxene (up to 6 mm) in groundmass of plagioclase, clinopyroxene, olivine, opaque minerals, and glass.

LCrat-8: Basanitoid of intermediate age (LC-111); uppermost flow 0.6 miles northeast of Easy Chair Crater. Black, with ultramafic xenoliths (up to 6 cm) and xenocrysts of olivine, clinopyroxene,and plagioclase (up to S mm) in groundmass of plagioclase, clinopyroxene, olivine, opaque minerals, and glass.

LCrat-9: Younger Basanitoid (LC-9); from unvegetated, rough black flows north of U.S. Highway 6. Black, vesicular, with ultramafic xenoliths (up to 16 cm), and xenocrysts of olivine, clinopyroxene, and plagioclase (up to 8 cm) In groundmass of plagioclase, cllnopyroxene, olivine, opaque minerals, and glass.

LCrat-10: Younger Basanitoid (LC-48); from unvegetated, rough black flows, 0.26 miles west of largest cinder cone in cluster just north of U.S. Highway 6. Description same as 9.

Early Access Exams

8

Table 1: Chemical Analyses of Lava Flow (Scott and Trask, 1971).

LCrat LCrat LCrat LCrat LCratOxide 1 2 3 4 5SiO2 48.46 49.59 47.67 47.24 47.76TiO2 2.73 1.88 1.84 2.40 2.57Al2O3 16.19 18.01 14.55 16.52 15.29Fe2O3 6.60 5.47 5.41 3.38 5.98FeO 6.30 4.88 6.45 8.75 6.84MnO 0.18 0.15 0.18 0.19 0.18MgO 5.17 4.28 9.37 6.30 6.53CaO 8.37 9.71 9.06 8.42 9.06Na2O 3.78 2.81 3.05 4.15 3.45K2O 1.25 1.17 1.00 1.52 1.16P2O5 0.49 0.54 0.33 0.51 0.48

LCrat LCrat LCrat LCrat LCratOxide 6 7 8 9 10SiO2 44.79 44.37 44.88 44.66 44.41TiO2 2.65 2.43 2.44 2.33 2.34Al2O3 15.27 15.65 15.60 15.35 15.45Fe2O3 4.21 4.44 4.29 3.39 2.74FeO 8.12 7.94 7.47 7.92 8.66MnO 0.19 0.22 0.21 0.21 0.22MgO 8.75 7.80 7.76 8.96 8.74CaO 10.02 9.62 9.94 10.40 10.49Na2O 3.51 4.32 4.13 3.67 3.99K2O 1.11 1.88 1.83 1.84 1.80P2O5 0.52 0.74 0.73 0.65 0.73

Early Access Exams

9

Selected ReferencesThe references below will give you an entry into the literature. The focus of your essay must be the petrology. Read the references to the nuclear waste repository if you’re interested. Discussion of the implications of volcanism and the volcanic field for the repository, however, should not be part of your essay.

Crowe, B. M., Self, S., Vaniman, D., Amos, R., and Perry.F, 1983, Aspects of potential disruption of high-level radioactive waste repository in southern Nevada: Journal of Geology, v. 91, p. 259-276.

Scott, D. H., and Trask, N. J., 1971, Geology of the Lunar Crater Volcanic Field, Nye County, Nevada: United States Geological Survey Professional Paper, v. 599-I, p. I1-I21.Smith, E. I., Keenan, D. L., and Plank, T., 2002, Episodic Volcanism and Hot Mantle: Implications for Volcanic Hazard Studies at the Proposed Nuclear Waste Repository at Yucca Mountain, Nevada: GSA Today, v. 12, p. 4-10.

Shepard, M. K., Arvidson, R. A., Caffee, M., Finkel, R., and Harris, L., 1995, Cosmogenic exposure ages of basalt lava flows: Lunar Crater volcanic field, Nevada: Geology, v. 23, p. 21-24.

Smith, E. I., Keenan, D. L., and Plank, T., 2002, Episodic Volcanism and Hot Mantle: Implications for Volcanic Hazard Studies at the Proposed Nuclear Waste Repository at Yucca Mountain, Nevada: GSA Today, v. 12, p. 4-10.

Wang, K., Plank, T., Walker, J. D., and Smith, E. I., 2002, A mantle melting profile across the Basin and Range, SW USA: Journal of Geophysical Research, v. 107, p. 10. doi:1029/2001JB000209.

General CommentsCreating an inquiry-based, early access exam paper requires thought and planning. The exam paper for the petrology course is six pages long a fair sized document. The exam paper must be clear, comprehensive, and precise. Students will examine every word before they even begin to use it as a guide to learning. The words written in the exam paper might as well be carved in stone. We also want students to buy into the idea of an early access exam. Consequently, we wrote the exam in the form of a personal communication to the student, addressing them with the second person pronoun, ‘you’.

Early Access Exams

10

PreambleThe preamble is an introduction to the problem. On first reading of the exam paper, most students had a look on their face which can only be described as stunned. By second reading we wanted students to begin to feel comfortable with the course; hence, the brief description of the geologic setting, a concept that third- and fourth-year students could relate to. The oblique reference to the Yucca Mountain Repository was mentioned to show even arcane subjects like igneous petrology can have implications for the real world. The preamble also tells the students about the data they need and what, specifically, they will do at the end of the course.

What instructors put in the preamble determines what they teach and what students learn in the course. Months after the course ended, students could discuss the material they developed for the final exam but anything covered in standard mid-term exams, for example granites and granitic rocks, had faded.

What Students Can Bring to the ExamWhat students are allowed to bring into the place of writing is an important feature of early access exams. For example, what students can bring con-strains the cognitive level of knowledge (Bloom, 1956) that the instructor can expect the students to show.

Geoscience is a visual science and students need figures to justify their infer-ences about a magmatic history. They would not have time to construct such figures during the exam period. Consequently, they were permitted to bring figures and tables to the place of writing but with restrictions that prevented them from using tables and figures as vehicles for clandestine essay passages. In the several years that early access exams were given, no one ever brought in a table.

Why Hand the Exam Paper Out Early?Students in the Geoscience Department at the University of Calgary had not encountered early access exams before. This section was meant to reassure them by explaining what was expected and what they could do for the exam. If we want students to behave ethically we are obligated to tell them what is permissible and what is not. Students soon realized they could take control of what they learned Some followed the instructions and did nothing more; others went to the library and spent considerable time with the thin sections

Early Access Exams

11

and hand specimens. The latter realized they were creating new knowledge; no one before them had inferred a magmatic history for this volcanic field at the level they were capable of doing it.

Additional MaterialAdditional material can help fine tune the exam and such material can be included within the exam paper itself or be made available in the class room or laboratory. Material included in the exam paper will be available to stu-dents while writing the exam whereas material available in the classroom and laboratory would not.

Within the Exam PaperWe provided an optional outline for the essay and a list of selected references.

The detail included in the outline will give the instructor some control on what level of organization they want the students to demonstrate. A detailed outline with included questions, for example, provides more guidance than does a bare-bones outline, one without the included questions. The outline was provided because exams given in earlier years did not contain one. The students who took the exam in the early years didn’t plan their essay before coming to the place of writing. The result was a stream of consciousness. One essay began with a two-page sentence, without comas, before the first period appeared. The outline made the essays much easier to read and evaluate.

The selected references provided those students who wanted to do library research an entry into the literature. In spite of their being third- and fourth-year university students, many had no experience in library research.

Outside the ExamXerox copies of the selected references were placed in the lab with the thin sections and hand samples. Nearly everyone read the source for the data (Scott and Trask, 1971) and most students gave the thin sections at least a cursory look.

Educational ValueThis sort of course and final exam set up has tremendous potential for students to make significant learning gains. Students are shown what a real data set looks like and are challenged to begin to think like geoscientists. They pose their own questions of inquiry about the data and use the scientific

Early Access Exams

12

method. Much has been published on the benefits of student engagement and inquiry-based learning in undergraduate teaching (Kuh, et al., 2005; Bloom and Edward, 1993; McKeachie, et al., 1986). An inquiry-based course allows students to determine what they should be learning, identify resources that will help them achieve learning goals, report on their learning and assess their progress. In the case of this senior undergraduate class, students knew what was expected of them and could use the entire term to prepare for their final exam. They posed their own questions about the data and sought to answer them with the guidance of the course instructor. Some students learned that the professor is not necessarily the all-knowing-oracle of information and an-swers, and that scientific ideas are built on a framework of problem-solving, collaboration, and critical thinking.

ReferencesBloom, B. S. (Ed.), 1956, Taxonomy of Educational Objectives. Volume 1:

Cognitive Domain: New York, NY, McKay.

Bloom, L. Z., and Edward, M. W. (Eds.), 1993, Inquiry: A Cross-Curricular Reader: Englewood Cliffs, NJ, Prentice-Hall, Inc.

Kuh, G. D., Kinzie, L., Schuh, J. H., Whitt, E. J., and Associates, 2005, Student

Success in College: Creating Conditions That Matter: San Francisco, California, Jossey-Bass.

McKechie, W. J., Pintrich, P. R., Lin, Y., and Smith, D., 1986, Teaching and Learning in the Classroom: A Review of Research: Ann Arbor, Michigan, National Center for Research to Improve Postsecondary Teaching and Learning.

Problems, Concers, and AlternativesQuality of WritingIn spite of supplying students with an outline, too many essays were not well organized. Many students wrote in the passive voice, wrote run-on sentences, failed to punctuate, and, in general, committed all the writing sins that in-structors have complained about for years - actually for decades.

Part of the problem arose from the situation, which cannot really be signifi-cantly changed: Students had to write an essay in two hours on a complicated

Early Access Exams

13

and complex subject; a stressful task in spite of the extensive preparation time they had. Such a situation is not conducive to creating a well-written and well-structured document.

With practice, however, we believe it likely that the writing could be im-proved. For example, during the term, each week at the end the lab period, students could take part in a twenty-minute workshop by writing a formal rock description. The observational data for a particular rock could be listed on the blackboard; ten minutes devoted to writing a description; and ten minutes devoted to students reading their descriptions and discussing how to improve them. Instructors of creating writing workshops commonly sched-ule such writing exercises to further the skills and talents of those taking the workshops. Even a modicum of organization and writing skill makes essays easier to grade.

Grading Numerous EssaysEssays take time to grade and there is no magic bullet that can change that fact. However, finding pleasure in reading essays in which students demon-strate their mastery of a subject makes the effort worthwhile. Grading to a rubric would make evaluating an essay more straight forward. The outline provided in the exam paper has many characteristics of a rubric. All it needs is a weighting scheme for the assignment of grades to be a complete one. Ex-amples of grading rubrics can be found on the web by doing a Google search on the key words: grading rubric.

Number of Figures Brought to the ExamStudents brought more figures to the examination than they could adequately describe and discuss during the time of writing. Their paragraphs (see above) were seldom more than two sentences long. Students seem to believe that if they construct a figure, then that figure deserves a place in their essay, regard-less of necessity.

Perhaps an exercise where students read and compare the effectiveness of illustrations in several papers, some effective and some not, would help solve this problem

Early Access Exams

14

Lunar Crater: Namesake of the volcanic field, a phreatomagmatic explo-sion crater.

Lunar Crater Volcanic Field: Cinder cones and lava flows in a view east across the lava field from the edge of Lunar Crater.

Early Access Exams

15

Figures for Essay: Examples of figures students used to illustrate their essays.