Individualised interactive formative assessments to promote independent learning

Available online at www.sciencedirect.comJournal of

J. of Acc. Ed. 26 (2008) 155–165

www.elsevier.com/locate/jaccedu

AccountingEducation

Teaching and educational notes

Individualised interactive formative assessmentsto promote independent learning

Paul Blayney *, Mark Freeman 1

University of Sydney, Faculty of Economics and Business, Sydney, NSW 2006, Australia

Abstract

Rules-based drill-and-practice questions are often used to promote learning in quantitativecourses. Individualised questions are, however, more likely to encourage deep approaches to learn-ing, especially when feedback goes beyond correct/incorrect signals. Individualised questions dis-courage inappropriate shortcuts, such as peers copying solutions. However, individualisationplaces extra work on an instructor, particularly in large classes. The objective of the paper is to pro-mote independent learning by describing an automated approach that allows instructors to deliverindividualised questions to students with minimal effort. The approach also allows students, at theirown discretion, to get individualised help and feedback. Instructors that use rules-based drill-and-practice problems as part of the learning repertoire should find many opportunities to use theapproach described in this article.Crown Copyright � 2008 Published by Elsevier Ltd. All rights reserved.

1. Introduction

Providing students with drill-and-practice rules-based questions, such as those used intypical university accounting courses, requires that instructors provide relevant and timelyfeedback if the questions are to effectively assist student learning. Individualising questionsto reduce student temptation to take inappropriate shortcuts places an even greater bur-den on instructors. While possible with very small classes, today’s typical class size is often

0748-5751/$ - see front matter Crown Copyright � 2008 Published by Elsevier Ltd. All rights reserved.

doi:10.1016/j.jaccedu.2008.01.001

* Corresponding author. Tel.: +61 2 93514355.E-mail addresses: [email protected] (P. Blayney), [email protected] (M. Freeman).

1 Tel.: +61 2 9065030.

mailto:[email protected]

mailto:[email protected]

156 P. Blayney, M. Freeman / J. of Acc. Ed. 26 (2008) 155–165

too large to make such an option practical. Multiple choice questions, while somewhatmore efficient for instructors if pre-existing questions are available, typically do not testhigher order application or provide extensive feedback that students can use to identifytheir own misunderstandings.

This paper describes an innovative approach instructors can easily use to create individ-ualised drill-and-practice questions that provide students with interactive feedback tobasic accounting problems. This approach eliminates the burden on instructors to markand provide feedback on individual questions. The computer spreadsheet is the vehiclefor this approach, which previous research has shown to be useful to students (Blayney& Freeman, 2004).

The teaching approach described in this article allows each student to access the sameassignment file which automatically generates an individual question for each student tocomplete. Students can receive various types of individualised feedback, ranging from spe-cific hints, such as final answers (like end-of-chapter solutions in a textbook), to more gen-eral comments about approach or study methods, to immediate feedback followingsubmission of their attempted answer. Each student receives individualised feedback, tocorrect and incorrect responses, which is relevant to their own specific question.

The described teaching approach does not require that instructors have a high level ofspreadsheet expertise. A 10-step Excel-based template2 guides the instructor from the typ-ical starting point of an existing question through a series of prompts that culminates inthe generation of a single file that can be distributed to students. Instructors insert intothe template not only their question but also their preferred solution. The template alsoprompts for alternative question data. Because the question data are entered in individualcells, and because the solutions are inserted as generalisable cell-referenced formulas, indi-vidualised questions and answers can be automatically generated. The template alsoprompts the instructor to insert helpful hints for those students who might experience dif-ficulty getting started and to insert more extensive feedback to answers students willsubmit.

The need for feedback for both incorrect and correct student answers is met by thisapproach. All such feedback is customisable by the instructor and the amount of feedbackis limited only by the instructor’s imagination and effort. The question format and presen-tation may also be customised.

The remainder of the paper is organised as follows. The relevant literature is reviewed inthe next section. The final learning object produced by the Excel-based template is thendescribed, followed by an illustration of how this template can be implemented with min-imal instructor effort. The final section presents limitations and suggestions for futureresearch.

2. Literature review

Feedback is essential for enhancing future learning, but providing feedback can be‘challenging, tricky and time-consuming’ (Ramsden, 2003, p. 183). All too often, feedbackis too late or too little (Higgins, Hartley, & Skelton, 2002) to be of real use to students.Epstein, Epstein, and Brosvic (2001) shows that the immediacy of feedback is important,

2 File available upon request by emailing [email protected].

P. Blayney, M. Freeman / J. of Acc. Ed. 26 (2008) 155–165 157

with students using multiple-choice questions marked by a scanner and returned at a sub-sequent class retaining less than students who completed a test on an answer-until-correctbasis. The current context of constrained resources and increasingly larger classes exacer-bates the challenge for accounting instructors keen to provide individualised feedbackbeyond multiple-choice questions.

Rules-based knowledge, like many areas in accounting education, is well suited to drill-and-practice questions for students to undertake to improve their learning. End-of-chapteranswer keys can alleviate the burden of marking and feedback since students can self-man-age some aspect of the process by assessing their answers. There are several possible dis-advantages to this approach. First, there are often a limited number of questions withwhich students can practice and to which there are answers. Second, some students maybe tempted to take inappropriate shortcuts or rely excessively on the work of a peer,and thus fail to learn the course material sufficiently to demonstrate understanding on asubsequent test or exam. Third, students may be deterred from investing further effort ifthe end-of-chapter information is limited to the final answer value without workings, sincethis is usually insufficient to resolve a misconception. Frequently students need formativeassistance during their efforts to complete a question and sometimes this means simplyknowing where to start. Biggs 2003, p. 141 emphasises the importance of self or formativeassessment ‘‘to know how learning is proceeding” and to identify what to do to overcomea mistake.

Feedback to individualised questions provided on an interactive as-needed basis duringthe completion of a question, particularly with large classes, is frankly out of the questionbecause of the human effort required. With spreadsheets now ubiquitous software for busi-ness and part of what Chickering and Ehrmann (1996) call worldware, namely applica-tions already widely available and used outside higher education, it is inevitable thatinstructors will use spreadsheets as the default delivery mechanism for this dilemma withdrill-and-practice accounting questions. For example, Lehman and Herring (2003) useMicrosoft Excel features to enable students to assess their own attempts to drill-and-prac-tice accounting questions on an interactive basis. Answers are checked against the correctvalue and formatted to show in red if incorrect and green if correct. This is an electronicequivalent to checking with the answer key provided at the back of the chapter or text. Afurther feedback solution provided by Lehman and Herring (2003) goes beyond the check-ing of values and involves instructors anticipating common student difficulties and provid-ing them as hints with the Excel ‘comment’ feature (e.g. ‘‘Hint: The asset should not bedepreciated below its salvage value, see p. 333”).

Blayney and Freeman (2004) also use Excel-generated assessments and distribute thefiles to students to complete for drill-and-practice learning and/or for summative assess-ment purposes. The approach has several advantages over Lehman and Herring (2003).First, the question data in the Blayney and Freeman (2004) approach is potentially differ-ent for every student thus discouraging superficial approaches and shortcuts, such as sim-ply getting the answer from a peer. Second, to be assessed as correct, students must entercell-referenced formulas thus promoting understanding of underlying concepts. Sinceerrors are inevitable when values are hard-coded into answer cells (Panko, 1998), requiringcell-referenced formulas also encourages sound spreadsheet practices. Third, cell locations(for question data and required answers) vary for each student thus discouraging shortcutssuch as formula copying. Fourth, the formative feedback can be as detailed as the instruc-tor desires and more than just the electronic equivalent to an end-of-chapter answer key.


Fifth, while the Lehman and Herring (2003) approach is restricted to formative assess-ment, the Blayney and Freeman (2004) approach can be used for summative purposesas well. The real strength of the Lehman and Herring (2003) approach is that it is relativelystraightforward for instructors to replicate.

In their study, Blayney and Freeman (2004) report positive student and staff feedbackwhen their approach is applied to rules-based management accounting. Eighty-two per-cent of students reported that the opportunity to receive formative feedback while com-pleting the assessment motivated them to keep trying. Seventy-six percent of studentsagreed that such assessments were a productive way to learn and 79% agreed that theself-assessment option was non-threatening for learning. Eight academics involved inusing the approach in a range of accounting courses were surveyed and reported it hadrelative advantages over other options in terms of time, cost, effectiveness, and qualityof results. However, almost all academics considered the approach too complex to useeffectively without assistance, since it has requirements that go well beyond lay compe-tency in Excel with its integration of sophisticated Visual Basic macros.

3. Individualised interactive formative assessments

Instructors begin with a question, designed to test student understanding and compe-tence of a rules-based procedure, and the template file3 developed within Microsoft Excel.The example used to illustrate the template is a cost-volume-profit analysis question thatmight appear in a typical textbook assignment or test as follows.

Blastoff Corp sells a single product, namely a high quality children’s shoe, for $100 perpair. The company’s variable costs include manufacturing costs of $32 per pair of shoesand a sales commission of $8 per pair. Blastoff incurs fixed costs for rent ($110,000), sal-aries ($370,000), advertising ($110,000) and other ($21,000). Last year the company sold12,000 pairs of shoes paying taxes at a rate of 50%. Calculate each of the following

1. Contribution margin per unit2. Contribution margin ratio3. Breakeven point (units)4. Profit after tax

On opening the Excel template instructors view a single worksheet (called Instructions)that displays further hidden worksheets as the template steps are completed. Macros mustbe enabled to allow the steps to be fully completed. The instructor is returned to theInstructions worksheet as each step is completed, although navigation aside from the pro-grammed sequence is possible. Instructor input varies considerably with each step. Ulti-mately a single multi-worksheet file is saved for future instructor use and a single file(with one visible worksheet) is saved for distribution to students.

We find it easiest to provide this single file to students by posting it into their courselearning management system (e.g. Blackboard or WebCT). Each student can downloadthe file at their convenience. The file is individualised on initial opening. Fig. 1 illustratesthe individualisation, namely the question data are different from the master data detailed

3 Interested readers can obtain the file by sending an email to [email protected].

Fig. 1. A. Individualised question with suggested hints. B. Individualised question with answers assessed.


above and cell locations are unique (with the ‘home’ cell location as G8 for this particularstudent). Fig. 1 also shows how students can obtain feedback. If this student selects theHelp Me button, suggested hints appear as illustrated in Fig. 1A. In this case the instructorhas chosen to provide both general advice as well as the specific values that the studentwould need to calculate using cell-referenced formulas. The student can use this informa-tion to get started. If the ‘Assess Me’ button is selected, feedback appears as illustrated inFig. 1B. In this case the instructor has chosen to provide both general feedback as well asspecific feedback on the individual answer attempts. In this particular case, only the firstrequirement has been successfully attempted. The remaining three requirements are incor-rect and various levels of feedback directing student efforts have been provided.

4. Creating an individualised interactive formative assessment using the template

To help instructors adopt this approach for creating individualised interactive forma-tive assessments for students, a 10-step template has been developed within MicrosoftExcel. The template, after this referred to as BLASTOFF (Blayney’s Learning & Achieve-ment System that offers Formative Feedback), can easily be used by interested instructors


to develop their own interactive assessments. The 10-steps are identified and described inthe following paragraphs.

4.1. Step 1. Identify a relevant question to support learning objectives

The first and most critical stage of using BLASTOFF should take place before the tem-plate is opened, namely the instructor determining the learning concepts that have beenintroduced and which now need to be reinforced. Once the type of question and the levelof difficulty have been determined, access to or development of an appropriate questionshould take place in Excel, together with its full solution. Step 1 in the Instructions work-sheet reminds instructors that cell referenced, generalisable solution formulas must beprovided for each ‘‘answer” cell. That these solution formulas are correct cannot beoveremphasised.

4.2. Step 2. Paste solved question into template

Selecting step 2 of the instructions provides instructors with the Question worksheet.Instructors paste their solved question with data and answers according to the specific lay-out described. While this layout appears somewhat rigid it allows the program to automat-ically generate individualised questions, marking code, and feedback. The layout for aquestion can be customised as described later in step 8.

� There cannot be any blank rows in the data range or in the answer range. The ‘datarange’ refers to the cells into which instructors place the data labels and values. The‘answer range’ refers to the cells into which are placed the labels and formulas forthe answers.� Data labels must be in the column immediately to the left of the data values (and sim-

ilarly for the answer labels and formulas).

Fig. 2. Demonstration question displayed with appropriate and inappropriate answer formulas.


� There must be at least one blank row between the data and the answers.� Preference for question and answer layout should not be implemented at this stage. For

example, answers can be separated into categories later in step 8 as can further fine tun-ing with formatting (e.g. changing column widths, cell formatting, etc.).� Answer cells must be inserted as generalisable cell-referenced formulas with no hard-

coded raw data. Fig. 2 provides a screenshot using the BLASTOFF demonstrationquestion to illustrate appropriate and inappropriate formulas. Contribution marginper unit in cell C40 is the only answer shown with an appropriate formula. In contrast,the contribution margin ratio in C41 is inappropriate because the cell contains the hard-coded value 0.6 instead of a cell-referenced formula. This error will be identified wheninstructors attempt to proceed to the next step and must be corrected before step 2 canbe successfully completed. Breakeven point (units) in Cell C42 illustrates a less obviouserror with a single hard-coded value embedded within an otherwise valid formula. CellC43 contains an error in accounting procedure entered by an instructor. Profit after taxequals tax expense coincidently because the tax rate used in this example is 50%. Thisinstructor error, because it is a cell-referenced formula, would go unchallenged by thetemplate. However, it would result in students receiving incorrect feedback generatedby the template based on the instructor’s incorrect formula. The integrity and valueof the entire self assessment process hinges on instructors providing correct and gener-alisable solutions.

4.3. Step 3. Identify data and answer cell areas in solved question entered

On executing step 3 in the Instructions worksheet, instructors are guided through theprocess of identifying those cells on the Question worksheet that contain the questiondata. This allows ‘range names’ for all cells containing question data to be created.Fig. 3 provides an illustrative screenshot. This process is then repeated for the answers(which will be cells B40 to C43 in Fig. 3 below).

Fig. 3. Selection of question data area.


4.4. Step 4. Trigger creation of separate answer key

On executing step 4 on the Instructions worksheet, the template automatically copiesthe Question worksheet (with the full cell-referenced solution) to a new worksheet calledAnswerKey and removes the answers from the Question worksheet. The AnswerKeyworksheet is needed to allow student attempts to be marked. This separate AnswerKeyworksheet will be hidden and protected from student access. The answer cells in the revisedQuestion worksheet will now contain a prompt for students (i.e. ‘Answer??’).

4.5. Step 5. Provide alternate question data

On executing step 5 on the Instructions worksheet, instructors are provided with theopportunity to supply alternate data in a new worksheet (called AltData) to be used togenerate individualised question data when the file is first opened by the student. Instruc-tors may enter as many sets of alternate data as desired. Fig. 4 displays a screenshot withthree sets of alternate data. Several suggestions for efficiently creating sets of alternate dataare provided and explained. Considerable care should be taken by instructors to verify thealternate data for both validity and reasonableness. For example, it is likely unreasonableto have the corporate tax rate at 50% (in the master data) and increasing to a maximum of66.55% in Fig. 4.

The alternate data can be used to generate question data by instructors specifyingwhether the alternate data is selected by the program as a ‘complete set’ or ‘totally ran-dom’. If instructors have selected ‘complete set’, students will be provided with data val-ues from only one alternate data column. Selecting ‘totally random’ will provide studentswith alternate data values randomly chosen from any of the alternate data columns.Therefore, it is important that instructors consider the interdependencies of the datawhen providing alternate sets of data and when making the final choice for how the datawill be presented to students. For example, if the sales commission expense is to be main-tained as 8% of the selling price in Fig. 4 below, an instructor would need to make thechoice of ‘complete set’. If ‘totally random’ is chosen, then a selection of values (e.g. suchas $110 selling price and $10.648 for sales commission) might occur which violates the8% relationship.

Fig. 4. Demonstration example with three sets of alternate data.


4.6. Step 6. Create helpful hints for novice students to get started

On executing step 6 on the Instructions worksheet, the template automatically createssome suggested hints to help students begin working on the question. The hints are auto-matically generated in a new worksheet called Help Me, which will be hidden and protectedfrom student view. The hints are in two parts, which can be altered by an instructor.Instructors are first prompted for general advice for students that would typically referto useful learning resources (e.g. textbook reading and examples). Instructors may thenalter the default hints that have been generated for each specific answer. The default specifichints are the final numerical answers that students’ formulas should return. If students sub-sequently use these numerical values as their solutions, the program will mark them asincorrect since cell-referenced solutions are required. Instructors who prefer to provide lessexplicit assistance can replace the default hints with any text.

4.7. Step 7. Create feedback for correct and incorrect student attempts

On executing step 7 on the Instructions worksheet, the template automatically creates anew worksheet (called Assess Me) that is used to generate feedback to assist students whenthey check if their answers are correct. The Assess Me worksheet will also be hidden andprotected from student view. Executing step 7 simultaneously inserts three buttons on theQuestion worksheet. These buttons enable the student to self assess, get assistance to start,or view frequently asked questions.

The self-assessment feature of BLASTOFF is designed so that only generalisable cell-referenced formulas will be marked as correct. The reasons for this are explained above(i.e. promoting learning of concepts, proper spreadsheet practices, and reduce temptationto take shortcuts). Students should be advised that the feedback generated will indicatethat correct values are assessed as incorrect if they are hard-coded or if the cell-referencedformulas are not generalisable.

The feedback is in two parts, which can be altered by an instructor. Instructors are firstprompted for general information or advice for students that would typically refer toupcoming relevant tests, good spreadsheet practices, or simply other learning resourcesthat the student might use to extend their learning beyond the current concepts beingreinforced.

Instructors may alter the default feedback that has been generated for each specificanswer. They may wish to change the content, or perhaps the tone, of the specific feedbackmessages. For example, students with correct answers could be strongly congratulated andencouraged rather than simply marked as correct. Students may also be provided with sug-gestions to attempt more advanced questions.

4.8. Step 8. Fine tune presentation and layout of question and answer

On executing step 8 on the Instructions worksheet, the template prompts instructors tomake formatting and layout changes to enhance the appearance of the formative assess-ment question. The cell locations of data and answers can be changed by cutting and past-ing the data (label and adjacent value) or answers (label and adjacent blank answer cell).Instructors may also choose to change formatting such as cell fonts, alter column widths,insert blank rows, or move buttons.


While data and answers can be moved around the Question worksheet in this step, col-umns must not be inserted between the data labels and data values. Similarly the answerlabels and answer formulas must remain in adjacent columns.

4.9. Step 9. Trigger saving of master instructor file

On executing step 9 on the Instructions worksheet, the template saves the file as it cur-rently exists, namely with multiple visible worksheets. This file allows reuse or refinementof the question in the future without having to start again.

4.10. Step 10. Trigger saving of master ‘cleaned-up’ student file

On executing step 10 on the Instructions worksheet, the template saves the file that canbe distributed to students. Various housekeeping tasks are automatically performed (e.g.hiding and protecting from student view all worksheets except the Question worksheet).

It is imperative that instructors comprehensively review a student’s file for errors. If nochanges are needed, instructors should close the student file without saving. The latter fileshould be the one distributed to students since it automatically generates an individualisedquestion. However, if the review process reveals that further changes are needed, the stu-dent file should be deleted. Required changes should be made to the instructor version ofthe file saved in step 9. Fundamental changes such as a revised formula will require steps3–10 to be repeated. On the other hand, if only fine tuning changes are required (e.g. for-matting), then only steps 8–10 must be redone. Once the required changes have been made,step 10 should be selected again to create a new student file.

Under no circumstances should the student file produced in step 10 be opened in Exceland resaved by the instructor before being distributed to students. The original file savedin step 10 is the appropriate file to distribute to students. Distributing a resaved version ofthe student file would mean all students would get the same question data and cell loca-tions as the individualisation procedure is programmed to execute once only, namely whenthe file is opened the first time. The file is ready for students at the completion of theabove-described 10 steps.

5. Conclusion

This paper describes an innovative approach instructors can easily use to create individ-ualised drill-and-practice questions that provide interactive feedback to basic accountingproblems. Such drill-and-practice questions are typically used in accounting educationcontexts, but they can easily be applied to other rules-based disciplines (e.g. science, taxand engineering). While students reported that interactive assessments are a productiveand non-threatening way to learn (Blayney & Freeman, 2004), the approach was difficultfor instructors to replicate without expert assistance. This paper details how the latterchallenge is overcome. A single, individualised question file can be generated for studentlearning by instructors following the 10 step BLASTOFF template. Student can, at theirdiscretion and without instructor intervention, immediately receive feedback and assis-tance relating to their individualised question.

The approach described in this paper is not without limitations and as such providesopportunities for future development and research. First, the current version facilitates


the provision of formative but not summative assessments. The second limitation relates tothe identification of errors and customisation of feedback. Similar to Lehman and Herring(2003), our approach does not identify the specific error a student has made for a partic-ular incorrect answer. Simply marking an answer as incorrect or correct provides studentswith little direction for their improvement efforts. However, our approach allows instruc-tors the opportunity to provide extensive feedback covering a variety of likely errors.While this may assist some students to self-diagnose specific mistakes and target futureefforts, novice learners may be overwhelmed by having the onus to identify their currenterror and the improvement advice that specifically applies to them.

To partially overcome this limitation, instructors can consider breaking the questionrequirements into separate steps. These steps should reflect the progressive and logicalconstruction of learning a particular concept. A more robust solution that links adviceto specific individualised errors and addresses the summative requirement is the focus ofcurrent research and development. A further opportunity for research relates to diffusionof this innovation and a Rogers (2003) model is one possibility currently being explored.Notwithstanding the necessity to undertake and provide such evidence, we believe that thistemplate will assist instructors in accounting and a range of disciplines to adopt a time-saving strategy that promotes learning rules-based procedures and concepts.

References

Biggs, J. (2003). Teaching for quality learning at university (2nd ed.). London: Open University Press.Blayney, P. J., & Freeman, M. A. (2004). Automated formative feedback and summative assessment using

individualized spreadsheet assignments. Australasian Journal of Educational Technology, 20(2), 209–231.Chickering, A. W., & Ehrmann, S. C. (1996). Implementing the seven principles: Technology as lever. American

Association for Higher Education Bulletin(October), 3–6, <http://www.tltgroup.org/programs/seven.html>.Epstein, M. L., Epstein, B. B., & Brosvic, G. M. (2001). Immediate feedback during academic testing.

Psychological Reports, 88, 889–894.Higgins, R., Hartley, P., & Skelton, A. (2002). The conscientious consumer: Reconsidering the role of assessment

feedback in student learning. Studies in Higher Education, 27, 53–64.Lehman, M. W., & Herring, C. E. (2003). Creating interactive spreadsheets to provide immediate feedback.

Journal of Accounting Education, 21(4), 327–337.Panko, R. R. (1998). What we know about spreadsheet errors. Journal of End-Use Computing, 10(2), 15–21.Ramsden, P. (2003). Learning to teach in higher education (2nd ed.). London: Routledge.Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.

Documents

Individualised interactive formative assessments to promote independent learning