This article was downloaded by: [University of West Florida]On: 06 October 2014, At: 09:55Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

European Journal of EngineeringEducationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/ceee20

Using multi-mode assessments toengage engineering students in theirlearning experienceS. M. Fei a , G. D. Lu a & Y. D. Shi aa Department of Mechanical Engineering, College of Mechanicaland Energy Engineering , University of Zhejiang , Hangzhou,310027, ChinaPublished online: 24 Apr 2007.

To cite this article: S. M. Fei , G. D. Lu & Y. D. Shi (2007) Using multi-mode assessments to engageengineering students in their learning experience, European Journal of Engineering Education, 32:2,219-226, DOI: 10.1080/03043790601118564

To link to this article: http://dx.doi.org/10.1080/03043790601118564

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

European Journal of Engineering EducationVol. 32, No. 2, May 2007, 219–226

Using multi-mode assessments to engage engineering studentsin their learning experience

S. M. FEI,* G. D. LU and Y. D. SHI

Department of Mechanical Engineering, College of Mechanical and Energy Engineering,University of Zhejiang, Hangzhou, 310027, China

(Received 12 April 2006; in final form 25 October 2006)

This paper describes the research and practice on the reforming of the examination method ofengineering graphics at Zhejiang University. Based on the theory of standardizing general examinationsand emphasizing key examinations, a new combined examination method of multiple modes hasbeen explored and practised. The examination modes include unified large-scale examination mode,interviewing mode, additional score mode by designing an object or writing a paper, and do-it-yourselfmode, i.e. self-questioning, self-answering and self-evaluating. The method forms a good synergybetween unified large-scale examination and free exertion. The final percentage score is given on thebasis of the assessment mode reform.

Keywords: Engineering graphics; Unified examination; Multi-mode examination; Special exami-nation; Self-evaluating; Examination database; Structure design

1. Introduction

Research on assessment suggests that examinations have traditionally dominated studentassessment procedures and the vast majority of current undergraduate courses continue toassess student learning by end-of-course examinations. Improper assessment mode maycause students to take a strategic approach and end up focusing on the superficial aspectsof their assessment tasks, rather than engage in meaningful learning activity. Just as Elton andJohnston (2002) pointed out, examinations tend to test for the lower levels in the hierarchy ofknowledge, such as recall and simple applications rather than for creativity, critical thinkingor the development of academic and/or life skills. In such cases, it is likely that the assessmentsystem is not aligned with the learning objectives as students are concentrating on thesuperficial and trivial. Therefore, examination methods are a yardstick to encourage the studyof the students (Gibbs 1999, Norton 2001, Otto 2000).

The reform of the examination methods should be beneficial to the enhancement of theability of the students on the one hand and the improvement of teaching methods on the other.As a result, the motivation and diligence of both the students and the lecturers are enhanced.

*Corresponding author. Email: zdfsm@yahoo.com

European Journal of Engineering EducationISSN 0304-3797 print/ISSN 1469-5898 online © 2007 SEFI

http://www.tandf.co.uk/journalsDOI: 10.1080/03043790601118564

Dow

nloa

ded

by [

Uni

vers

ity o

f W

est F

lori

da]

at 0

9:55

06

Oct

ober

201

4

220 S. M. Fei et al.

The reform of the examination methods should put emphasis on the knowledge, quality andability of the students. All these aspects should be examined so that better environment andconditions could be created to further the development of the students.

Therefore there is a need to combine the traditional unified examination with multiple modesof assessments. Through the reforms of assessment methods and final score evaluation, thestudents’ motivation could be enhanced fully. As a result, they could learn with enthusiasmand self-motivation. If the traditional two-hour examination could be transcended into twodays, two weeks or even the whole study process, the lecturer would have already achievedinitial success.

The engineering graphics course of Zhejiang University exposes First Year engineeringstudents to conventional drafting techniques, instrument techniques, computer graphics andbasic knowledge of engineering design. The typical class consists of first-year students andspans a wide range of previous experience in engineering graphics.

2. Assessment mode

The multi-mode assessment includes standard examination mode and special examinationmode.

2.1 Standardize the examination mode

The standard examination mode means unified examination based on national examinationdatabase.

2.1.1 Unified examination. The university has requested to standardize the examinationin four aspects: unified examination setting and marking; unified teaching requirement andexamination standard. To reflect the drawing course as a “public platform combined withsynergetic improvement”, the common and basic platform parts have been taught and examinedseparately. As a result, the credit, effectiveness and discern ability of the examination havebeen increased. The examination could evaluate the quality of the students’ learning moreeffectively, completely and accurately. The teaching quality has been stabilized and improved(Lu et al. 1999, 2001).

Unified examination paper setting and format. The course leader should organize unifiedexamination paper setting and format. The questions should be both objective and subjectiveand consist of different types and proportions. Intelligent examination database for automaticpaper setting could be applied, which should reflect the changes of teaching reform.The examination papers should concentrate on the examination of the students’ understandingof basic theory, knowledge and skills together with the improvement of problem analysis andproblem-solving abilities. The examination mark distribution should be as close to Gaussianas possible.

Unified examination and unified examination marking. For those students who have a basicunderstanding of the theory, knowledge and skill of the course will be awarded pass mark.For those who have passed and also have certain ability in analysing and solving problemsa good mark will be awarded. The last category is the students who not only have the justmentioned aspects but also posses certain creativity. Distinction will be awarded.

To make the examination more effectively, completely and accurately reflect the qualityof the studies of the students and enhance the motivation and creativity, the increase of thefrequency of quizes and the weighting of the quiz in the final score is necessary. At the various

Dow

nloa

ded

by [

Uni

vers

ity o

f W

est F

lori

da]

at 0

9:55

06

Oct

ober

201

4

Multi-mode assessments 221

stages of teaching the lecturer could carry out small quizes in the form of written exams,discussions, essays, small quizes or computer lab. Good results should be included in thefinal score.

2.1.2 National examination database. In parallel with this project, we have been activein building the national examination database on engineering drawing. From 1992, we havepublished two versions through the Higher Education Publisher. One version is the olderscanning matrix database and another is the vector intelligent database. The undergoing project(sponsored by the Ministry of Education) of new century internet course engineering – vectorintelligent examination database (v2.0) – has passed quality checking on 7 December 2003in Beijing.

The vector intelligent examination database (v2.0) is based on the original version 1.0. Thesolutions for all the questions in the database have provided plus the new development in theteaching and reforming of the engineering drawing course. The structure has been optimizedbut the basic structure of the version has been kept. The new examination database has thefollowing functions completed:

1. Intelligent creation of solutions. In completing the vector questions of an examinationpaper, the system is able to create the vector solutions automatically the right solutions forthe reference to the students and lecturers.

2. The ability to modify the solution. Through the function the lecturer is able to correct thenewly found problems (wrong answers or multiple solutions) quickly so that the databasecould improve gradually.

3. The ability to modify the solutions of an examination paper. Due to the personalunderstanding difference to some of the solutions there is a need to modify the papersolution with personal touch. It is possible to modify a particular solution (or solutions)of an examination paper. This modification will not change the original correct solution inthe database. Only the solutions of a particular examination paper have been modified.

4. To provide functions for adding and deleting questions and solutions. The lecturer couldadd new questions and solutions into the examination database based on parameters suchas the question types, knowledge focus, degree of difficulty, the estimation of examinationduration.

5. The function of previewing and printing of the examination paper and solutions. The versionis able to realize the previewing and printing of the examination questions and solutions onpaper format no. 3 and 4. At the same time it is possible to magnify, reduce and translateof the examination paper and solution.

6. The ratio of the examination types has been adjusted to realize the organic integration ofthe question database and the solution database.

2.2 Furthering the special examination mode

Modern society requires people to have the following capabilities: language expression ability,graphic thinking ability, computing skills. Language ability includes oral expression ability(including Chinese and foreign language), writing (including text and graphic expression);graphic thinking ability includes graphic expression, graphic analysis and graphic assembly;computer skills include the abilities of text processing, graphic processing, programming andsoftware application. It can be said each of the above aspect is closely related to the drawing.The reforming of the examination method helps to reinforce.

Dow

nloa

ded

by [

Uni

vers

ity o

f W

est F

lori

da]

at 0

9:55

06

Oct

ober

201

4

222 S. M. Fei et al.

Based on standardized examination, we have tried to explore the reforming of examinationmode. The following aspects have been included.

2.2.1 Interviewing to replace examination. In the second semester and among theMechanical Engineering classes, students with good quiz marks, active in thinking, havingexcellent large coursework are exempted from the examination of Engineering Drawing.Instead they are interviewed. For those qualified students, their average quiz mark shouldbe over 85 marks. The large coursework has to be excellent. They have to apply for exemptionand then permissions will be decided by the lecturer. The percentage of the students allowedfor exemption should not exceed 20% in a small class. The interviews should be conductedby the lecturer and the appointed lecturers from the teaching office.

The interview should be started with an oral presentation by the student in order to examinethe nature and ability of oral presentation of the student (Samuelowicz 2002). The interviewshould mainly cover important concepts, analysing and solving difficult questions, combinedanalysis and application, the common mistakes in drawing, experience on the course studiedand the general understanding on the course and its development. The interview is carried out ina relaxing atmosphere and not much review is required. No extra burden is added to the students.

The oral presentation occurs in the last week of the second semester, and is the culminationof a number of interactions between students and academic staff. The presentation lasts for15 minutes, followed by a short time for questions and answers. The criteria and markingschedule for the oral presentation are shown in table 1.

2.2.2 Structure expression as small design. In order to improve the motivation andcreativity of the students, small design has been carried out in the stage of mechanical drawing.The lecturer specifies the functional requirements of the components and based on which thestudents will perform the structure design and drawing. These small designs follow the basicstructure and the rational expression of the components and the assembly. Emphasis is put onthe novelty and creativity of the structure of the components and assembly. Attention is alsopaid to the accuracy of the drawing expression and the suitability of the drawing projection.Higher proportion is given in the combined score to the designs made by the students who areactive, creative and correct in design.

The small design of component structure expression has been carried out during the stageof drawing expression and component drawing. The lecturer will provide the optical axis,the roughcast of the supporting frame and body box. The components to be assembledto the axis will be specified together with the requirement of sealing and the joints fix-ing in the circumferential and axial direction. The machining method of the axis and theaxis structure will also be given. The projection drawing will be drawn based on the abovespecifications. Based on the function of the supporting frame and the body box the com-ponents will be completed. Properly designed drawing projections will be drawn. For more

Table 1. Assessment criteria for oral presentation.

Clarity of presentation Mark

Very poor/no real preparation 1Minimum acceptable (evidence of planning, possible to

follow content)2

Good (planned and reasonable speaking style) 3Very good (well-structured, covered main points, spoke well) 4Excellent 5

Dow

nloa

ded

by [

Uni

vers

ity o

f W

est F

lori

da]

at 0

9:55

06

Oct

ober

201

4

Multi-mode assessments 223

complex components a rough drawing expression, etc., could be given first besides providingthe roughcast of the components.

The types of assembly structure expression designs include: the common componentsdesigned by the students, the connection assembly structure of the fastening component,the assembly structure including axis, gear, nuts and bolts; the design of a sealing structure;the design of a convey structure; drawing the schematic function and structure, drawing thestructure assembly, etc.

The small design of the structure expression is a new challenge to the lecturer. It alsoincreased the amount of work. The lecturer has to make careful preparations. They have to notonly prepare questions of different contents and types but also motivate the students throughtypical examples. Through the small design of the structure expression, the thinking of thestudents could be motivated. Their views will be broadened. The inspiration and creativity ofthe students will be stimulated.

2.2.3 Writing small essays. The entry of WTO bringing the international competition,foreign language becomes very important. However, it cannot be denied that the mothertongue may be more important for those who live inside China. If a small essay with drawingand text could be written in this course, particularly it is in digital form, the students wouldbenefit a lot. Obviously, small essay is different from literature survey or laboratory report.

The main purpose of the final report is to provide a self-assessment tool for the student toreview and identify their progress and weaknesses to date, and the final report is submittedby students at the end of the semester. This report is a brief quantitative assessment of thestudent’s progress to date and is approximately 3000–4000 words in length. A brief guidancegiven to the students is that the report should contain a summary of the following details:

• basic conceptions and solution activities;• general understanding on the course and its development; and• self-assessment.

The criteria for marking are outlined below in table 2.In general, we should encourage the students to think, analyse and deepen their

understanding and mastery of the related problems (Katariina 2004). It can be either a par-ticular knowledge point appearing in the study or the correlation of the various knowledgepoints. It can also be the macro comparison and analysis. The marks of the small essay willhave certain portion in the final mark.

2.2.4 Self-question setting, testing and evaluating by the student. In order to keep thestudents having the persistent pressure and self-motivation, a complete new approach has beenexplored. The role of the student is reversed. In the examination model the student is the centre

Table 2. Marking criteria for final report.

Final report criteria Mark range

Completely unsatisfactory. 0–2Unsatisfactory. Aims not met. Nothing worthwhile

produced.2–3

Satisfactory. Report is well written and structured. 3–4Good. Report is very well written and structured in

every way. Student able to talk with confidence.4–5

Dow

nloa

ded

by [

Uni

vers

ity o

f W

est F

lori

da]

at 0

9:55

06

Oct

ober

201

4

224 S. M. Fei et al.

so that they could face the new challenge (Price et al. 2003). Through the process the studentshave made real improvement.

We have proposed and executed the “three self-evaluation” model of self-questioning,self-testing and self-evaluation. That is to say, the student will set the questions themselvesand then answer the questions and report the experience of the process. This is an operationalnew approach. From the beginning of the term the students are told the availability of suchan examination mode. It is new to the student and they will think it’s a challenge at the sametime. The middle-term examination results could be used as the main criterion. Those whoare in the top 20% in the class will be allowed to apply for the “three self-evaluation” test.In this way the first half of the semester is still based on basic theory and using the traditionalexamination mode. In the second half of the semester, enough space is given to development.The examination time has been extended effectively and the examination is embedded in theprocess of study. The examination and study are integrated organically. A good student wouldtypically do a thorough review of the subject he/she learned before contemplating the set-ting of the self-examination questions. This process is particularly helpful for the student toget a general view of the subject and to broaden their knowledge base. The student has alsoto consider from a lecturer’s standpoint the criteria of good understanding. Based on these,the student would select questions from Auto CAD exercise books or other available sources.After completing the questions the student would compare the results with the answers given inthe answer book. Then analyses have to be given on any errors and imperfections in their ownanswers. All these processes have to be recorded and submitted as a report for the lecturer toaward a mark, which would be combined with other marks such as middle-term examinationsand coursework marks.

2.2.5 Other models. We should encourage the students to propose the various models sothat they can prove that they have mastered the content of the course and demonstrated theircreativity and ability (Scouller 1998; Zimitat 2003). All these students should be exemptedfrom examination. For example, the student could be encouraged to draw the proper A3 formatengineering drawing in their spare time. They can use CAD or drawing instrumentation. Ifthe standard is satisfactory it should be accepted and the marks will be considered in the finalmark. This practice would enhance the confidence in learning the engineering drawing to thosestudents who are weak in the imagination of space but good at problem solving.

2.3 The reforming of the model of mark evaluating

The reforming of the mark evaluation is a very important part of examination reforming.We have to change the method of deciding the students’ mark based on a single examinationpaper. We should encourage the students to realize their full potential and they are able tomake correct judgment. This is beneficial to the healthy growth of the students’ ability.

The examination paper should not be the only criterion to judge the students’ marks.Emphasis should be focused on the judgment of the ability of the student. We believe thatthe final mark should be a combined reflection of the various abilities of the students in thestudy of engineering drawing. These abilities include the ability of hand on, self-learning,oral expression, creativity and space thinking, etc. Because of this the various factors areconsidered in the final mark evaluation. Also different methods are adopted during the publicplatform stage and improvement stage. More space is also made to the lecturer in makingthe evaluation. The coursework mark mainly includes coursework, behaviour in the classand computer sessions, etc. The course quiz includes the unit examination and middle-termexamination, etc. At the same time, the small essay and structure design are considered.

Dow

nloa

ded

by [

Uni

vers

ity o

f W

est F

lori

da]

at 0

9:55

06

Oct

ober

201

4

Multi-mode assessments 225

3. Implementation and feedback

The practice of standardizing the examination mode was carried out in the semester one ofacademic year 2003–2004 on the class of 2003 among 3000 students from the University ofZhejiang. Of the students, 375 were female and 2625 were male. The vast majority came fromscience faculties such as Mechanical Engineering, Energy Engineering, Material and ChemicalEngineering, and so on. Based on experience obtained regarding the above-discussed pointsthe engineering drawing course, the practice of furthering the special examination mode wascarried out in the semester two of academic year 2003–2004 on the class of 2003 among 700.

The multi-mode assessment has been shown to provide a number of benefits. Feedbackelicited in class is outlined below in table 3

The effect was good as reflected by both the students and lecturer. Results, which are notpossible to achieve by conventional examination, have been realized. Just as a student said:essays seem not to be a matter of rereading the textbook or learning it off by heart, but a matterof thinking. The following are some of the feedbacks of self-testing from the students.

Lou Hai Yan of the Zhu Ke Zhen College (3013001126) reported that: “I felt that I hadlearnt a lot from the examination paper. First the review for examination paper setting is verydifferent from the conventional review process. In order to set the examination paper, I haveto read book with a purpose and from a point of view of the lecturer to understand the student.The common errors and essential contents are processed . . .”.

Chin Chi of the Zhu Ke Zhen College (3013001127) wrote that: “When I first heard thatto set examination paper could add marks or be exempted from examination I thought itwould be easier than a full review since we could check many materials. But soon I found itwas not a simple process after two days of examination paper setting. It may be even moretime consuming than reading and review. At the same time I have learnt a lot through theprocess . . .”.

Guo Chin of the Zhu Ke Zhen College (3013001124) wrote that: “By setting examinationpaper to test myself on whether I have a good understanding and mastery of the subject forthe first time is interesting and special . . .”.

The authors have been involved in the teaching of the subject before the assessment reform.The general impressions are that the students’ abilities have been improved significantly in thedesign related subjects in the third and fourth years through informal contact and monitoring.

Table 3. Quantitative student feedback on the multi-mode assessment.

Strongly Neither agree Stronglydisagree (%) Disagree (%) nor disagree (%) Agree (%) agree (%)

Helped me to improve myunderstanding of course concepts

0.0 6.1 24.8 61.1 8.0

Improved my overall knowledge ofcourse material

0.0 10.6 31.5 56.9 1.0

It is was a valuable learningexperience

0.0 15.8 24.9 53.9 5.4

Made me spend more time learningabout the content of the course

0.9 11.7 20.4 56.3 9.3

Challenged me to think more aboutthe course content

0.0 5.7 25.7 59.3 8.3

Helped me to prepare for themid-term exam

2.7 19.8 20.9 54.8 1.8

Helped me to understand better thedifferent types of knowledge I cangain from this course

0.0 6.8 14.7 60.4 14.1

Dow

nloa

ded

by [

Uni

vers

ity o

f W

est F

lori

da]

at 0

9:55

06

Oct

ober

201

4

226 S. M. Fei et al.

4. Conclusion

This paper has discussed the multi-mode assessment used to assess student learning. It isrevealed that the reform of the examination method is important for higher-level learning.We have combined both traditional and new examination methods in a synergistic way undermultiple modes such as traditional unified examination, interview based on coursework marksand the mode of bonus marks based on design and essays. The “three self-evaluation” modesare actually a synergistic combination of traditional unified examination and self-exertion.The reform of examination methods has improved the diligence, motivation and creativity.At the same time, it raises demands and standards to the lecturers.

The examination method revealed in this paper is engineering graphics-based, but the focusis on generic rather than subject-specific criteria, so the examination method is intended to beadaptable to other disciplines.

References

Elton, L. and Johnston, B., Assessment in universities: a critical review of research, 2002. Available online at:http://www.ltsn.ac.uk/application.asp?app_resources.asp&process_full_record&section_generic&id_13.

Gibbs, G., Using assessment strategically to change the way students learn, in Assessment Matters in Higher Education,edited by S. Brown and A. Glasner, pp. 41–53, 1999 (SRHE & Open University Press: Buckingham).

Katariina, A., Portfolio assessment on chemical reactor analysis and process design courses. Eur. J. Engng Educ.,2004, 29, 267–273.

Lu, G.D., Tan, J.R. and Zhang, S.Y., The research and practice on the reformation of engineering graphics coursesystem. J. Engng Graphics, 1999, 20(4), 92–97.

Lu, G.D., Tan, J.R. and Zhang, S.Y., Teaching reformation and subject construction of engineering drawing. J. Res.Higher Ed. Engng (supplement), 2001, 25(4), 77–80.

Norton, L.S. and Tilley, A.J., The pressures of assessment in undergraduate courses and their effect on studentbehaviors. Assess. Eval. Higher Ed., 2001, 26, 269–284.

Otto, R., Assessment of student learning: evolution of objectives in engineering education and the consequences forassessment. Eur. J. Engng Educ., 2000, 25, 339–350.

Price, M., Rust, C. and Donovan, B.O., Improving students’ learning by developing their understanding of assessmentcriteria and processes. Assess. Eval. Higher Ed., 2003, 28, 147–164.

Samuelowicz, K. and Bain, J.D., Identifying academics’ orientation to assessment practice. Higher Ed., 2002, 43,173–201.

Scouller, K.M., The influence of assessment method on student’s learning approaches: multiple choice questionexamination versus assignment essay. Higher Ed., 1998, 35, 453–472.

Topping, K., Peer assessment between students in colleges and universities. Rev. Ed. Res., 1998, 68, 249–276.Zimitat, C. and Miflin, B., Using assessment to induct students and staff into the PBL tutorial process. Assess. Eval.

Higher Ed., 2003, 28, 17–32.

About the authors

Shao-mei Fei was born in 1966. She got her PhD in Internal Combustion Engine in 1993 at theUniversity of Zhejiang (China). She is currently an Associate Professor. Her main teachinginterests are Engineering Graphics and Computer-aided Drawing. She has a particular interestin assessment.

Guo-dong Lu is currently a Professor involved in the teaching of Engineering Graphics,Computer-aided Design and Computer Graphics. He is the deputy director of undergraduatestudies.

Yue-ding Shi is a professor dedicated to educational sciences with more than 30 years ofacademic research and teaching in engineering drawing and design. He is a popular lecturer.

Dow

nloa

ded

by [

Uni

vers

ity o

f W

est F

lori

da]

at 0

9:55

06

Oct

ober

201

4