Upload
anita-allenson
View
214
Download
0
Embed Size (px)
Citation preview
A system for enhancement A system for enhancement of manual exam reviewing of manual exam reviewing processprocess
Ivan BudišćakBoris Milašinović
Faculty of Electrical Engineering and Computing
University of Zagreb, Croatia
EnvironmentEnvironment1st year of studyProgramming and Software
EngineeringData Structures and Algorithms
◦>=700 students on each course◦Continuous assessment
Using software for automatic evaluation of assignments (Ahyco) Homeworks Quizzes (Multiple choice tests)
Manually reviewing 2 mid-term exams and final exam Class activity
Traditional manual exam Traditional manual exam reviewing (with pencil and reviewing (with pencil and paper)paper)Pros:
◦“Natural” way to review exams◦Can be done everywhere (in case you
have access to exams )Cons:
◦Difficult to split exams on more than one reviewer (lot of merging and sorting)
◦Unable to have dislocated reviewers◦Students have to be able to see reviewed
exams (additional time and place)◦Needs place for storing exams
Digital exams archivesDigital exams archivesRetain manually reviewing process Keep students exams and reviewing
remarks together in digitalised archiveMotivation:
◦ Potentially useful resource for further research i.e. finding common mistakes
◦ No need to keep thousands of papers 3 x 700 students x more that 4 papers per each
student => huge paper towers ◦ Enable dislocated reviewers◦ Enable dividing particular exam on 2 or more
reviewers◦ Avoid long queues of students just want to see
their exams
Former attempt (few Former attempt (few years ago)years ago) Students were writing code in labs under
assistants supervision and uploaded final code Code was manually reviewed using HTML editor Pros:
◦ No problems with recognizing handwriting ◦ Code indentation was able to be solved if program
had been syntactically correct Cons:
◦ Time consuming on both sides of examination process: Small lab’s capacity Many of students in the first year of study still write faster
than they type◦ Technology drawback:
Reviewing extremely time consuming using rudimental HTML editor
Limited functionality of reviewing process
New technology – new New technology – new chancechanceSilverlight
◦Cross platform and cross browser◦Rich (and fast) user interface inside
browser 2D graphics Animations
◦HTML DOM access from managed code
◦Web services
How it works (2)How it works (2)
Main toolbar◦Browse assessments◦Browse questions◦Enter score◦Save remarks
How it works (3)How it works (3)
Exam toolbarBrowse exam papers (pages)Enter text, draw line and
polylineSelect remarkZoomSave remark template
Reviewers opinionsReviewers opinionsTakes little bit longer than “normal”
reviewing with paper and pencilNot suitable for small numbers of exams
◦Time needed to get a grip on tool’s interface and mouse movement
Opposite situation: more exams – more same errors◦Once written, comments of common errors
can be just dragged and dropped on the exam
Saves time for exams insights◦Still has to be hold, but last significantly less
Students opinions - Students opinions - questionnairequestionnaire
Question Total answers 1(%) 2(%) 3(%) 4(%) 5(%) -(%)
How often do you attend exam insights (for all courses): 1. Almost always 2. Often (cca 75%) 3. 50% 4. Rarely (cca 25%) 5. Almost never
133 6,02 8,27 17,3 34,6 33,1 0,75
When and why do you attend exam insights for manually reviewed exams: 1. Never, because there is always a large number of other students 2. I attend just to see what I did wrong but I achieved expected score 3. I attend because I want to try to get some extra points but I know what I had done wrong and I have achieved expected score 4. I usually attend because I like to know what I had done wrong and my score is less than I expected 5. I will attend because I don't understand official answers and would like reviewer to explain me difference between my answers and official answers
133 18,8 6,77 4,51 58,7 6,02 5,26
With score from 1 to 5 rate your satisfaction after exam insight (1 - not satisfied, 5 - very satisfied)
133 3,76 12 33,8 27,8 4,51 18,1
Do you attend exam insight for online reviewed exams: 1. Never, because I saw what I had done wrong and I achieved adequate score 2. I attend because I know what I had done wrong but I think I should achieve greater score 3. I attend because I don't understand reviewer's comments 4. I attend to try to get some extra points 5. I never attend exam insight
133 57,1 16,5 5,26 4,51 11,3 5,26
With score from 1 to 5 rate usefulness of online exam reviewing (1 - not useful, 5 - very useful) 133 0,75 1,5 1,5 23,3 66,9 6,02
Very useful because it is now possible to see my own mistakes, and the reviewer comments, and thus reducing the number of students who go on insights which are less crowded and everything is quickly resolved.Most of us are also curious and eager, we want to see why we do not have the expected number of points and that is why online access to the exams is very useful to us. Also, I think we are in most cases too lazy to go to the insights to see our mistakes and in this way, from pure curiosity, we all (or at least most of us) will see where we made a mistake and what official answer should look like. In this way we also learn a thing or two.
Just one question... WHY THIS THING WAS NOT INTRODUCED FOR THE ENTIRE FACULTY A LONG TIME AGO! : P
Students opinionsStudents opinions – – comments (1)comments (1)
This is an excellent idea and should be used on as many courses as possible. On the Web, we can see where we made a mistake and there is no need to go to exam insight, which is usually crowded. Also, thanks to this system, the reviewing criteria are much clearer and more uniform. [Otherwise, this is the only questionnaire that I filled because I believe that it really can achieve positive change.] Once again, every honour for this initiative and I hope that more students will complete the questionnaire, and that this method of assessment will be used for all students on this course, and in the (near) future on other courses.
Online insights are a great thing, because it allows all of us direct access to our exams. Thus saving us time that would have otherwise been lost (often in vain), on the insights and waiting in line. Praise the idea, performance and assistants in a detailed and well performed job even though this is additional burden for them!
Students opinionsStudents opinions – – comments (2)comments (2)
Might be useful if there is an official communication channel that would allow online access (something different from the e-mail). It would be very helpful for fellow colleagues that are traveling to the faculty. Distance from the faculty is one of my reasons why I don’t attend the insights and consultation.
Only 2 “negative” comments:◦ Online insights are not useful to me because I don’t
understand official exam solutions and I want assistant to explain me the difference between official and my solution.
◦ Some assistants are to harsh Note: Probably was comparing his/her and friends exams
Students opinionsStudents opinions – – comments (3)comments (3)