School: Computer Science Subject: EVS and Peer …jiscdesignstudio.pbworks.com/w/file/fetch/66200924/ITEAM EVS case... · School: Computer Science Subject: EVS and Peer Assessment

School: Computer Science Subject: EVS and Peer Assessment ___________________________________________________________________

1) How did you use EVS in your teaching? It was used primarily for peer evaluation on two courses: E-Media Design, a first year 15 credit course, and Multimedia Specification, a 30 credit masters course. That latter course was where myself and Trevor Barker first used EVS. That success, led us to adopt it for first year students. Besides that, it has also been used for pop quizzes on first year lectures, and one in-class test.

2) Why did you use EVS in this context? In the masters’ course, students had to produce group projects: namely prototype websites, and final websites. On each occasion, students had to present their work in front of the class. Then the whole class would vote. The vote would appear immediately after the presentation. Then at the end of the day, I would factor in mine and Trevor Barker’s marking, and then recalculate the score (20% from the students, 80% from the tutors). With first years, instead of marking each other in front of the class, which might prove intimidating for them, we instead got them to mark a previous cohort’s work, and give them marks according to how similar their marking was to mine and Trevor Barker’s. To a certain extent this is a questionable strategy (how can you be awarded credits for mere coincidence of opinion?) However, the value of the assignment is very small (1.5 credits) – but it nonetheless forces all students to communally participate in the act of judging quality. By doing so they have a much clearer idea of what is expected. 3) What were the benefits of your application of EVS for you?

Using EVS gives us the chance of doing EVS publicly, and live. That means the impact is much greater. Also, using TurningPoint means managing the data (e.g in the first years case, 27 different votes x >200 students – e.g >5400 separate student decisions made) is actually very easy.

For lecturing the benefit is to stop the students getting too bored or restless – it punctuates the lecture with things to make the students think, and to allow the students to participate.

4) What do you think were the benefits of your application of EVS for your

students?

For the peer evaluation, it means they get their results at phenomenal speed –basically the software part of the assignment gets marked within 8 hours!

5) Did you experience any drawbacks or problems with using EVS that may prompt you to modify or develop the use outlined above?

The one drawback EVS has over something like QMP is that it becomes quite difficult to ask complex questions (e.g multiple response or fill in the blanks). It also means that the whole class must answer questions at the same speed. Overall though, it is a fantastic technology which makes teaching and learning so much more enjoyable. Sometimes there are some lingering issues of which-clicker-belongs-to-which-student – as well as accessibility/disability concerns.

School: Education Subject: Use of EVS in small groups ___________________________________________________________________

1) How did you use EVS in your teaching?

Mainly as a pedagogical tool. I would give the students a statement and ask them to vote from strongly agree to strongly disagree. We would then discuss answers and allow for a re-vote and reflection.

2) Why did you use EVS in this context? As I wanted to challenge the students’ thinking, get them to talk to each other and engage in personal reflection.

3) What were the benefits of your application of EVS for you? It is very easy to administrate as it doesn’t involve ‘signing in’ specific sets and allows us to tailor sessions according to responses and misconceptions.

4) What do you think were the benefits of your application of EVS for your students? Through discussion with them, they felt that they were engaged in higher order thinking as a direct result of the technology. Also, as they were a small cohort, they felt that their vote had more ‘worth’ and encouraged them to think about their response more than they otherwise would have.

5) Did you experience any drawbacks or problems with using EVS that may prompt you to modify or develop the use outlined above? The biggest drawback is to ensure that it is not used as an ‘entertaining aside’ as it will be perceived as a novelty which will wear off. The absolute key to avoiding this is to ensure that great thought is put in to the question asked.

6) Further information

EVS builds student confidence

Builds mutual trust

Provides anonymity

Good for generating debate

Less dominant members of group given a voice

Forced all members to have ‘a view’

Individual votes counted – only ten students so one vote made a difference which led to deeper thought

Disadvantages: students may ‘play safe’ due to small numbers, results easily skewed due to small numbers.

Question setting most important part, MCQs mainly used yes/no/maybe – in future plan to use yes/no

Different way of learning; students found it useful and enjoyed it.

EVS wasn’t a novelty that wore off

Reflection and key points: 1. asked an EVS question 2. delivered the lecture 3. ask the same question again – deeper level of learning and thinking 4. immediate feedback 5. promotes dialogue 6. reflective 7. leads to higher order thinking

Would like more information on the life cycle of EVS – purchase process, distribution, loss and replacement.

School: Nursing, Midwifery and Social Work Subject: Nursing Programme/module: Pre-registration Nursing ___________________________________________________________________

1. How did you use EVS in your teaching?

We use them during the new starter event for nursing. Louise McEarlean sets a quiz on A&P and candidates are invited to vote on which response to a question is true or false (from a set menu).

2. Why did you use EVS in this context?

It’s good fun and a good way of getting a new group to work together and for individuals in it to evaluate their knowledge in a fairly safe way

3. What were the benefits of your application of EVS for you?

Facilitated group work and sense of community 4. What do you think were the benefits of your application of EVS for your

students? Learning in a safe way

5. Did you experience any drawbacks or problems with using EVS that may

prompt you to modify or develop the use outlined above?

Access to handsets which has now resolved as new students are issued with handsets

School: Health and Emergency Professions (Health and Social Work from 1/08/12) Subject: Debates to support deep learning and the development of criticality and team working skills Programme /module: Year 3 Physiotherapy BSc (Hons) ___________________________________________________________________


The cohort was split into groups of 24 that was then further split into teams of 6 students. Each week a team was allocated to argue either for or against a motion and had to produce a short presentation. A tutorial on Monday each week supported the students developing their ideas. The debates occurred in a seminar at the end of the week. The teaching session consisted of Vote using EVS Presentation by team arguing for Presentation by team arguing against General discussion Vote using EVS. The presentation and results of the votes were placed on StudyNet, this had the advantage of students being aware that there were different opinions as sometimes one group would vote for and another against.

2) Why did you use EVS in this context?

The subject material for this module was dry and potentially uninspiring; by using the element of competition and getting students to engage in critical discussion it motivated them to participate actively in learning. Team working is a core skill for health professions and this approach supported the development of team working skills. The voting supported peer review and building of a learning community. What were the benefits of your application of EVS for you? It reduced peer pressure and made all students feel that they had a valid voice. It supported participation and active learning as the students listened intently to the presentations as they needed to take action afterwards - vote. The competition motivated students to actively search, critically appraise and develop strong skills in justifying their stance.

3) What do you think were the benefits of your application of EVS for your students? Students identified the following benefits in a follow up questionnaire. They had experienced three debates using a show of hands and three debates using EVS.

Anonymous and reduces peer pressure “If you have friends in both teams and want to give an honest opinion, then EVS is the best way of doing this without anyone else knowing which team you voted for”

Engagement “Made the debate seem more important” “Tend to participate more (listen better) as it feels more professional

Fun “Easy to use enormous fun” “Futuristic – cool!”

Graphical representation “Good visual feedback” “Could see percentage rather than just number of people”

Accurate “Represents votes more accurately” They obviously thought my hand counting wasn’t very accurate! It was noticeable that in debates 4-6, where the EVS was used, when compared to 1-3 when a show of hands were used, the students were more willing to change their mind and vote differently after the presentations than before. This may be related to the anonymity provided by EVS.

5) Did you experience any drawbacks or problems with using EVS that may prompt

you to modify or develop the use outlined above?

It is important to give clear instructions and ground rules for the debate prior to the start. For example late comers cannot vote as they didn’t vote at the start this resulted in an attendance rate of 98.6% and a prompt start each week.

1) Any further information

The technology was easy to use and yet had a significant impact.

School: Life Sciences Subject: Human Physiology Programme / module: BioScience and Sports Science programmes ___________________________________________________________________


EVS was widely used in my teaching, including formative questions during lectures, workshops and practicals, and summative drop quizzes. I also used EVS in a workshop to brief students about peer marking of a full lab report. I will provide some information about the latter.

A workshop has been set up to brief students about the benefits of peer assessment, how to do it and how to write a good laboratory report. At the beginning of the workshop, I used EVS to ask students 4 questions about their initial thoughts when they heard the peer assessment. Then we went through the rationale and procedure of peer assessment, and viewed some feedback and evaluation results from the previous year’s cohort. At the end of the workshop, the same 4 questions were asked again using EVS. The more positive attitude towards peer assessment proved the effectiveness of the workshop.


EVS provides instant feedback of students’ opinion with complete anonymity in this case.

3) What were the benefits of your application of EVS for you?

We were able to make the workshop more suitable for students’ needs, and to check the results immediately afterwards. It also provided information for improvement during the peer marking process.

4) What do you think were the benefits of your application of EVS for your

students?

They can express their true views without revealing their identity. 5) Did you experience any drawbacks or problems with using EVS that may

prompt you to modify or develop the use outlined above?

No. 6) Any further information

It is advisable to tell students that you are not using their EVS device numbers (participant list) in this session.

School: Life Sciences

1) How did you use EVS in your teaching? 1. Regular drop-quizzes on materials taught both during the lecture and in previous

sessions.

2. As an ice-breaker to a subject and to ascertain existing understanding of new topic material.

3. Answering questions generated by students‟ peers in small groups

4. As a practice technique for answering MCQs.

2) Why did you use EVS in this context? For improving memory and recall of key facts; to understand how students‟ knowledge is developing and identify areas for additional teaching or revision support.

To overcome stigmas of responding in public to questions and remove fear of giving „wrong‟ answer.

To encourage students to reason what would be an incorrect answer to a question as well as the right answer; developing a deeper understanding of the material & help identify differences between potentially similar answers.

Overcoming responding stigma, provides ideal opportunity to explain incorrect answers;

students appreciate their own performance to prioritise revision subjects. 3)What were the benefits of your application of EVS for you?

Helped me to identify students’ knowledge to adjust future teaching and requirements for support.

4) What do you think were the benefits of your application of EVS for your students? Helped students appreciate their own performance

Communicated clearly the expected level of knowledge required for success in examinations.

Removed barrier of embarrassment, stigma etc when responding in public; improves student engagement with material.

Added an element of „fun‟ to lectures – novel technology and often fun and

amusement (!) when discussing answers.

Broke up lengthy lectures into shorter segments.


Maintaining student focus when discussing answers – they may become more interested in the range of responses than on understanding why their selection was correct/incorrect.

Students who have forgotten EVS handsets may become isolated from sessions.

Occasional difficulties in receiving votes.

More advanced uses of EVS (e.g. for summative assessments, attendance monitoring etc.) could be unappealing as further training may be required.

Time spent initially preparing slides can be significant.

If not used with care there is a risk of trivialising subjects into simple facts.

Reliance on an old laptop computer which is underpowered for EVS software.

School: School of Life Sciences ___________________________________________________________________


EVS was used for teaching mathematics and statistics to bioscientists, firstly in a diagnostic maths test, followed by workshops.


I was dealing with a large student cohort (about 180 students in one session). It was important to establish that the students were following the explanations and able to process the information correctly. EVS was the only means of checking that all of the students were engaging and had understood the material.


For the diagnostic test, I was also able to capture the responses for each individual student using the quiz feature of Turning Point. At the end of the test, the marks were easily downloaded into an Excel spreadsheet and published on Studynet using just the Student ID number. This also meant that I could allocate students to workshops that met their learning needs, based on how they scored in each section of the test. Without EVS I would have marked the test by scanning OMR sheets which would have taken a few hours of my time instead of just a few minutes.

4) What do you think were the benefits of your application of EVS for your students?

During the session students were able to confirm that they had the correct answer to the problem, so feedback was immediate. This provided the opportunity to run through explanations again if necessary.


No, but I have found that instead of just using multiple choice questions for the answers I can just as easily ask for the numeric answer to be input, adding a new dimension to using EVS.

School: School of Health and Emergency Professions Subject: Anatomy, Physiology and Pathology Programme/module: BSc (Hons.) Diagnostic Radiography and Imaging Anatomy, Physiology. Pathology and Image Appearances (Level 5) ___________________________________________________________________

1) How did you use EVS in your teaching? An electronic voting system (EVS) was used to enable conditional branching at key decision-making points during a series of imaging scenarios, where a conditional branch (in computer programming) is an instruction that directs the computer to an-other part of the program based on the results of a compare. In practice conditional branching refers to the application of the EVS to allow the student group to control the order of Power-Point™ slides in a presentation, based on the responses received to posed questions.

2) Why did you use EVS in this context? EVS was used to provide immediate, relevant feedback to students in a non-threatening manner while enhancing the student experience in a large cohort that is diverse both in terms of age and ethnicity. A project was used to investigate the feasibility and suitability of embedding conditional branching as an integral teaching tool for a large group throughout an undergraduate module.

The EVS answers allow the lecturer immediate feedback both on areas where understanding is not clear, and where there are misconceptions. This immediate feedback on teaching is extremely valuable. Having this feedback while still in the classroom allows difficulties to be addressed immediately. This allows the activity to build on correct understanding. It was recognized that we all learn from mistakes and the conditional branching allowed the lecturer to exploit this and allow students to see the consequences of errors in patient management, without causing actual harm.


One of strengths of conditional branching was found to be the ability to link the imaging scenarios closely to clinical practice. Relevant images were included at key decision making points and subsequent explanations were described to the group. Students were able to experience the effect of how an inappropriate decision could have a variety of different adverse outcomes for a patient.

During the teaching, the focus of the conditional branching was as a tool for helping the students to understand topics as a whole and as an opportunity to develop their critical thinking and decision making skills. Prior to each conditional branching session the students had had a number of teaching sessions on a given topic area.

It was clear that throughout the teaching the conditional branching was a successful method of increasing student engagement and classroom interactivity, despite the large size of the group. Although students had individual handsets there was continued evidence of student collaboration and social learning as students discussed options with small groups of their peers prior to voting.


The lack of right or wrong answers in the clinical scenarios is initially alien compared to the more common use of EVS and therefore required a level of reassurance. The main challenge was the length of time that the preparation of the scenarios took, a key factor being the need to reduce the complexity of real life cases into a simplified format that could be used and understood more easily as teaching examples.

School: Nursing Subject: Pressure Ulcer Prevention ___________________________________________________________________ 1) How did you use EVS in your teaching?

I participated in an external study day for qualified healthcare professionals. The study day was held on the premises of an NHS Trust in conjunction with a Nurse Specialist who was facilitating the day for staff. My involvement spanned four sessions; EVS was used in three. The first session was used to follow up a lecture given in the morning of the study day. In this case EVS was used to test the attendees’ recall of pressure ulcer categories. It was clear that despite their contention that they knew their categories well they didn’t, it was also clear from the responses that they had widely differing opinions. The implications of not knowing pressure ulcer categories is that prevention and treatment can be incorrect and Trust level reporting is inaccurate. The impact is therefore both clinically and financially relevant. The second session involved an exploration and discussion of the attendees’ knowledge of pressure ulcer equipment. Again, responses indicated deficits in knowledge that could then be discussed and corrected. The third use was to evaluate the day. This gave instant feedback and provided report data for both the host organization as well as ourselves as visiting lecturers. This was valuable in terms of evaluating the success of the day and planning future events.

2) Why did you use EVS in this context? An important part of the day was to gauge staff knowledge and provide information and education accordingly. The anonymity of EVS enabled staff to be honest about their knowledge. The immediacy of the reposes enabled us to adjust our content accordingly. 3) What were the benefits of your application of EVS for you?

Was able to adjust content and timings according to the need of the attendees.


The audience could see immediately where their own knowledge limitations were and also they could see how their differences of opinion could have a negative effect on patient care. The need to have consistency and agreement in context of this day was very important and was re-enforced by using EVS in this way.

5) Did you experience any drawbacks or problems with using EVS that may prompt you to modify or develop the use outlined above? No

School: Business School Subject: Business Mathematics Programme /module: Module - Operational Research Models ___________________________________________________________________


For seven weeks over a semester, an EVS drop quiz for summative feedback was given in the lecture. This best five quiz results were then used as part of their assessment and worth 10% of the overall module mark. Each quiz had six questions and had a mix of both qualitative and quantitative questions. The information being tested in the quiz was from the previous week’s lecture, to give students time to absorb the lecture information and prepare. MCQs were created from a bank of questions that had been written by previous students, on the same module, as part of their assignment.

A weekly league table was uploaded to StudyNet to sho how students were progressing. The results were shown by student registration number rather than name. Students achieving more than 70% were highlighted in green, from 40% - 70% in amber and be-low 40% in red. Comments were also attached next to the mark on the league table.


Students are familiar with the use of multiple choice questions to test their knowledge and find the technology easy to use. From personal experience of using EVS in previous modules, it was clear that students enjoyed using it for formative feedback. EVS also appears to engage all students and not just the good students. It is easy to develop an EVS test using TurningPoint software, as it provides an extra tab in Power-Point to insert an MCQ question and capture student responses.


To provide academic staff with immediate feedback regarding the misunderstanding of concepts and ideas being communicated

EVS encourages a high level of interaction within lecture sessions

Using EVS in lectures increased student attendance (average 84%)

Experience showed that students would put in extra effort even for a small proportion of marks, thereby encouraging a learner centred approach

It is possible to identify early in the semester which students need help and put in support mechanisms


EVS provides instant formative feedback to students, which may also be directly relevant to summative assessments

Students do not feel threatened or ‘stupid’ if they do not know the answer, as the responses are anonymous to their peers.

International students who do not normally participate in lectures are happy to submit their answers using EVS.



It took several weeks for students to collect their handsets, despite frequent reminders and 9-5 opening of the office. Handsets have to be distributed from this office as the handsets needed to be registered to a student to enable their scores to be used for summative assessment and to ensure return of handsets at the end of Semester B so that they can be used by level 4 students in 2011-12.

The report format provided by TurningPoint is not intuitive. Many different reports are provided and perhaps there is too many different reports to choose from?

Some student handsets did not appear to work, although it could be the students trying to get round having to do the quiz. If this happened, the students were requested to return the handset and get a new one. The downside of this was that students then had two handset numbers attached to their name and updating the marks for summative feedback took extra time.

If students forgot to bring the handsets, it was not possible to capture their marks.


EVS could easily be used for attendance monitoring and collecting student feedback for a module.

Subject: Management Skills for Adult Nursing Programme /module: Level 5 & 6 ___________________________________________________________________ 1)How did you use EVS in your teaching? Lead lecture - Negotiation and conflict resolution


Allows students to opt for preferred choice /actions anonymously. 3) What were the benefits of your application of EVS for you?

The benefits are that within a safe environment students can opt for the action that they consider they would take within a ‘conflict situation’

Example Scenario :

Mrs Smith is concerned that her husband is again Nil by Mouth. You explain that his swallowing reflex will be reassessed soon. Mrs Smith asks “What do you mean by soon?” and demands to speak to a senior nurse. You explain politely that the staff nurse is rather busy with another patient, but that you will get her to see Mrs Smith as soon as possible.(The staff nurse is in fact in the side room setting up a syringe driver for a patient who is on the Liverpool Care Pathway.) Mrs Smith is a less than happy with your answer. What do you do?

Choose one of the following options: a) Explain to Mrs Smith that Staff Nurse is very busy with another patient and that she will have to wait.! b) Knock on the side room door and ask to speak to Staff Nurse c) Wait for Staff Nurse to emerge from the side room,( you are fully aware that the patient is deteriorating)

d) Bleep the H.O concerned (vowing to met him at the door just in case he doesn’t know that Mr Smith is again N by M ) e) Explain to Mrs Smith that Staff Nurse is with a sick patient and that as soon as she is free you will get her to speak to Mrs Smith f) Ignore Mrs Smith, walk away and look very busy


Students actively debate how they would manage a potential conflict trigger, key skills:, verbal ,non verbal communication , privacy ,dignity , relatives and patients anxiety , documentation , actions taken in ‘real time’ not being perfect etc Less vocal students opinions are acknowledged, the anonymity value is evident with 100% participation (class size 120-150)


Awareness of value in engaging discussion means that the link to theoretical principles has to be consistently underlined.


The manner in which I use the EVS is successful (modular feedback). However this is not doubt due to the year group (3rd year), the subject area and the fact that EVS still has a novelty factor. I build in time to my session to allow discussion , each scenario builds upon preceding slides which cover the theory of negotiation and conflict resolution and thus the session is timed to last for 1 ½ hours . My advice for anyone using EVS for debating/ critical discussion purposes is that it is a very valuable resource which works in this format when time for discussion is factored in.

Subject: Electrical / Mechanical Engineering School: SET


I used it in Classroom/Lab for both Formative & Summative Assessments (Lab has no hardware so we borrow USB dongle from school)


To improve the students satisfaction rate providing them instant feedback Also create Q&A databank for my module 3) What were the benefits of your application of EVS for you?

Students gets solution with feedback and learn from each other’s mistake

I learn the level of problems with students for different topics in the module so that I

can prepare my further lectures accordingly

Less stressful to students as compared to StudyNet Quizzes as they cannot

participate in classroom environment.

In past I used StudyNet Quizzes but they are laborious and cannot be used for

Summative Assessments and also complicated systems


It improves the overall pass rate! Its interactive, we make fun by putting music and sound effects for different types of answers i.e OOPS, OHHHHHHHHH, YAAAAAAAAA, WOW.. downloaded from Google and also recorded from the classroom Majority of students participate in Formative Test and keep record of it as I provide them the pdf file and they takeaway overall result that they can reflect when they prepare for true Summative assessment. 5) Did you experience any drawbacks or problems with using EVS that may prompt


Some devices do not work in classroom, hence we must have some spare set ready for replacement (i.e. in Assessed Sessions)

School: Law Programme/module: Graduate Diploma Law ___________________________________________________________________

1) How did you use EVS in your teaching? To mediate debates on contentious ethical issues e.g. abortion, where views tend to be very diametrically opposed and emotive. Group size of about 40. Students were given the debate proposition before the class and asked to prepare a statement with a supporting rationale stating their position. In class all students vote on the two opposing views presented. Then all students have an opportunity to state their view. Voting proceeds with students polling their views as the debate develops 2) Why did you use EVS in this context?

To engage students in the subject matter

To promote interactivity and discussion

To help the students develop skills in construction an argument/articulating a case/justifying a view point/countering an argument

To promote research skills (preparation for presentation)

To promote reflective skills (reflection on what they hear from other people, helping them to respond to new information as it emerges, develop their arguments and ideas).

What were the benefits of your application of EVS for you?

EVS allows efficient debate management because it minimises physical movement in the class i.e. makes better use of classroom space and promotes a more collegiate environment

The anonymity meant that less vocal/confident students have a voice

It promotes inclusivity by allowing everyone to engage in the debate fully 3) What do you think were the benefits of your application of EVS for your students?

Promotes a whole range of skills (see 2.) 4) Did you experience any drawbacks or problems with using EVS that may prompt you to modify or develop the use outlined above?

No.

School: SET Subject: Engineering ___________________________________________________________________


For the first year programming module. The quizzes were used at the end of the lectures as part of their assessment (formative).


I hoped that this would encourage students to attend lectures and thereby benefit from it.


It was a good means to encourage students to pay attention to the lectures.


Recapitulate lectures and assessment being in small units.



Students attendance did not necessarily improve. They found excuses for not attending lectures in relation to EVS.

- EVS could not be well used for all lectures. However, use of EVS gave me a nice idea to formulate a short tutorial at the end of the session which made students think, workout results and give their answers.

- some teething problems with EVS because of personal lack of experience.


SET has had mixed experience with the use of EVS. The School have noted these problems and will be looking at addressing them better for the next academic year.

Documents

School: Computer Science Subject: EVS and Peer …jiscdesignstudio.pbworks.com/w/file/fetch/66200924/ITEAM EVS case... · School: Computer Science Subject: EVS and Peer Assessment