Beef up your backchat:

Using audience response systems

to assess student learning

Elizabeth Yates,

Liaison/Scholarly Communication Librarian,

Brock University

WILU 2014 May 22, 2014

Imagine IL classes like this!

Image source: Giphy http://gph.is/XJUzpS

Instead of this

Image source: Giphy http://gph.is/XJYDqc

Learning outcomes

Participants will recall:

• characteristics of audience response

systems and how they are used in PSE for

formative assessment

• ”best practice” strategies for incorporating

audience response systems into library

instruction sessions

Poll time!

Have you used audience response systems in

library instruction?

Hands up OR

Please go to:

www.govote.at

and enter 35 51 92 to vote

Audience response systems

• Think “clickers without the hardware”

• Instructors ask questions and students

respond using web-based software which

collects and displays their answers

• Can be used with desktop computers &

mobile devices

• Some allow texting

Also called:

(open-ended OR student OR classroom OR personal)

AND response systems) OR web-based polling OR

audience response technology … etc.

• “… the unofficial channel for the class, consisting of

interactions among the audience, or perhaps with

those outside the class. (Aagard, Bowen & Olesova,

2010).

• “… the ongoing, co-constructed, meta-content

discussion that can accompany live demonstrations

of nearly any type.” (Higdon, Reyerson & McFadden,

2011)

What’s a backchannel?

Formative assessment:

• Provides immediate, ongoing feedback

• Allows instructors to improve their

teaching

• Allows students to identify strengths

and weaknesses and target areas that

need work

(Carnegie Mellon Eberly Centre, 2013)

System $ Platform Question type User limit

Downloadresults

Poll Everywhere

FreePaid plans for larger audiencesTexting charge may apply

BrowserTextingTwitter

Multiple choiceShort answer

40 Yes

Mentimeter Free Browser Multiple Choice None Only with premium

Socrative FreePaid plans for larger audiences

Browser or app

Single answer or quizzes;Multiple choiceTrue/FalseShort answer

50 Yes

Top ARS tools

Some Blooming* examples

Knowledge = remembering:

• Start session by asking students to recall

material covered previously

• Mid-lesson check-in

• End session by asking students to recall

info covered that day

*based on Bloom’s taxonomy of learning

Comprehension

Understanding facts:

• Use text polls to discuss a question e.g. is this a credible source? (small class)

• Use multiple choice to classify e.g. what are acceptable scholarly info sources

• What’s still unclear?

Application

• Ask students to discover features of a

database and share via poll

• Ask text-based questions > students can

collaborate and write a paragraph and

then post via poll; students can see &

discuss each other’s work

Analysis

• Ask students to identify database search

filters and answer via multiple-choice or

share findings via text-based answer

• Compare two websites and vote for most

credible source

Synthesis

• Small class, text answers:

–Ask students individually to create

search strategies with keywords, search

operators & post via ARS

–As group, evaluate search strategies

Evaluate

• Evaluation for instruction sessions or

student self-evaluation > multiple choice

or text-based

• Quiz comparing info resources e.g. Google

Scholar vs. SuperSearch

A little different:

• Free for profs; students pay

$20/semester or $38/5 years

• create questions or discussions

• more question types eg matching,

sorting, word answer

• assignments and quizzes

• includes gradebook

• some LMS integration

Top Hat interface

Question

menu

We’re in the

home stretch

Any burning questions?

Interactive

Fu

n Engages students

Multiple question types

Anonymous

Imm

ed

iate

Increases

focus

Boosts participation

Incorporates

technology

Students

dislike

monitoring

Tricky to craft questions

Best practices

• Be clear: explain how the tool works, why

you are using it, what they need to do

(Aargard)

• Ensure it is used constructively

• Ensure everyone has access

• Align ARS with instructional design > don’t

just throw it in for “fun” (Dennis)

Think+pair+share

1. Think of how you could use ARS in

your instruction sessions (1 min)

2. Pair up (1 min)

3. Share your ideas!

• Must be really comfortable getting students on the

system – text or verbal

• Where are the students? Use ARS to help understand the

audience

• Icebreakers

• Interactivity in a really big class

• all have a non electronic issue

• Inappropropriate or silly answers – strategy: spin this in

your favour

Think, pair & share feedback

Question design

“Ideal questions for ARS are challenging

enough that students need to carefully

select their response, but also accessible

enough that a student can select a

response within a few minutes.”

-- Abate, Gomes & Linton (2011)

Question design, part 2

Effective questions:

• Address a specific learning goal

• Uncover misconceptions

• Explore ideas in a new context

• Elicit a wide range of responses

--Kay & LeSage, 2009

Tips, tricks & next steps

• Ask questions at 20-min intervals

• Be sure to hide answers until you’re ready

for whole class to view

• Test, test & triple-test

Watch for: ARS with social media, multimedia,

gamification e.g. Course Peer (classroom

response > classroom engagement)

Summing up

• ARS are great for student engagement, active learning, formative assessment

–Can align with Bloom’s taxonomy

• Care needed to craft questions

• Important to clearly define use/rules

• Mixed evidence on learning outcomes

Questions or comments? eyates@brocku.ca

Aagard, H., Bowen, K., & Olesova, L. (2010). Hotseat: Opening the backchannel in large

lectures. Educause Quarterly, 33(3), 2. Retrieved fromhttp://www.educause.edu/library/EQM1031

Abate, L. E., Gomes, A., & Linton, A. (2011). Engaging students in active Learning: use of a blog and

audience response system. Medical Reference Services Quarterly, 30(1), 12–18.

doi:10.1080/02763869.2011.540206

Bazylak, J., McCahan, S., Weiss, P. E., & Anderson, P. (2013). Take Out Your Cell Phones-Class is Starting–

Revisited. Proceedings of the Canadian Engineering Education Association. Retrieved from

http://library.queensu.ca/ojs/index.php/PCEEA/article/view/4803

Carnegie Mellon Eberly Centre. (2013). What is the difference between formative and summative

assessment? Retrieved April 22, 2013 from:

http://www.cmu.edu/teaching/assessment/basics/formative-summative.html

Chan, E. K., & Knight, L. A. (2010). Clicking with your audience. Communications in Information Literacy,

4(2), 192–201.

Connor, E. (2011). Using Cases and Clickers in Library Instruction: Designed for Science

Undergraduates. Science & Technology Libraries, 30(3), 244–253. doi:10.1080/0194262X.2011.592787

Deleo, P. A., Eichenholtz, S., & Sosin, A. A. (2009). Bridging the information literacy gap with clickers.

Journal of Academic Librarianship, 35(5), 438–444.

References

ReferencesDennis, M. R., Murphey, R. M., & Rogers, K. (2011). Assessing information literacy comprehension in first-year students. Practical Academic Librarianship: The

International Journal of the SLA, 1(1), 1–15.

EDUCASE Learning Inititiave. (2011). 7 things you should know about opne-ended response systems. ELI 7 things you should know. Retrieved on April 4, 2012

from: http://www.educause.edu/library/resources/7-things-you-should-know-about-open-ended-response-systems

Gewirtz, S. (2012). Make your library instruction interactive with Poll Everywhere: an alternative to audience response systems. College & Research Libraries

News, 73(7), 400–403. Retrieved fromhttp://crlnews.highwire.org/content/73/7/400.full

Eva, N., & Nicholson, H. (2011). DO get ytchnical! using technology in library instruction WILU 2011, Regina, SK. Partnership: The Canadian Journal of Library &

Information Practice & Research, 6(2), 1–9.

Higdon, J., Reyerson, K. & McFadden, C. (2011). Twitter, Wordle, and ChimeIn as student response pedagogies (EDUCAUSE Quarterly) | EDUCAUSE.edu. (n.d.).

Retrieved April 17, 2013, from http://www.educause.edu/ero/article/twitter-wordle-and-chimein-student-response-pedagogies

Hoppenfeld, J. (2012). Keeping students engaged with web-based polling in the library instruction session. Library Hi Tech, 30(2), 235–252. Retrieved

from http://www.emeraldinsight.com/journals.htm?articleid=17032265&show=abstract

Hoyt, A., McNulty, J. A., Gruener, G., Chandrasekhar, A., Espiritu, B., Ensminger, D., … Naheedy, R. (2010). An audience response system may influence student

performance on anatomy examination questions. Anatomical Sciences Education, 3(6), 295–299.

Kay, R., & LeSage, A. (2009). Examining the benefits and challenges of using audience response systems: A review of the literature. COMPUTERS &

EDUCATION,53(3), 819–827.

Lin, J., & Rivera-Sanchez, M. (2012). Testing the information technology continuance model on a mandatory SMS-based student response system.

Communication Education, 61(2), 89–110.

Liu, F. C., Gettig, J. P., & Fjortoft, N. (2010). Impact of a student response system on short-and long-term learning in a drug literature evaluation course.

American Journal of Pharmaceutical Education, 74(1). Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2829154/

Trew, J. L., & Nelsen, J. L. (2012). Getting the most out of audience response systems: predicting student reactions. Learning, Media and Technology,37(4), 379–

394. doi:10.1080/17439884.2011.621957