Online Assessment Presentation

Introduction to Online Assessment

By the end of this presentation you should be able to:

•  Define assessment

•  Describe the importance of assessment

•  Describe differences between online versus traditional assessment

•  List online assessment implementation strategies

•  Know and describe the four categories of assessments

•  Know how to write learning objectives

•  Write measurable learning objectives

•  Know and describe the assessment triangle

•  Describe academic integrity issues with online assessment and ways to prevent this

What is Assessment?

•  Assessment is the general term that includes a full range of procedures used to gain information about student learning and the formation of value judgments concerning learning progress.

•  It is the way to measure student/learner growth, understanding, and performance in a course, on a particular content standard, or across multiple content standards.

Online versus Traditional Assessment

“Despite its popularity, online learning is not without its challenges. As instructors are finding their way in the online environment and paying more attention to good course design and delivery, they are discovering that traditional forms of assessment of student work-such as test and quizzes-that serve them well in the face- to-face classroom may not work quite as well online” (Palloff and Pratt, 2009).

Online versus Traditional Assessment, continued

•  Online and traditional assessments can be similar, but implementing them online, especially if for the first time, requires ample planning.

•  Ideally the assessment process informs the teacher and the learner about learner progress and at the same time, contributes to the learning process. In theory, good assessment: •  Occurs during a learning activity •  Measures meaningful learning outcomes •  Does so in a fair, reliable, accurate way •  Is easy to administer, score, and interpret •  Informs the teacher about student performance and how they are

interpreting course experiences •  Results in meaningful feedback to the learner •  Is itself a learning experience

Online Versus Traditional Assessment, continued

Traditional Online

Exams Discussions

Quizzes E-Portfolios

Peer-Assessments

Self-Assessments

Rubrics

Quizzes

Collaborative Activities

Wikis

Blogs

Can you think of any more examples?

Implementing Online Assessments

•  For online assessments to be implemented effectively, the instructor must have a solid understanding of how assessment fits into the scheme of course development (Palloff and Pratt, 2009).

•  Instructors need to understand the learning outcomes (i.e. what students should be able to know or do by the end of the lesson), their importance in the learning process, and how to achieve them (Palloff and Pratt, 2009).

Implementing Online Assessments, cont.

•  Online assessment should be ongoing and both graded and non-graded.

•  Online assessment should involve student collaboration.

•  Online assessment should have a plan for technical difficulties.

Online Assessment Strategies

We will go into detail on some of these strategies during our synchronous lesson.

E-Portfolios Essays

Quizzes Peer-Assessments

Discussions Concept Maps

And more!

What’s Next?

Now that you have learned about online assessment, you will be introduced to assessment categories.

Assessment Categories: Formative and Summative

Assessments can be categorized into either formative or summative depending on their overall purpose.

Formative Assessment

•  The goal of formative assessment is to gather feedback that can be used by the instructor and the students to guide improvements in the ongoing teaching and learning context. These are low stakes assessments for students and instructors. Overall, they should: •  Provide the student with feedback to improve student learning •  Provide feedback to students to assist in correcting their errors

•  Examples: •  Asking students to submit one or two sentences identifying the

main point of a lecture •  Have students submit an outline for a paper •  Early course evaluations

Formative Assessment, continued

•  Research shows a positive relationship between formative assessment and student motivation and achievement (Brookhart 1997, 2008).

•  “Assessment-elicited evidence of student’s status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning tactics” (Popham 2008).

Summative Assessment

•  The goal of summative assessment is to measure the level of success or proficiency that has been obtained at the end of an instructional unit, by comparing it against some standard or benchmark.

•  Typically occur at the end of student learning to identify student and/or course achievement.

•  Examples: •  Assigning a grade to a final exam •  Critique of a Senior recital •  University Faculty Course Evaluations •  The outcome of a summative assessment can be used formatively, however, when

students or faculty take the results and use them to guide their efforts and activities in subsequent courses they become summative

Assessment Categories: Objective and Subjective

Assessments are also categorized based on nature of products that result from student performance.

Objective Assessment

•  An objective assessment is one that needs no professional judgment to score correctly (although interpretation of the scores requires professional judgment). There is a right and wrong answer.

•  Example: •  Multiple-choice test

Subjective Assessment

•  Subjective assessments yield many possible answers of varying quality and require professional judgment to score (Suskie, 2004, p. 99).

•  Examples: •  Essay

•  Written Assignments

What’s Next?

Now that you’ve learned some different categories of assessments, let’s review learning objectives.

Learning Objectives

•  Assessments must align with lesson/instructional objectives.

•  What are learning objectives? •  Statements written in specific and measurable terms that

describe what the learner will know or be able to do as a result of engaging in a learning activity. •  Can be called learning objectives, outcomes, terminal

objectives, performance objectives, etc.

•  Example: •  Students will list three characteristics that make the family

medicine physician distinctive from other specialists in the health care system.

Learning Objectives, continued

•  Purpose •  By knowing where you intend to go, you increase the chances of you

and the learner ending up there!

•  Learning objectives… •  Specify learning outcomes •  The skills, knowledge, and attitudes to be taught

•  Guides the teacher relative to the planning of instruction and evaluation of student achievement

•  Guides the learner; helps him/her focus and set priorities •  Communicates with learner •  Determines the instructional strategy •  Allows for analysis in terms of the levels of teaching and learning •  Establishes criteria for evaluating performance


•  One of the themes of Gagné’s theory is distinguishing the types of outcomes that learning has. Gagné describes five categories of human performance established by learning. These provide the foundation for describing how the conditions of learning apply to each category.

•  Gagne’s learning domains:

•  Affective

•  Attitudes, mental states that influence the choices of personal actions.

•  Psychomotor

•  Motor skills, executing movement in a number of organized motor acts such as playing sports or driving a car

•  Cognitive

•  Having certain techniques of thinking, ways of analyzing problems, and having approaches to solving problems

•  Verbal

•  Being able to state ideas, “knowing that,” or having declarative knowledge

•  Intellectual

•  “knowing how,” or having procedural knowledge


•  Four Parts (ABCD) •  Audience •  Who are your learners?

•  Behavior •  What you expect learners to be able to do. Overt, observable

behavior. •  Condition •  How? Under what circumstances or context will the learning

occur? What will students be given or already be expected to know to accomplish the learning?

•  Degree •  How much will be accomplished, and to what level?

Learning Objective Example (ABCD)

Given a series of 10 previously unstudied paragraphs, the learner will identify the topic sentence in each paragraph correctly 9 out of 10 times.

Who is the audience? What is the behavior? Condition? Degree?

Audience


Behavior


Condition


Degree


Effective Objectives Are…

•  Consistent with the goals of the curriculum

•  Clearly stated

•  Clearly measurable

•  Appropriate for the level of the learner

•  Worthy (important)

Remember…

•  State objectives in terms of student performance or end of instruction.

•  Use action verbs. •  Click here for list of Gagne verbs:

https://deekayen.net/gagne-verbs

•  State each objective as an intended learning outcome. •  Example: •  At the end of the lesson the student will be able to write a

measurable learning objective.

More Examples: Physical Education

•  Affective: Student will be able to (SWBAT) listen to directions while sitting still and not touching any equipment with no warnings.

•  Psychomotor- SWBAT demonstrate how to kick a ball to a target 5/8 times.

•  Cognitive: SWBAT demonstrate knowledge of jumping rope cues (over, hit, jump) by reciting these aloud during drills 100% of the time.

•  Verbal Information: SWBAT state the rules of a basketball game.

•  Intellectual Skills: After participating in both vigorous activity and walking, SWBAT identify that the heart beats faster after moving vigorously.

Now, You Try It!

•  Go to discussion board 1. •  Link to discussion board 1 is in the

menu at the top of the webpage.

•  Think of any content area.

•  Write two examples of learning objectives (in two different learning domains) Remember to include each part (ABCD).

•  Return to this presentation once you write your objectives.

•  Feedback will be given by our group prior to synchronous lesson.

Tying it All Together

Now that you are familiar with online assessments, categories of assessments, and learning objectives, let’s see how it all fits together.

The Assessment Triangle

•  The assessment triangle consists of objectives, learning activities (listening to a lecture, engaging in a group discussion, making rounds with a clinician, etc.), and assessment.

•  If these three components are congruent, then teaching and learning is enhanced.

What Does This Mean?

•  Assessments should provide us, the instructors, and the students with evidence of how well the students have learned what we intend them to learn.

•  What we want students to learn and be able to do should guide the choice and design of the assessment.

•  There are two major reasons for aligning assessments with learning objectives.

•  First, alignment increases the probability that we will provide students with the opportunities to learn and practice the knowledge and skills that will be required on the various assessments we design.

•  Second, when assessments and objectives are aligned, “good grades” are more likely to translate into “good learning”.

•  When objectives and assessments are misaligned, many students will focus their efforts on activities that will lead to good grades on assessments, rather than focusing their efforts on learning what we believe is important.

What Does This Mean, continued

•  There are many different types of activities that can be used to assess students’ proficiency on a given learning objective, and the same activity can be used to assess different objectives.

•  To ensure more accurate assessment of student proficiencies, it is recommended that you use different kinds of activities so that students have multiple ways to practice and demonstrate their knowledge and skills.

•  We will learn about different activities during the synchronous piece of this lesson.

•  When deciding on what kind of assessment activities to use, it is helpful to keep in mind the following questions:

•  What will the student’s work on the activity (multiple choice answers, essays, project, presentation, etc.) tell me about their level of competence on the targeted learning objectives?

•  How will my assessment of their work help guide students’ practice and improve the quality of their work?

•  How will the assessment outcomes for the class guide my teaching practice?

What’s Next?

Now that you are familiar with online assessment, let’s review academic integrity in the online environment.

Academic Integrity

“Academic integrity is the pursuit of scholarly activity in an open, honest, and responsible manner.”

Penn State University

http://academicintegrity.cas.psu.edu/Statements.html

Academic Integrity Affects…

•  Reputation of the University/College

•  Reputation of the school’s alumni

•  Reputation of degrees awarded

•  The quality of education given

•  How emplyers view graduates

•  “Online students have less commitment to the integrity of distance programs than to traditional programs” (Bell & Whaley, 1991).

•  We have strategies to change this!

•  “The more distant people feel, the more likely they are to cheat” (George & Carlson, 1999).

•  So, how do we make students feel close?

Academic Dishonesty

•  Unfair advantage

•  Un-permitted collaboration

•  Student receiving help from another source

•  Plagiarism

•  Not the student’s own work

•  Reuse of student’s own work

Academic Dishonesty Statistics

•  75% of college students reported cheating during their enrollment (Dick, et al., 2003).

•  95% of students who reported cheating said they were never caught (Bushweller, 1999).

•  50% did not believe that cheating was necessarily wrong (Kleiner & lord, 1999).

•  Cheating increases with the age of the student up to 25 (Cizek, 1999).

Faculty and Student Definitions Differ

•  Students see it as: •  Helping each other

•  If most students are doing it or have a copy of the test, they think it is OK

•  If it is not for a grade, it is not cheating

•  “Cut and Paste” is not cheating

•  Reusing their own work is not cheating

Reasons for Academic Dishonesty

•  Lack of research or citation skills

•  Lack of understanding of the assignment, plagiarism, copyright laws

•  Did not understand it was cheating

•  Old tests are available

•  Easy to do/laziness

•  Instructor did little or nothing to prevent cheating and/or was reluctant to take action

•  Scholarship competition

•  Faculty do not recognize it

Experts Agree…

If an online course and assessments are well designed, cheating will not be a significant problem.

Do you agree?

Academic Integrity Approaches

•  Virtues Approach

•  Policing Approach

•  Preventative Approach

(Hinman, 2000)

Virtues Approach

•  Develop students who do not want to cheat. •  Use an honor code

•  Communicate the school policy, provide a link

•  Have students sign policies

•  Give an assignment on ethical decision-making

•  Give an assignment on plagiarism

Policing Approach

•  Catch and punish those who cheat. •  Examine course stats, content areas, day and time of

login, length of login, use of communication tools, etc.

•  Distrust “technical problems” excuses

•  Be prepared to retest with a different version

•  Be prepared for alternate assignments

Preventative Approach

•  Eliminate or reduce cheating opportunities. •  Clarify academic dishonesty and plagiarism

•  Change test items each year

•  Use detection software

•  Use different versions of the test

•  Reduce pressure to cheat by being flexible with deadlines

•  Reduce pressure to cheat by focusing on learning and not grades

Preventative Approach, continued

•  Use short assessments throughout the course

•  Small, sequential, student-centered personal responses

•  Have a high level of instructor/student interaction

•  Orient assessments towards higher order thinking skills

•  Design writing assessments with specific goals and instructions

•  Use personal real-life situations known only to the student and avoid trivial assignments


•  Reduce opportunities to engage in academic dishonesty. •  Use assignments that are learner-centered

•  Promote the use of discussion boards

•  Use progressively developed assignments

•  Use reflective assignments

•  Use fair assessments

•  Allow open-book for online exams, quizzes, tests (McCabe & Pavela, Change 2000)


•  Randomize questions or distractors

•  Use pools of questions or exam generator

•  Use timed or forced completion

•  Put one question per page

•  Prevent copy/print of online material

Your turn!

•  You will read a short article that deals with online academic dishonesty and write a short reaction. Please see discussion board 3 for details. •  A link to discussion board 3 can be found on in the

menu bar at the top of this webpage.

•  Return to this presentation once you have completed the assignment.

Summary •  You have been introduced to: •  Assessment •  Online Assessment versus Traditional Assessment •  Implementation Strategies (more will be covered) •  Learning Objectives •  Categories of Online Assessments •  The Assessment Triangle •  Academic Integrity in Online Environments

•  You are now ready to take the formative quiz over the content presented in this presentation. •  The quiz is located on the “quiz” link in the website menu bar.

•  Feel free to view this presentation again before taking the quiz.

References

•  Bell, J.B. and Whaley, B. (1991). Cheating and deception. New York: Translation Publishing.

•  Bushweller, K. (1999). Generation of Cheaters. The American School Board Journal. www.asbj.com/199904/0499coverstory.html

•  George, J. and Carlson, J. (1999). Group support systems and deceptive commuinication. 32nd Hawaii Intl. Conf. on Systems Sciences, 1038.

•  Hinman, L.M. (2000). Academic Integrity and the World Wide Web. http://ethics.acusd.edu/presentations/cai2000/index_fi les/frame.htm

•  Kleiner, C. and Lord, M. (1999). The cheating game: Cross-national exploration of business students’ attitudes, perceptions, and tendencies toward academic dishonesty. Journal of Education for Business. 74(4), 38-42.

•  www.adea.org/publications/library/.../bestpracticesforteachingonline.pdf

•  Resources from Dr. Kinuthia’s Fall 2014 Online Assessment Course