Diverse Models and Challenges: Assessment for Learning in Online Settings

SAS Center for Educational and Instructional Technology Research (CEITR)

Presented By: Mansureh Kebritchi, Ph.D. & Kate Andrews, Ph.D.

Context

Online education changes all components of teaching and learning including assessment.

Many empirical studies were conducted in examining effective assessment techniques.

Problem

Lack of literature reviews to provide an integrative report on effective assessment techniques and existing challenges in assessment of learners

Problem

Often online educators have to go through daunting task of sifting the increasingly expanding literature to identify these strategies and issues.

Purpose

Presents the results of a critical literature review on models, best practices, and challenges of assessment of learners in higher education online courses.

Method

Cooper’s (1988) procedure was used: Formulate the problem Collect data Evaluate the appropriateness of the data Analyze and interpret relevant data and Organize and present the results

Empirical studies and literature reviews conducted for online courses between 2004 and 2014

The keywords: “Online assessment”, “Assessment models”, “Assessment Techniques”

Searching sources:oOnline journals oAcademic search engines and databases (EBSCO HOST, ERIC, Google Scholar)oRelated conference websites

Method

Method: Data Analysis and Evaluation

26 articles were found: 12 focused on issues related to assessment models 8 discussed assessment techniques 4 focused on assessment challenges The remaining articles that did not discuss specific

issues in the above three aspects were excluded from this review

Findings: Models

Using mix of various assessment models:

Dynamic assessment Formative assessment Summative assessment Diagnostic assessment

Findings: Dynamic AssessmentRecommended Model

Has four components: Individual differences Assessing the very processes of systematic

thinking, learning, and problem solving not the product

Assessing learning potential, educability, and the zone of proximal development

Mediation(Haywood, 2012)

Findings: Dynamic AssessmentIndividual Differences

Identify each individual’s unique qualities, strengths, weaknesses

Discover individually useful methods for maximizing individuals’ learning effectiveness.

Findings: Dynamic AssessmentAssessing the Process

“an important way to learn about the characteristics of events that are not directly observable is to change those events and then to observe the effects of the changes”.(Haywood, 2012, p.220)

Direct observation of learning Comparison of effectiveness of help or instruction Use of transfer paradigm

Findings: Dynamic AssessmentAssessing Educability

What is versus what could be Learners with larger zone of proximal development (ZPD)

will do much better in school Cognitive development as a dynamic phenomenon

(Vygotsky, 1934,1986) Need for assistance to be provided for learners in the

assessment of their ZPD

Findings: Dynamic AssessmentMediation

A very special kind of interaction between a learner/performer and a more cognitively competent person (e.g., mentee and mentor)

Goals: To discover obstacles to effective performance, To neutralize those obstacles whenever possible, To assess the effects of “obstacle removal” on subsequent

performance on the same or similar tasks.

Findings: Dynamic Assessment

Practical Framework: Assess Feedback Intervention Review

(Lauchlan, 2013)

Findings: Techniques

Techniques including: Use of proctored tests Maintain continuous communication with learners Maintain dynamic interaction Modify traditional tools for online assessment Use of alternative assessments such as: authentic,

performance, portfolio assessment(Gaytan, 2005)

Findings: Challenges Cultural diversity o attitudes toward and approaches to learningo failure to have developed a strong cultural

identity; often associated with transculturality Knowledge gapo Inadequate store of information Ineffective motivation components o Emphasis on task-extrinsic sources of

satisfaction and consequent failure to have developed a task-intrinsic motivational orientation (Haywood,2012)

Conclusion This literature review identified assessment

models, techniques, and challenges in online settings

Online educators and instructional designs, may consider the identified issues to improve assessment of online courses

­ Black, P., & Wiliam, D. (2009). Developing the Theory of Formative Assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ829749&site=ehost-live

­ Clark, I. (2012) Formative assessment: Assessment is for self-regulated learning. Educational Psychology Review, 24(2), 205-249.

­ Crossouard, B., & Pryor, J. (2012). How theory matters: Formative assessment theory and practices and their different relations to education. Studies in Philosophy and Education, 31(3), 251-263.

­ Dodge, J. (2013). What are formative assessments and why should we use them? Tips for using formative assessments to help you differentiate instruction and improve student achievement. Scholastic Inc. Retrieved from http://www.scholastic.com/teachers/article/what-are-formative-assessments-and-why-should-we-use-them

References

­ Feuerstein, R., Feuerstein, R. S., Falik, L. H., & Rand, Y. (2004).The dynamic assessment of cognitive modifiability. Jerusalem: International Center for the Enhancement of Learning Potential. DOI: 10.1177/0734282906290365

­ Fullan, M. & Donnelly, K. (2013). Alive in the swamp, assessing digital innovations in education. Retrieved from http://www.michaelfullan.ca/wp-content/uploads/2013/06/13_Alive_in_the_Swamp.pdf

­ Gaytan, J., & McEwen, B.C. (2007). Effective online instructional and assessment strategies. American Journal of Distance Education, 21 (3), doi: 10.1080/08923640701341653

­ Griffith, D. & Mellor, M. (2014). Making accountability meaningful. Retrieved from http://www.ascd.org/publications/newsletters/policy-priorities/vol20/num02/Making-Accountability-Meaningful.aspx

­ Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112.

­ Haywood, H. C. (2012). Dynamic assessment: A history of fundamental ideas. Journal of Cognitive Education and Psychology,11(3), 217-229.

References

­ Herold, B. (2014). Testing digital tools to improve formative assessments. Retrieved from http://www.edweek.org/ew/articles/2014/03/13/25personalized.h33.html

­ Lai., E.R. & Viering, M. (2012). Assessing 21st century skills: integrating research findings . Retrieved from http://researchnetwork.pearson.com/wp-content/uploads/Assessing_21st_Century_Skills_NCME.pdf

­ Morrissette, J. (2011). Formative assessment: Revisiting the territory from the point of view of teachers. McGill Journal of Education, 46(2), 247-265.

­ Niemivirta, M. (2006). Assessing Motivation and Self-Regulation in Learning within a Predictive Design: Incorporating Systematic Elements of Change. Educational Psychology Review, 18(3), 255–259. Retrieved from 10.1007/s10648-006-9020-5

­ Offerdahl, E. G., & Tomanek, D. (2011). Changes in instructors’ assessment thinking related to experimentation with new strategies. Assessment & Evaluation in Higher Education, 36(7), 781-795.

­ Price, M., Carroll, J., & O'Donovan, B. (2011). If I was going there I wouldn’t start from here: A critical commentary on current assessment practice. Assessment & Evaluation in Higher Education, 36(4), 479-492.

References

­ Randall, L., & Zundel, P. (2012). Students’ perceptions of the effectiveness of assessment feedback as a learning tool in an introductory problem-solving course. Canadian Journal for the Scholarship of Teaching and Learning, 3(1), 1-17.

­ Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189.

­ Stecher, B. & Hamilton, L.(2014). How to assess 21st century competencies 12 key lessons. Retrieved fromhttp://blogs.edweek.org/edweek/global_learning/2014/02/how_to_assess_21st_century_competencies_skills_12_key_lessons.html

­ Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3-14.

­ Wolsey, T. (2008). Efficacy of instructor feedback on written work in an online program. International Journal on ELearning, 7(2), 311–329.

References

Thank you! Any Questions?

Contact Information:

Mansureh Kebritchi: Mansureh.kebritchi@phoenix.edu

Kate Andrews: Kateandrewss@email.phoenix.edu

SAS Center for Educational and Instructional Technology Research (CEITR)EducationalTechnology@Phoenix.edu