Implementing the Scholarship of Teaching and Learning (SOTL)

We will:-Define SOTL-Identify types of SOTL projects-Examine steps of SOLT projects-Identify resources to develop SOTL

projects-Evaluate potential SOTL projects

Scott Cottrell, Ed.D.Assistant Professorscottrell@hsc.wvu.edu304-293-0410

What is the SOTL?

The scholarship of teaching and learning is an investigation of a problem relating to teaching or learning.

The study of the problem is realized through methods appropriate to disciplinary epistemologies (e.g., quasi-experimental design, case studies), and the results are communicated to peers for critical reflection (e.g., journals, conference presentations, online delivery).

What is Considered Scholarship?

www.aamc.org/mededportal

PowerPoint Presentations

Web Sites

Virtual Patients

Lab Guides

CD-ROMs

Cases (PBL, OSCE, Team Learning, CPC, etc)

Faculty Development Resources

Why is SOTL important?

SOTL breaks down barriers between research and teaching.

SOTL yields an opportunity for teachers to frame their existing responsibilities as an investigation.

SOTL allows teachers to present educational reforms and innovations to peers for critical reflection.

SOTL helps satisfy expectations of research in an academic institution.

(AAMC- Advancing Educators and Education: Defining the Components and Evidence of Educational Scholarship)

SOTL focuses on student learning and its improvement.

What is the purpose of research?

• Describe- pharmacy students’ academic success across demographics

• Predict- e.g., residents’ professional behavior in future practice

• Improve- e.g., effectiveness of intervention to improve teaching skills of nursing faculty

• Explain- subsumes all three

Data AnalysisResearch Questions

Research DesignData Collection

Theoretical Framework

ScholarshipSeeks to Explain

Assessment Seeks to Describe

Theoretical Development

Distinction Between SOTL and Assessment

Cottrell, S.A. (2006). A Matter of Explanation: Assessment, Scholarship of Teaching and their Disconnect with Theoretical Development. Medical Teacher. 28(44), 305-08.

What is a theory?

• A theory is an explanation of events in terms of the structures and processes that are presumed to underlie them.

• Theory consists of constructs that specify how the constructs are related.

What is a construct?

• A construct is a type of concept used to describe a structure of process that is hypothesized to underlie particular observable phenomenon.

• e.g., residents’ professionalism, teaching skills, motivation, leadership

Important Caveat

Don’t confuse theory with principles!!

For example, adult learning principles are not a theory.

Most researchers use theory much like a drunkard uses a light post – more for support than for illumination.

Example of Confusing Principles with Theory

Educators should encourage students to exercise an innate ability to self-assess and

evaluate their ability to perform a task.

• This belief has set the stage for several studies that examine students’ self-assessment skills in a variety of learning contexts.

Assumption is not Supported with Theoretical Support

“How do they know what they don’t know”

• Table 1. Table illustrating what an individual learner does and does not do well, and whether the learner is aware or unaware of abilities.

What we do well What we don't do well

Know it A B

Don't know it C D

Self Assessment? Don’t Bother

• Kruger and Dunning's research, for example, argues that learners who have clear deficits and would most likely benefit from self-assessment tend to be poor self-evaluators relative to learners who are doing well.

• Self-Reflection? - A potential construct for exploration, which is not related to accuracy of evaluation skills.

The Essentials of SOTL1. Identify a problem relating to teaching

and learning

2. Articulate research questions3. Adequate preparation/literature review4. Identify an appropriate research design

to answer your question5. Analyze the assessment results to

address the research problem6. Connect results with extant research

findings7. Dissemination

The Essentials of SOTL

1. Identify a problem relating to teaching

and learning?

Examples:

We need to improve residents’ teaching skills.

How should we implement 360 evaluations?

Are self-evaluations helpful??

 

Articulate Research Questions

2. Define exactly what is it you want to

know?

Example:

What impact does on online course have on nursing students’ interest in teaching, knowledge of educational theory, and assessment skills?

The Essentials of SOTL

3. Adequate preparation

Informing investigation with existing literature and a theoretical framework offers several advantages, including:

Direction

Focus

Coordination

Essentials of SOTL

4. Identify an appropriate research design to

answer your question.4. Identify an appropriate research design

to answer your question

Things to consider:

How are your questions framed? A good question lends itself well to a particular research design.

Disciplinary epistemologies (e.g., quasi-experimental design, case studies)

Essentials of SOTL

Identify learning expectations by developing

measurable learning objectives.

Examples include:

1. The resident will be able to communicate complicated material to medical students.

2. The resident will be able to identify three strategies to partner with health care managers and health care providers to assess, coordinate, and improve health care.

3. The medical student will be able to demonstrate compassion for a patient, as evidenced by sitting down to communicate important information, maintaining direct eye contact and listening/responding to questions.

Essentials of SOTL

Operationalize: Identify existing or create new

assessment methods to capture evidence that the

residents acquired the learning objectives.

Examples include:

Patient evaluations of the resident

Chart reviews

Written exams

Student evaluations

Journals

Clinical exams

OperationalizeConsiderations for Assessment Tools:

• Credible• Comprehensive• Reliable• Valid

– Content– Concurrent– Predictive– Construct– Consequential

• Useful/Practical in your program

SOAP

? = Don’t Know or Potential opportunities to incorporate

SOAP

23

Example of a Blueprint for Articulating Objectives

Essentials of SOTL

5. Analyze the assessment results to answer the

research problem.

Contributing Scholarship

6. Connect results with extant research

findings.

Otherwise, your results are isolated.

Essentials of SOTL

7. Dissemination

- Effective communication of work to intended audiences helps move forward what is collectively known about the phenomenon.

- It also invites reflective critique to improve quality of future work.

Examples of SOTLat WVU

Dr. Paul Ogershok recognized that teaching basic information about pediatric medicine was little fun for both faculty and students. Therefore, he designed a game to help students acquire a strong foundation of pediatric medicine. He wanted to know if students learned from the game, and whether students reflected that the game was a useful learning opportunity. As evidenced by student comments and observations on evaluation forms, it was determined that the game was a valuable and fun way to help students comprehend important information.

.

Where do I Publish??

• http://www.mlb.ilstu.edu/ressubj/subject/educat/sotl.htm• Multidisciplinary:

Journal of Continuing Education in Nursing

Journal of Nursing Education

Journal of Instructional Psychology

Biochemistry and Molecular Biology Education

Chemical Educator

Journal if the International Association of Medical Science Educators

Quality of Research?

• What population was it based?

• Does it clearly explain constructs?

• Does it generate questions?

• Is it practical?

Glassick's Criterion* Application to Teaching

Clear goals Clear, achievable, measurable objectives

Adequate preparation Up-to-date knowledge; identification and organization of an appropriate quantity and level of material specific to objectives

Appropriate methods (1) Selection of appropriate teaching methods(s) to meet defined objectives (2) Selection of appropriate assessment measures to evaluate outcomes

Significant results (1) Measures of quality/effectiveness of presentation (2) Demonstration of learners' accomplishment of objectives

Effective presentation Making results/process available to colleagues

Reflective critique Critical analysis of teaching activity that results in changes to improve

*From Glassick CE, Huber MR, Maeroff GI. Scholarship Assessed - Evaluation of the Professoriate. San Francisco, CA: Jossey-Bass, 1997.

Submitting Abstracts:Every Word Counts

1. Identify clearly the purpose of you work.Why is it important?

2. Rely on literature to frame an argument.3. Explain the methodology like I am a three-year old? 4. Results: Just the facts.5. Discussion: Was purpose achieved? Limitations? How

does it contribute to literature?6. Conclusion: How does your work contribute to

scholarship? New directions for research? What is the “take-home” message?

Activity/Exercise

Identify at least one problem you would like to address about teaching and learning.

Considerations:

What specific objectives would you like students/residents/faculty to learn?

What assessment methods would help you determine whether the objectives were acquired?

What are your thoughts about methodology?

ReferencesBeattie DS.  Expanding the View of Scholarship (Editorial).  Academic Medicine, Vol 75, N0. 9 / September, 2000, pp. 871-876.

Bloom, B.S. (ed.). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. White Plains, N.Y.: Longman, 1956.

Boyer, E.L. Scholarship reconsidered: Priorities of the professoriate. Princeton, N.J.: Carnegie Foundation for the Advancement of Teaching; 1990.

Cottrell, S.A. (2006). A Matter of Explanation: Assessment, Scholarship of Teaching and their Disconnect with Theoretical Development. Medical Teacher. 28(44), 305-08.

Cottrell, S.A. & Jones, B.A. (2002). Researching the scholarship of teaching and learning: An analysis of current curriculum practices. Innovative Higher Education, 27, 3, 10-16.Cross, K.P. Classroom research: Implementing the scholarship of teaching. San Francisco: Jossey-Bass; 1996.

Huba, M.E. & Freed, J.E. Learner-centered assessment in college campuses: Shifting the focus from teaching to learning. Needham Heights, MA: Allyn & Bacon; 2000.

Fincher RE, Simpson DE, Menin SP, Rosenfeld GC, Rothman A, McGrew MC, Hansen PA, Mazmanian PE, and Turnbull JM.  Scholarship in  Teaching: An Impreative for the 21st Century.  Academic Medicine, Vol 75, N0. 9 /

September 2000, pp. 887-894. 

Palomba, M.B. & Banta, T.W. Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass; 1999.

Shulman, L.S. (1998). Course anatomy: The dissection and analysis of knowledge through teaching. In Hutchings, P. (eds.), The course portfolio: How faculty can examine their teaching to advance practice and improve student learning. (pp. 5-12). Washington, D.C.: American Association for Higher Education.