Course Delivery Cycle

What will the students be able to do and

how will the students think when they

complete my course?

What evidence will students

and instructors have of success?

How will I conduct class

to assist students

in their learning/success?

Are the students meeting

the learning outcomes/succeeding?

What is working/not working

in my course?

What changes will be

incorporated in the

next course offering?

• Homework • Exams • Portfolios • Presentations • Written reports • Course survey data

Reflection/

Documentation

Course Portfolio

Use Think/Pair/Share

Do Demonstration

Write Reflections

Conduct Lectures

Model thinking

Course Learning Outcomes

Prior Knowledge?

Your Expectations

Center for Teaching Excellence

at Texas A&M University

Workshop Series

Workshop No. 1: Writing Effective Course Learning Outcomes

Workshop No. 2: Preparing a Formative Course Assessment Plan

Workshop No. 3: Applying Research-based Instructional Strategies

Workshop No. 4: Course Design

Workshop Learning Outcomes

• Construct a complex learning outcome

• Design an assignment that would provide evidence on which to base evaluation of the learning outcome

• Design a scoring scheme with which to evaluate student achievement with respect to the complex learning outcome.

Workshop Action Plan

• Generate a complex learning outcome

• Construct an assignment for student demonstration of the learning outcome

• Construct expected characteristics/dimensions/etc. for evaluation of student work

• Construct levels of achievement for each characteristic

Purpose of Assessment

The primary purpose of assessment is to enable decision makers (i.e., faculty and students) to make better decisions!

Assessment Terminology

• Formative Assessment

– The purpose of formative assessment is to provide information to learners (students and teachers) to help them improve their learning.

• Summative Assessment

– The purpose of summative assessment is to evaluate learning at the end of a prescribed period of time.

• “A recent review (Black and William, 1998) revealed that classroom-based formative assessment, when appropriately used, can positively affect learning.....students learn more when they receive feedback about particular qualities of their work, along with advice on what they can do to improve” (National Research Council, 2001)

• National Research Council (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: National Academies Press.

• Black, P., & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74.

Feedback through Systematic Formative Assessment

Exercise: Generate Learning Outcome

• Think-Pair-Share

– Generate a learning outcome for a course you

teach at one of the following levels in Bloom’s

Revised Taxonomy:

• Analyzing

• Evaluating

• Creating

– It is expected that the learning outcome will

involve several aspects of learning

Exercise Results

1. Design an environmental advertisement using emotional appeal 2. Construct a research proposal related to development of a new

drug 3. Evaluate power requirements for a pump 4. Design an organizational chart according to strategic plan of the

organization 5. Use AUXIUM system effectively 6. Generate legitimate strategic communication campaign 7. Produce a 3D model representing basic space-shape relationships 8. Justify selection of a teaching technique for teaching certain

grammatical structures 9. Design a model that shows blood circulation of the heart

Complex Learning Outcome Options

Exercise Results

1. Design an environmental advertisement using emotional appeal

2. Construct a research proposal related to development of a new drug

3. Design an organizational chart according to strategic plan of the organization

4. Design a model that shows blood circulation of the heart

Complex Learning Outcome Options

Exercise: Assignment

• Think-Pair-Share –Construct a learning assignment through

which would be used to evaluate achievement with respect to the selected learning outcome.

Design an organizational chart according to strategic plan of the organization

Exercise Results

• Attached to this email is the existing organizational chart for

VOS and the strategic plan for the next 10 years.

• List the strengths and weaknesses of the existing

organizational chart with respect to the new strategic plan.

• Prepare 3 very different organizational charts for the new

strategic plan that address the weaknesses and accentuate

the strengths.

• Select one organizational chart and provide detailed

rationales for the selection.

Complex Learning Assignment

Session: Action Plan

• Generate a complex learning outcome

• Construct an assignment for student demonstration of the learning outcome

• Construct expected characteristics/dimensions/etc. for evaluation of student work

• Construct levels of achievement for each characteristic

1 2 3 4 5

Characteristic 1

Indicators of achievement; Suggested improvements

Characteristic 2

Characteristic 3

Characteristic 4

Characteristic 5

Exercise: Characteristics

• Think-Pair-Share

–Construct different

characteristics/dimensions/etc. with

which student achievement with

respect to the selected learning

outcome will be evaluated

Selected Learning Assignment

Exercise Results

• Quality of strengths and weaknesses of existing org. structure wrt strategic plan

• How well did select org chart address strengths and weaknesses, rationale for final selection of org chart

• How well did each new structure address strengths and weaknesses

• Quality of 3 different designs • Relationships between 3 org charts and strategic plan • Justification of new org charts with reference to

literature • Detail of the rationale

Characteristics

Exercise: Multivote Characteristics

• Insert candidate characteristics here

Selected Learning Outcome

Exercise Results

1 2 3 4 5

How well did selected org chart address strengths and weaknesses, i.e., quality of rationale

How well did selected org chart address strengths and weaknesses, i.e., quality of rationale

Exercise: Levels of Achievement

• Think-Pair-Share

– For the selected characteristic, construct five different levels of achievement. For each level, provide

• Description of how achievement will be discerned

• Description of steps for students to improve their learning with respect to the characteristic

– Start with the middle level

Selected Characteristic

Exercise Results

1 2 3 4 5

How well did selected org chart address strengths and weaknesses, i.e., quality of rationale

Rationale makes almost no valid arguments with respect to identified strengths and weaknesses

Rationale makes valid arguments with respect to about half of identified strengths and weaknesses

Rationale makes valid arguments with respect to almost all of identified strengths and weaknesses

Selected Characteristic

Exercise Results

1 2 3 4 5

Quality of 3 different designs (presumes N established guidelines for design)

? Meets about half of the guidelines

?

Selected Characteristic

What are some benefits of using rubrics?

Students • Rubrics add meaning to grades and allow students to

understand the expectations of their instructors.

• Rubrics provide feedback to students about what they have learned and what they have yet to learn.

Agreed Upon Values • Rubrics allow stakeholders to discuss and determine agreed

upon standards and values.

Data • Rubrics promote consistent scoring.

• Rubrics provide rich, descriptive data that can be used to improve instruction.

Enhancing Learning by Improving Process Skills in STEM (ELIPSS)

• http://www.elipss.com/

Minute Paper

• Write brief answers to the following questions: – What is most valuable or

helpful about developing an assessment plan?

– What is the “muddiest or most confusing point” about developing an assessment plan?

Exercise: What is your course assessment plan?

• Individual Reflection

–Think about a course you will be teaching next semester.

–Describe, in writing, your current assessment plan for the course.

Classroom Assessment

Jeffrey E. Froyd TEES Research Professor

Texas A&M University froyd@tamu.edu

Classroom Assessment Techniques

Angelo, T. A., & Cross, P. K. (1993). Classroom Assessment Techniques: A Handbook for College Teachers (Second ed.). San Francisco, CA: Jossey-Bass.

• Background knowledge probe

• Muddiest point

• One-sentence summary

• …

• Questions:

– Give a five-to-ten-line summary of last night’s reading. Include two or three main ideas.

– What were three of the most important points from yesterday¹s discussion?

– If you were summarizing today¹s discussion for a friend who was absent, what two ideas do you think are the most essential?

– Define in your own words the term ________________.

– Tell me three things wrong with this statement: ____________.

Classroom Assessment Techniques

• At the end of each class period, write brief answers to the following questions: – What is most valuable or

helpful idea or concept that you learned today?

– What is the “muddiest or most confusing point” about in today’s lecture?

Minute Paper

• At the end of each class period, write brief answers to the following questions: – Write a one-sentence summary of

the content of the class period today.

– What is one potential application of today’s content to your career aspirations?

– What is the muddiest point in today’s class period?

Variations on a Minute Paper

How well does using minute papers in courses work? • Findings: “This result suggested, as we hypothesized, that the use of the

one-minute paper improves student performance. Its coefficient implied that the use of the one-minute paper increased student performance by approximately .5 of a point on the postTUCE exam, ceteris paribus.”

• Findings: “This evidence suggests that the benefit to students from using the one-minute paper does not depend on the instructor who implements it.”

• Findings: “This evidence supported our initial hypothesis that the benefit to students from using the one-minute paper does not depend on their ability level.”

• Assertion: “When asked by college teachers to identify the single pedagogical innovation that would most improve their teaching, Light (1990, 35) always responds with the one-minute paper, an idea that ‘swamped all others.’”

Chizmar, J. F., and Ostrosky, A. L. (1998). The One-Minute Paper: Some Empirical Findings. The Journal of Economic Education, 29(1), 3–10

How well does using minute papers in courses work?

• Findings: Overall results indicate that performance on subsequent essay quizzes was significantly higher by students who wrote one-minute papers than performance by students who did not write the papers.

• Findings: Of particular interest to instructors was that the increase in quiz scores when one-minute papers were not graded was significantly higher than when the one-minute papers were graded.

Almer, E. D., Jones, K., and Moeckel, C. L. (1998). The impact of one-minute papers on learning in an introductory accounting course. Issues in Accounting Education, 13(3), 485–495

Classroom Response Systems

Jeffrey E. Froyd TEES Research Professor

Texas A&M University froyd@tamu.edu

Classroom Response Systems

• Ask a question

• Tally student response using a classroom response system

– Clickers

– Poll Everywhere

– Socrative

– Learning Catalytics

–…

Classroom Response Systems

• Fies, Carmen, & Marshall, Jill. (2006). Classroom response systems: A review of the literature. Journal of Science Education and Technology, 15(1), 101-109. doi: 10.1007/s10956-006-0360-1

• Caldwell, Jane E. (2007). Clickers in the large classroom: Current research and best-practice tips. CBE Life Science Education, 6(1), 9-20. doi: 10.1187/cbe.06-12-0205

• Kay, R. H., & LeSage, A. (2009). Examining the benefits and challenges of using audience response systems: A review of the literature. Computers & Education, 53(3), 819-827. doi: 10.1016/j.compedu.2009.05.001