Upload
belle
View
20
Download
0
Tags:
Embed Size (px)
DESCRIPTION
IN top FORM Assessment Plan. Please sit with other faculty members from your program of study. Goals for today’s session. By the end of today’s session, you should: Understand basic assessment of student learning; - PowerPoint PPT Presentation
Citation preview
INtopFORM Assessment Plan
Please sit with other faculty members from your program of study
Goals for today’s session• By the end of today’s session, you should:– Understand basic assessment of student learning;– Tentatively identify when and where assessment of the
six ITF LOs will take place in your program;– Develop a plan for assessment measures to be used (ITF
rubric, major field test, standardized final, etc.);– Develop a plan for collecting baseline data (preferably
during Spring 2014);– Decide when you and your colleagues will discuss
assessment results;– Know what your responsibilities are for reporting; and– Be familiar with the ITF assessment calendar.
Assessment Basics
What is assessment?
1. Collecting data so we can report it to SACS (or THEC, TBR, AACSB, etc.)
2. Something the folks in Dossett Hall came up with to keep faculty busy
3. A useful teaching and learning tool
Assessment is
Deciding what we want our students to learn
Making sure they learn it
From Linda Suskie, courtesy of Jane Wolfson, Towson University
http://img4.wikia.nocookie.net/__cb20100313135124/simpsons/images/4/47/Dabf20.gif
Assessment Cycle
Excerpted from Assessing Student Learning by Linda Suskie, Jossey-Bass, 2009
1)
2)
3)
4)
Program-Level Assessment for INtopFORM
Assessment Template
• Outcomes: Your six discipline-specific ITF LOs• Opportunities: Where will students practice ITF skills?• Measurement Sites: Which courses and assignments?• Measures: Projects, papers, exam questions, clinicals• Sampling (if used)• Baseline Data: When, where, and how collected• Sharing: Who can act on results to improve student
learning?
Step 1: Establish Learning Goals
• Six INtopFORM Learning Outcomes• What do they mean in YOUR
discipline?– See Parts 1 and 2 of your planning
worksheets
QUESTIONINGSEEKING
EVALUATINGUSING
COMMUNICATINGRECOGNIZING
Record draft ITF learning outcomes for your program in Column A of the Assessment Plan Template
Step 2: Provide Learning Opportunities• Must allow students to practice and develop skill in
each of the six INtopFORM Learning Outcomes
• Where will this take place in your curriculum?– See results of Curricular Mapping exercise
• What methods might you use?– See Teaching Ideas (Part 3 of planning worksheets) for
each ITF LO
Record possible learning opportunities for your students in Column B of the Assessment Plan Template; please be specific (course + assignment or activity)
Step 3: Assess Student Learning
• Where will you measure student learning?• Must be measurable or observable• Goal– Collect “reasonably accurate, truthful evidence”• Doesn’t have to be worthy of publication
– Good enough to use with confidence• Possibilities include single assignments,
capstone courses, projects, final exam items
Assessment Measures• Two options:– Modify the ITF assessment rubric to evaluate
papers or projects (preferred);– Study students’ performance on designated items
on major field tests, standardized final exams, or locally designed tests of information fluency
Record where assessment might take place (course, assignment) in your curriculum in Column C of the Assessment Plan Template. Include possible measures (exam items, rubrics) for your program in Column D.
QUESTIONING: Students ask questions that facilitate the solution of problems and the pursuit of opportunities.
Questions define the scope of the area of interest. Questions identify key issues to be resolved. Questions demonstrate understanding of the problem, task or topic. Questions encourage a broad search for information. Questions are relevant to the problem, task or topic. Questions bring needed focus or structure to the problem, task or topic. Questions examine diverse perspectives on the problem, task or topic. Questions reflect attitudes of openness and curiosity. Overall Rating (required item) Other items (subject to approval):
SEEKING: Students locate or create information needed to accomplish their purposes. Sources used are credible. Sources used are appropriate for the purpose. The student uses several sources. The student uses different kinds of sources. Sources and information used indicate a thorough search process. Sources and information used indicate persistence and resourcefulness in seeking information. The student insightfully evaluates the search process. The student conducted an iterative search process. The student conducted an efficient search process. The student correctly identifies limitations of research he or she conducted. Overall Rating (required item) Other items (subject to approval):
EVALUATING: Students apply critical thinking skills in evaluating sources, information and search processes.
The student correctly analyzes the reliability of sources or information. The student correctly analyzes the potential bias of sources. The student insightfully discusses the implications of information. The student demonstrates comprehension of information. The student correctly identifies assumptions underlying information. The student identifies questions raised by information. The student analyzes his or her own and others’ assumptions. Overall Rating (required item) Other items (subject to approval):
USING: Students assemble and synthesize information to accomplish their purposes. Information used is relevant to the purpose. The amount of information provided is sufficient for the purpose. The student draws sound conclusions from information. The student skillfully synthesizes information from multiple sources. The student draws non-obvious insights from information. The analysis of information flows logically from point to point. The student does not oversimplify complex information. The student clearly relates information to the larger purpose. The student draws upon diverse perspectives. Overall Rating (required item) Other items (subject to approval):
COMMUNICATING: Students communicate information effectively. The student aids others’ understanding of information. The student provides useful explanations of information. The student presents information accurately. The student presents information completely. The presentation of information helps others see connections or larger themes. The presentation of information arouses others’ interest. The presentation of information evokes questions or further study by others. The student openly acknowledges when information is inconsistent or inconclusive. Overall Rating (required item) Other items (subject to approval):
RECOGNIZING: Students recognize the responsibilities and consequences related to information ethics and intellectual property.
The student clearly identifies sources of information used. The student respects confidentiality rights associated with information. The student shows fair use in the quotation of sources. Overall Rating (required item) Other items (subject to approval):
INtopFORM Assessment Guidelines• Must evaluate all six ITF LOs• Must occur annually• Must provide data on the culminating skills
and abilities of students in a program of study– Ordinarily in the senior year / 4000-level courses
• Must yield findings applicable to all majors – Can evaluate all majors’ work or that of a
sufficient representative sample
Sampling• More evidence collected means higher
confidence in conclusions, but be practical• If you use a sample, it should adequately
represent your student population– Online vs. on ground– Adult vs. traditional
Record from whom you will collect assessment information in Column E of the Assessment Plan Template; describe sampling (if applicable)
Baseline Data• Must be collected during Year 1
– Spring 2015 preferred• Use standard reporting template (provided) to include:
– Overall assessment for each of the six outcomes• Mathematical average (can be weighted) or separate rating
– Number and % of students rated as:3 = Excellent2 = Satisfactory1 = Unsatisfactory
– Whether each student was rated satisfactory on all outcomes
Record ideas of how your program might collect baseline data in Column F of the Assessment Plan Template.
Student ID A B C Dept. TotalsQuestioning: Students ask questions that facilitate the solution of problems and the pursuit of opportunites
Students raise appropriate questions regarding a particular issue 3.00 3.00 3.00 3.00The line of argument anticipates and adequately responds to relevant objections 3.00 3.00 2.00 2.67Overall assessment of Questioning 3.00 2.00 3.00 2.67Seeking: Students locate or create information needed to accomplish their purposes
Uses evidence-based sources and peer reviewed literature to support all aspects of the paper 3.00 2.00 3.00 2.67Overall assessment of Seeking 3.00 2.00 3.00 2.67Evaluating: Students apply critical thinking skills in evaluating sources, information, and search processes
Identifies appropriate problem for case presented 3.00 2.00 3.00 2.67Demonstrates clear understanding of response to treatment 3.00 3.00 3.00 3.00Correctly determines next steps in process 2.00 2.00 3.00 2.33Overall assessment of Evaluating 3.00 2.50 3.00 2.83Using: Students assemble and synthesize their information to accomplish their purposes
Adequately describes problem using professional terminology 3.00 3.00 3.00 3.00Demonstrates logical progression of relevant data and facts 3.00 3.00 3.00 3.00Develops accurate and appropriate AQI statements 3.00 2.00 3.00 2.67Plans appropriate strategies for monitoring interventions 3.00 3.00 3.00 3.00Overall assessment of Using 3.00 2.75 3.00 2.89Communicating: Students communicate information effectively
Addresses significance of the issue and facts pertinent to the issue 3.00 3.00 3.00 3.00Strong conclusion summarizing data, facts, and information related to issue 3.00 3.00 2.00 2.67Uses appropriate fonts, font sizes, spacing, cover page, bibliography and headings for each section 3.00 3.00 2.00 2.67Uses appropriate word choice, free from grammatical and spelling errors 3.00 2.00 3.00 2.67Overall assessment of Communicating 3.00 2.75 2.50 2.83Recognizing: Students recognize the responsibilities and consequences related to information ethics and intellectual property
Sources cited appropriately using APA citation style 3.00 1.00 3.00 2.33Gives credit when paraphrasing or rewording 3.00 2.00 3.00 2.67Overall assessment of Recognizing 3.00 1.50 3.00 2.50
Example Program of Study QEP Rubric
Step 4: Use the Results
• Assessment is worthless if it isn’t useful– How will results be analyzed? By whom?– How and when will results be shared? – Who can act on your assessment data to make
improvements?
Discuss how your program might use the assessment information you collect and record in Column G of the Assessment Plan Template.
You are not alone• INtopFORM provides support during Faculty
Learning Communities• INtopFORM staff will hold meetings with each
program to discuss assessment and implementation plans
• The Director of Assessment (with assistance from the Director of the QEP) will – Assist faculty in designing assessments– Approve each programs’ assessment protocols
Assessment Goals of INtopFORM
Institutional Assessment Goals
For each program:• Improvement in Years 2-5 compared to baseline data in
Year 1
• >30% of the rubric ratings or test scores on each INtopFORM learning outcome are “excellent”
• >90% of the rubric ratings or test scores on each INtopFORM learning outcome are at least “satisfactory”
• >90% of program majors achieve rubric ratings or test scores demonstrating at least “satisfactory” performance on all INtopFORM learning outcomes
California Critical Thinking Skills Test• ETSU’s Senior Exit Exam, administered since 2005 for
general education assessment• 34 questions, 45 minutes in length• Five areas: Analysis and Interpretation, Inference,
Evaluation and Explanation, Deductive Reasoning, and Inductive Reasoning
• CCTST maps well to EVALUATING and USING INtopFORM learning outcomes
• Office of Assessment will collect CCTST data annually and provide reports to programs of study
CCTST Score Interpretation
CCTST Area Description
Score Interpretation
Weak Moderate Strong
Analysis (maps to EVALUATING)
used when pulling apart arguments and points of view 0 – 2 3 – 4 5 or
more
Inference (maps to USING)
used whenever drawing conclusions based on reasons and evidence
0 – 5 6 – 11 12 or more
Evaluation (maps to EVALUATING)
used when deciding believability of a given statement 0 – 3 4 – 7 8 or
more
Inductionused when deciding that evidence supports truth in given conclusion
0 – 5 6 – 11 12 or more
Deduction
used when deciding a false conclusion is impossible considering all arguments are true
0 – 5 6 – 11 12 or more
CCTST Total Score
• 8-12 (Weak)
• 13-18 (Moderate)
• 19-23 (Strong)
• ≥24 (Superior)
CCTST Trend, 2005-06 to present
2005-06
2006-07
2007-08
2008-09
2009-10
2010-11
2011-12
2012-13
2013-1416
16.2
16.4
16.6
16.8
17
17.2
17.4
17.6
17.8
18
Academic Year
Mea
n CC
TST
Tota
l Sco
re
Year N Mean score Percentile Rank2005-06 1135 17.5 61st – 68th 2006-07 2677 17.4 54th – 61st 2007-08 1815 17.7 54th – 61st 2008-09 1593 17.2 55th 2009-10 1512 17.3 56th 2010-11 1672 17.5 59th 2011-12 1859 17.6 57th 2012-13 1723 16.5 49th 2013-14 2016 17.0 51st
Goals for CCTST PerformanceFor each program:• Mean CCTST Total Scores and subtest* scores should
exceed baseline in Years 2-5
• >30% of scores on each CCTST subtest* at or above “strong”
• >90% of scores on each CCTST subtest* at or above “moderate”
• >30% of CCTST Total Scores are at least 19 (“strong”)• >90% of CCTST Total Scores are at least 15
(“moderate”)*mapped to an INtopFORM learning outcome
Annual Assessment and Improvement Meetings
• Each fall in Years 3-5• Includes Director of Assessment, QEP Director,
program leaders, and other interested faculty• Purpose
– Review previous year’s assessment data– Use results to refine plan for enhancing information fluency– As needed, define improvement actions, responsible
parties, and calendars• Annual assessment and improvement report drafted
by Directors and approved by program faculty
Overview of Assessment Timeline Programs of Study, Cohort 2
Year 1: 2014-15 Year 2: 2015-16Su Fa Sp Su Fa Sp
Develop program assessment plan X X X (refine) Collect baseline data X Collect and analyze results, refine plan, make improvements
X X X
Year 3: 2016-17 Year 4: 2017-18Su Fa Sp Su Fa Sp
Develop program assessment plan
Collect baseline data Collect and analyze results, refine plan, make improvements
X X X X X X
Year 1 Assessment Calendar,Programs of Study
First Year of ParticipationWhen What WhoFall Create a plan to help all students in the
program attain the six INtopFORM learning outcomes
Program of Study Faculty
Design assessment of ITF learning outcomes
Program of Study Faculty
Spring Collect and report baseline assessment data Program of Study Faculty
Fall, Spring, and Summer
Collect and report baseline CCTST data Office of Assessment
Summer Send participating programs of study analysis of baseline data, including CCTST
Office of Assessment
Year 2 Assessment Calendar,Programs of Study
Second Year of ParticipationWhen What WhoFall Meet to review baseline data and discuss
implications for planned improvements in program
Director, QEP; Director, Assessment; Program of Study Faculty
Fall and Spring Implement plans to help students attain learning outcomes
Program of Study Faculty
Collect and report data on ITF learning outcomes
Program of Study Faculty
Fall, Spring, and Summer
Collect and report CCTST data Office of Assessment
Summer Send participating programs of study analysis of data, including CCTST
Office of Assessment
Years 3-5 Assessment Calendar,Programs of Study
Third, Fourth, and Fifth Year of ParticipationWhen What WhoEarly Fall Meet to evaluate prior year assessment
data and refine program’s INtopFORM planProgram of Study Faculty; Director, Assessment; Director, QEP
Fall and Spring Collect and report data on ITF learning outcomes
Program of Study Faculty
Fall, Spring, and Summer
Collect and report CCTST data Office of Assessment
Summer Send participating programs of study analysis of data, including CCTST
Office of Assessment
Decide what you want your students to learn
Make sure they learn it
Contact Information
Cheri ClavierDirector of Assessment
136 Sherrod