Leveraging Blackboard Outcomes Assessment as a Continuous ...leaptx.org/wp-content/uploads/2020/02/Leveraging... · • Consider a continuous improvement strategy for your own institution’s

Leveraging Blackboard Outcomes Assessment as a Continuous Improvement Strategy at Amarillo College

Tina Babb Director of Institutional Effectiveness

Learning OutcomesParticipants will…• Identify several practices Amarillo College has built around the

Blackboard Outcomes Assessment application• Consider a continuous improvement strategy for your own

institution’s assessment and accreditation efforts • View a demonstration of artifact assessment• Examine the reporting outputs of this tool that trigger discussions

with faculty

Outcomes Assessment Then and Now…

• New streamlined approach• Manual assessment of hard-copy

artifacts vs. online in current LMS (Blackboard)

• Sampling?• Analysis time-consuming and

completed separately• Lacked structure for goal setting• Lacked “Closing the Loop”

Green Bay, Wisconsin | January 14, 2019

AC Assessment Plan Components


• 8.2.a – Student Outcomes: Educational Programs• Program Student Learning Outcomes (PSLOs) defined by each department on a

Curriculum Map and assessed

• 8.2.b – Student Outcomes: General Education• Institutional Student Learning Outcomes (ISLOs) defined by the Texas Higher

Education Coordinating Board and assessed• Communication, Critical Thinking, Empirical & Quant Skills, Personal Responsibility, Social

Responsibility, and Teamwork

Alignment - SACSCOC Standards 8.2.a and 8.2.b


Building Practices with Outcomes Assessment: Core Curriculum and General Education Competencies


Building Practices with Outcomes Assessment:General Education Competencies – Teamwork Example


Building Practices: Development of Planning Template


Building Capacity: Leveraging Technology


Building Capacity: Continuous Improvement Strategy

Assessment Results Provided to Faculty - Identify Improvements




Implement Improvements &

RE-ASSESS

Personal ResponsibilityStudents Upload

Work to Bb Common Assessment Folder

Faculty Evaluate Artifacts


RE-ASSESS



Empirical & Quantitative Skills

Students Upload Work to Bb Common

Assessment Folder


Critical Thinking PILOTStudents Upload

Work to Bb Common Assessment Folder



RE-ASSESS

Overall Assessment Map

Amarillo College Assessment Plan

Gen Ed Competency Map

General Education Competencies

Foundational Component AreaCommunicationCritical ThinkingEmpirical & Quantitative SkillsPersonal ResponsibilitySocial ResponsibilityTeamwork

American HistoryXXXX

CommunicationXXXX

Creative ArtsXXXX

Government/Political ScienceXXXX

Language, Philosophy, & CultureXXXX

Life & Physical SciencesXXXX

MathematicsXXX

Social & Behavioral SciencesXXXX

Schedule of Gen Ed Assessment

Amarillo College 5 Year Assessment Schedule of General Education Competencies

2016-17 (Pilot)2017-182018-192019-202020-21

General Education Competency (ISLO)*FALLSPRINGFALLSPRINGFALLSPRINGFALLSPRINGFALLSPRING

Communication Skills (COM)PILOTPlanning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IE

Critical Thinking (CT)PILOTPlanning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IE

Empirical & Quantitative Skills (EQS)Planning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESS

Personal Responsibility (PR)Planning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify Improvements

Social Responsibility (SR)Planning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify Improvements

Teamwork (TW)Planning Template & Place Assignment in Common Assessment Folder Faculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To DeptsDepts Analyze Results & Identify ImprovementsDepts Implement Improvements & RE-ASSESSFaculty Complete Assessment-Sample Provided by IEIE Data Analysis & Results To Depts

*BEFORE TERM BEGINS Identify course sections via the planning template and place assignment & instructions in Common Assessment folder in Bb.

Faculty will complete the assessment in the Blackboard Outcomes Assessment

*ISLO = Institutional Student Learning Outcome, IE = Institutional Effectiveness, Bb = Blackboard

Revision in Progress-Dec 2019

General Education Competencies2017-182018-192019-202020-21

FALLSPRINGFALLSPRINGFALLSPRINGFALLSPRING

Communication SkillsPILOTStudents Upload Work to Bb Common Assessment Folder Faculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate Artifacts

Critical ThinkingPILOTStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate Artifacts

Empirical & Quantitative SkillsStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESS

Personal ResponsibilityStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify Improvements

Social ResponsibilityStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify Improvements

TeamworkStudents Upload Work to Bb Common Assessment FolderFaculty Evaluate ArtifactsAssessment Results Provided to Faculty - Identify ImprovementsImplement Improvements & RE-ASSESSFaculty Evaluate Artifacts

Gen Ed Steps

AC ISLO/PSLO Assessment Cycle

*ISLO = Institutional Student Learning Outcomes (General Education Competencies)

*PSLO = Program Student Learning Outcomes (Curriculum Map)

*IE = Institutional Effectiveness

STEP 1: Prior to Semester Start: Departments Identify Courses/Artifacts on the Bb Outcomes Assessment Planning Template + Identify Faculty Evaluators

STEP 2: IE Samples Artifacts as Semester Ends and Provides Evaluation Link to the Faculty Evaluators via Bb Outcomes Assessment Tool

STEP 3: Depts/Faculty Evaluators Assess Sampled Artifacts the Following Semester

(ASSESSMENT PART 1)

STEP 4: IE Provides Analysis & Reports of Assessment Results to Program Chairs/Coordinators

STEP 5: Program Chairs/Coordinators along with Faculty Analyze Results and Identify Improvements

STEP 6: Program Chairs/Coordinators/Faculty Implement Improvements for the ISLOs/PSLOs

STEP 7: Re-Assess with Implemented Improvements (ASSESSMENT PART 2)

ASSESSMENT PART 3: CLOSING THE LOOP

STEPS 2-5 Repeated on the Improvements - Data analyzed for continuous improvement

PSLO Cycles

2017-182018-192019-202020-212021-22

Course(s) Identified on Curriculum MapFALLSPRINGFALLSPRINGFALLSPRINGFALLSPRINGFALLSPRING

AssessDepts Complete Assessment-Sample Provided by IEIE Data Analysis & Report Results To DeptsDepts Analyze Results & Identify ImprovementsRE-ASSESS & Implement ImprovementsDepts Complete Assessment-Sample Provided by IEIE Data Analysis & Report Results To DeptsDepts Analyze Results & Identify ImprovementsAssessDepts Complete Assessment-Sample Provided by IE



Legend:

Departments identify course assignments to be assessed via the planning template and place assignment in the Common Assessment folder in Bb.

Departments will complete the assessment in the Bb Tool.

*ISLO = Institutional Student Learning Outcome

IE = Institutional Effectiveness


Blackboard Outcomes Assessment: Goal Sets, Goal Types and Outcomes

Goal Types can include: General Education Competencies (ISLOs), Program StudentLearning Outcomes (PSLOs), Accreditation Standards, QEP, etc.


Blackboard Outcomes Assessment: Creating Rubrics for Alignment with Courses, Goals and Student Artifact Collection

Ok, So How Do We Perform the Actual Assessment?• An assessment session can be created when these three criteria are in place in

Blackboard Outcomes:• Students have uploaded work into ”Common Assessment” drop-box• Faculty attach instructions to the drop-box• Institutional Effectiveness attaches the alignments to the drop-box

• An email from Blackboard is generated to all faculty evaluators to begin their assessment

• Faculty have approximately 4 weeks to assess student work• Video: You Are Now a Faculty Evaluator

https://use.vg/usbDnvXwFvDc

Blackboard Outcomes Assessment Application: Results

2 reports are generated at the close of each assessment session: 1. Excel spreadsheet-contains raw data of faculty scoring2. Data visualization graphs in .pdf

Identifying Gaps in Student Learning by Measuring Extent of Achievement of Learning Outcomes

Frequency Distribution – How Are Students Doing Overall? Where Do Students Struggle?


• “I felt that the Bb Assessment pilot worked much better than the old system of artifact collection. Being able to assess the artifacts is so much easier than doing it by hand and trying to get everyone together to meet. I have no suggestions for improvement. I thought it went smoothly and was efficient.” –Professor of Social and Behavioral Sciences

• “I think that the key factor in the whole process is to allow plenty of time for training of developing the rubrics and in learning to assess the artifacts in Bb and in doing the actual assessment. This new process of assessing was faster and more streamlined than the old process of assessing by hand.” –Professor of English

• “I found the tool for reviewing easy to use and simple to navigate.” –Instructor of Humanities

Faculty Feedback


Closing the Loop…


• Communication• 52 departments/135 academic programs

• Leveraging Technology• Blackboard Outcomes Assessment tool• Institutional Research data dashboards

• IE Collaboration • Center for Teaching and Learning built into assessment cycle

• Faculty Development and time to reflect• Faculty Development Model

• Incentivize (https://www.actx.edu/ctl/resources)

Building Capacity: Increasing Faculty Engagement

https://www.actx.edu/ctl/resources

Building Capacity: Data Resources for Faculty(dair.actx.edu)

Example: Criminal Justice Common Assessment


Example: Closing the Loop on Empirical Skills for Criminal Justice Students


A Faculty Member’s Perspective: Helpful Hints & Improvements Student Engagement Techniques

• TILT• Capstone Assignment

Design a high-quality, efficient assignment• Signature assignment

Train your faculty evaluators

Reflection – Build time in the assessment cycleCollaboration – Faculty development opportunitiesData Driven – Use of assessment data to make informed decisionsFaculty-Led Improvement strategies

Building Capacity: Faculty Development as a Focal Point of Student Success

Contact Information:Tina M. BabbDirector of Institutional Effectiveness(806) [email protected]/ie/

Questions?

mailto:[email protected]://www.actx.edu/ie/training-materials

Slide Number 1Learning OutcomesOutcomes Assessment Then and Now…��Slide Number 6Slide Number 7Slide Number 8�Building Capacity: Leveraging Technology�Slide Number 10Slide Number 11Slide Number 12 Ok, So How Do We Perform the Actual Assessment?Blackboard Outcomes Assessment Application: ResultsIdentifying Gaps in Student Learning by �Measuring Extent of Achievement of Learning OutcomesFrequency Distribution – How Are Students Doing Overall? Where Do Students Struggle?��Building Capacity: Data Resources for Faculty�(dair.actx.edu) Example: Criminal Justice Common AssessmentSlide Number 22Slide Number 23Slide Number 24Slide Number 25Slide Number 26

Documents

Leveraging Blackboard Outcomes Assessment as a Continuous ...leaptx.org/wp-content/uploads/2020/02/Leveraging... · • Consider a continuous improvement strategy for your own institution’s