Using the Capstone Course to Generate

Student Learning Outcomes

Texas State UniversityThomas E. Castleberry

Patricia M. ShieldsHassan Tajalli

Student Learning Outcomes

• A description of the knowledge, skills, values and abilities attained by students as a result of their educational experiences.

Examples:Students will display effective written communication skills.

Students will develop adequate knowledge of budgeting and financial processes.

Student Learning Outcomes

• describe specific behaviors that a student of your program should

demonstrate after completing the program

• focus on the intended abilities, knowledge, values, and attitudes of

the student after completion of the program

– What is expected from a graduate of the program?

– What is expected as the student progresses through the program?

– three questions• What does the student know? (cognitive)• What can the student do? (psychomotor)• What does the student care about? (affective)

Mission-Driven Assessment

• Outcomes = Educational Values = Mission

• Mission-driven assessment derives desired outcomes from the mission, and chooses outcomes that are designed to further that mission.

• Outcome measures then determine how successfully the mission is being accomplished.

Methods of Assessment

• Operationalize outcomes so that they can be measured, by providing evidence of their demonstration or existence.

Examples: Tests, Papers, Surveys, Portfolio Evaluation, etc..

Measuring Learning Outcomes

• should provide an objective means of quantifying the

outcomes, quality, efficiency or productivity of

programs, operations, activities or services

• should indicate how you will measure each of your

outcomes

• should indicate when you will measure each outcome

• provide at least two ways to measure each outcome

Direct and Indirect Methods

• Direct Methods: Processes deliberately designed to measure evidence of student learning by evaluating performance.

e.g., Tests, Projects, Course Grades

• Indirect Methods: Processes that measure perceived student learning.

e.g., Surveys, Transcript Analysis, GPAs

Describing Measurement Methods• What are you going to use?

– presentation, assignment, test, survey, observation, performance rating

• Of and/or by whom?– student, mentor, focus group, alumni

• Context (e.g., where or when)?– point-of-service, capstone, throughout the year, end of

program• For what purpose?

– desired learning outcome

• example: Test the students at the end of the program for their level of knowledge in XYZ

Short, Intermediate, and Long-term Outcomes

• Outcomes for students studying to be practitioners should relate to job performance.

• Challenge: Short/Intermediate outcomes must accurately gage long-term career outcomes within the academic environment.

Example

Intermediate Outcome:

Knowledge of effective management practices

Long-Term Outcome:

Demonstrates effective management practices

Capstones as Assessment Methods

• Provide evidence of the cumulative effect programs have on student learning.

• Less obtrusive and more comprehensive than stand-alone examinations.

• Students use higher-order thinking skills to combine prior learning experiences.

• Opportunities for multiple measures and involvement of external constituencies.

Bloom’s Taxonomy of the Cognitive Domain

• Knowledge: requires and answer that demonstrates simple recall of facts

• Comprehension: requires an answer that demonstrates an understanding of the information

• Application: requires an answer that demonstrates an ability to use information, concepts and theories in new situations

• Analysis: requires an answer that demonstrates an ability to see patterns and classify information, concepts and theories into component parts

• Synthesis: requires an answer that demonstrates the ability to relate knowledge from several areas to create new or original work

• Evaluation: requires an answer that demonstrates an ability to judge evidence based on reasoned argument.

Capstone Models

• Applied Research Project

• Group Project

• Internship

• Capstone Course

Texas State UniversityMPA Program

Mission

The MPA Program's mission is to prepare students for careers as managers and leaders in the public service.

Texas State UniversityMPA Program

Learning Outcomes

• Students should demonstrate the ability to communicate effectively in writing.

• Students should demonstrate the ability to communicate effectively in orally.

• Students should demonstrate the ability to see patterns and classify information, concepts and theories in public policy and administration.

• Students should demonstrate the ability to judge public policy and management evidence based on reasoned arguments.

• Students should demonstrate knowledge and comprehension of National Association of Schools of Public Affairs and Administration’s accrediting standards curriculum components.

Texas State UniversityMPA Program

Student Learning Outcomes: Direct Measures

Intended Student Learning Outcomes

AssessmentMechanism

OperationalizationStrongly Agree//Agree//Neutral//Disagree//

Strongly Disagree

1. Students will display effective written communication skills.

ARP & OralExam:Assessment byoutside practitioner

1. The student’s writing was clear2. The student’s writing used correct grammar and

punctuation.3. The student’s use of referencing was

appropriate.

2. Students will display effective oral communication skills.

ARP & OralExam:Assessment byoutside practitioner

4. The student’s speaking style was clear.

5. The student’s discussion communicated a mastery of the subject matter.

3. Student will be able to see patterns and classify information, concepts and theories in public policy and administration.

ARP & OralExam:Assessment byoutside practitioner

6. The student demonstrated the ability to see patterns and classify information, concepts and theories in public policy and administration.

4. Student will be able to judge public policy and management evidence based on reasoned arguments.

ARP & OralExam:Assessment byoutside practitioner

7. The student demonstrated the ability to judge public policy and management evidence based on reasoned arguments.

Texas State UniversityMPA Program

Student Learning Outcomes

5. NASPAA Curriculum Components

Assess. Mechanism

Operationalization of Knowledge and Skills

To what extent did the ARP deal with the following topics.

Somewhat Significantly Not At All

Management of public service organizations

Application of

quantitative and qualitative techniques of analysis

Understanding of the public policy and organizational environment.

Applied Research Project:

Assessment by outside practitioners

1. Human Resource management

2. Budgeting or financial processes

3. Information management/technology applications

4. Policy and program formation

5. Program implementation/evaluation

6. Decision-making

7. Problem solving

8. Political/legal institutions & processes

9. Economic/social institutions & processes

10. Organization/management concepts

11. Ethical dilemmas or application

Texas State UniversityMPA Program

Student Learning Outcomes: Indirect Measures

Intended Student Learning Outcomes

AssessmentMechanism

OperationalizationStrongly Agree//Agree//Neutral//Disagree//

Strongly Disagree

1. Students will display effective written communication skills.

Student Exit Survey Collected at Oral Exam.

The MPA contributed to the development of my skills in writing.

2. Students will display effective oral communication skills.

Student Exit Survey Collected at Oral Exam.

The MPA contributed to the development of my skills:a. Oral Communicationb. Presentations

3. Student will be able to see patterns and classify information, concepts and theories in public policy and administration.

4. Student will be able to judge public policy and management evidence based on reasoned arguments.

Texas State UniversityMPA Program

Student Learning Outcomes

5. NASPAA Curriculum Components

Assess. Mechanism

Operationalization of Knowledge and Skills

To what extent did the ARP deal with the following topics.

Somewhat Significantly Not At All

Management of public service organizations

Application of

quantitative and qualitative techniques of analysis

Understanding of the public policy and organizational environment.

Student Exit Survey Collected after Oral Exam

The MPA program contributed to my knowledge of

a. Decision making/problem solving

b. Financial resources

c. Human resources

d. Policy process

e. Ethics

f. Information systems technology

g. Law

h. Public Management

i. Organizational processes

Texas State University: MPA Program Student Learning Outcomes

Direct Method: Practitioner Evaluation

• Writing Ability 95% Positive• Oral Communication 93% Positive• Analytical Ability 82% Positive• Reasoning 85% Positive

• Human Resources 33% Significant Coverage• Budgeting/finance 30% Significant Coverage• Info. Mgm/Tech Application 31% Significant Coverage• Policy &Prog. Formulation 70% Significant Coverage• Program Imp./Eval. 85% Significant Coverage• Decision-making 63% Significant Coverage• Problem solving 67% Significant Coverage• Political/legal Inst. & Processes 44% Significant Coverage• Eco./Sco. Inst. & Processes 48% Significant Coverage• Org./Mgmt concepts 56% Significant Coverage• Ethical Dilemmas or Application 33% Significant Coverage

Texas State University: MPA Program Student Learning Outcomes

Indirect Method: Student Exit Survey

KNOWLEDGE AREA• Decision Making Problem Solving 97% Positive• Financial Resources 89% Positive• Human Resources 95% Positive• Policy Process 95% Positive• Intergov. Relations 90% Positive• Ethics 97% Positive• Info. System Technology 81% Positive• Public Management 94% Positive• Organizational Theories 96% Positive• Applied Research 99% Positive• Law 79% PositiveSKILLS• Writing 98% Positive• Quantitative Analysis 97% Positive• Oral Communication 95% Positive• Team work 83% Positive• Financial Analysis 87% Positive• Presentation 93% Positive• Interpersonal Relationships 86% Positive• Project Management 89% Positive• Supervisory 79% Positive• Program Evaluation 83% Positive• Assessment 92% Positive• Use of Technology 80% Positive

Re-Cap of ProcessStep 1: Define Program Mission

Step 2: Define Program Goals

Step 3: Define Student Learning Outcomes

Step 4: Identify assessment methods and targets for each learning outcome

Step 5: Operationalize the Assessment Method

Step 6: Findings