Program Assessment: A Method for Integrating Curriculum, Field

G E R I C R A W L E Y - W O O D S

K A R E N G R E E N

D O U G L A S C . H O R N E R

S H E P H E R D U N I V E R S I T Y

Program Assessment: A Method for Integrating Curriculum, Field,

and the EPAS

This presentation may be accessed at:

H T T P : / / W W W . S H E P H E R D . E D U / S W W E B / C O M P E T E NC Y E X A M A N D P R I M A R Y T R A I T A N A L Y S I S . H T M L

B Y C L I C K I N G O N “ B P D 2 0 1 2 P O W E R P O I N T ”

T H I S L I N K W I L L A L S O T A K E Y O U T O A L L O F T H E T O O L S W E W I L L B E D I S C U S S I N G T O D A Y

WHAT ARE TWO CHALLENGES YOU ARE FACING WITH THE TEN COMPETENCIES AND 41

PRACTICE BEHAVIORS?

DISCUSS WITH A PARTNER FOR ABOUT 5 MINUTES

In Terms of Assessment---

Elements of Program Assessment

Formative

Measures implemented throughout the individual courses which document the expanding knowledge and skills and developing values

Summative

Link the Program goals, EPAS competencies, and practice behaviors

Two major quantitative tools Comprehensive Exam

Field Evaluation

IDENTIFY THE TOOLS/ELEMENTS IMPORTANT FOR

SUMMATIVE ASSESSMENT IN YOUR PROGRAM

For Discussion in Dyads:

Summative Program Assessment:Two Dimensions

The Comprehensive Exam- includes an oral review

The Field Evaluation- includes individualized learning agreement

The Comprehensive Exam

Ten areas of knowledge, values, ethics, and skills embedded in the Program’s goals, EPAS competencies, and practice behaviors.

Links the curriculum to the student’s field experience

Evaluated using a Primary Traits Analysis rubric

The Practice ContextThe Practice Context The Field ContextThe Field Context

The Profession

The Community

The Organization

Assessment of Existing Social Policy

Utilization of Community Resources

Problem Identification and Assessment

Selection of an Intervention Plan

Implementation of the Plan

Evaluation and Feedback Regarding Intervention Plan

Overall Feedback to the Agency

The Ten Elements of the Exam

The Process of Writing in Drafts

Fall Semester, Senior Year in Methods 3

•Professional Context of Practice

•Community Context of Practice

Spring Semester, Senior Year in Capstone Seminar

•Organizational Context of Practice

•Assessment of Existing Social Policy

The Review Process

All faculty read each exam and evaluate using the Primary Trait Analysis rubric

The exam is 50% of the senior capstone course grade

The Exam is worth 500 points: 400 for the written component, 100 for the oral review

The Oral Review

Oral review sessions are two hours each and include two-three students from similar settings-health care, corrections, child welfare, etc.

The goal is to review the similar themes, concerns, challenges, etc. raised by the BSW candidates in a collegial and professional environment – “the tea”

Provides feedback to the faculty regarding both implicit and explicit curricula

E X A M P L E : T H E C O M M U N I T Y C O N T E X T O F P R A C T I C E

4 Q U E S T I O N S

Constructing a Comprehensive Exam

The Community Context of Practice

Question 1

Describe the community context of your practicum experience, e.g. the demographic & ecological characteristics of the city, town, region (urban, rural, suburban), in which the agency is located; diverse populations, including ethnicity, class structure; significant minority groups.

ANALYZE/INTERPRET THE IMPLICATIONS OF THIS INFORMATION FOR PRACTICE.


Question 2

Were your services directed toward any specific subgroup or special social program within the community, or segment(s) of the community which have particular relevance to your practice?


Question 3

What are the predominant problems, issues and concerns of the people you were serving? What are the particular social problems your agency attempts to address? What are other social problems in the community of which you are aware?


Question 4

In what ways have poverty, discrimination and other forms of social injustice become evident to you?

Integration with EPAS Competencies and Practice Behaviors

Question

1

2

3

4

Competency Practice Behavior

2.1.3 13

2.1.4 14

2.1.9 27

2.1.3 13

2.1.4 14

2.1.3 13

2.1.7 23

2.1.1 1

2.1.3 13

2.1.4 16

2.1.5 18

Two Examples of Question One Linked to :

Competency

2.1.3

Apply critical thinking to inform and communicate professional judgments

Practice Behavior

13

Demonstrate effective oral and written communication with individuals, families, groups, organizations, and communities

Question One-Linked to:

Competency

2.1.9

Respond to contexts that shape practice

Practice Behavior

27

Continuously discover, appraise, and attend to changing locales, populations, scientific and technological developments, and emerging societal trends to provide relevant services

Describes community demographic characteristics including stratification, cultural diversity, and discusses the implications of these data.

Connection of services and specific social problem and/or group of citizens

Descriptions of social problems in the community.

Role of agency re: social problems in the community

Analysis of poverty, discrimination, and social injustice in relation to social problems in the community of interest.

Summary Criteria for Evaluating “Community Context” using Primary Trait Analysis (PTA)

4 P O I N T S C A L E

Community Context of Practice PTA Criteria

4 points4 points 3 points3 points

Sophisticated description, referencing, and analysis of community characteristics in relation to social problems experienced by citizens

Detailed and integrated analysis of the agency as a change agent

Clear and integrated presentation of poverty and social injustices may be connected to the unique community context of practice

Discusses the implication of the data presented.

Adequate description with some analysis of relationship of the social problems to a specific community

Description of the agency as a change agent

Recognition that poverty and social injustice may be related to social problems.


2 points2 points 1 point1 point

Some description of community characteristics

Limited recognition/description of community-based social problems

Limited recognition/description of agency role in social problem area

Limited exploration of the role of poverty and social injustice as potential issues in social problems

Elementary description of a few community characteristics

Social problems in community ignored or simplistic

Role of agency missing or confused

No recognition or description of poverty or social injustice as possible in the community


Challenges

Very time consuming in April—the cruelest month

In larger programs, it is impossible for every faculty member to review each exam; working groups would need to be established, perhaps by areas of the faculty member’s specialization.

Arranging appropriate groupings of students-only one student in a particular setting and/or personalities

Coordination of schedules of students and faculty

Scheduling of space

Strengths

Students demonstrate behaviorally that their skills meet EPAS requirements

Documents the integration of course work and the field experience with the competencies and practice behaviors

Provides the opportunity for all the faculty to review and evaluate each student’s work

The primary trait rubric increases inter-rater reliability

Provides solid basis for employment and graduate school references

Oral Review provides Final Feedback to Program

Last 10 to 15 minutes of 2 hour review

Invites BSW candidates to give verbal feedback on elements of implicit and explicit curriculum

Comments are recorded, compiled, and analyzed

Common Themes: Course requirements

Experiences in the classroom and the field

Suggestions for changes to the program

Epiphanies and testimonials

Type Topic Positive Suggested Innovations NegativeImplicit Class Structure “I like the small classes – it is a luxury to have all

the attention.” (2008) Students need to be aware and there be

consequences for not attending classes. (2006)

Remember issues of working students and part-time students. (2009)

There were challenges in handling disrespectful students in class and the difficulties created for other students. (2010)

Problems with only 10 minutes between classes in some cases. (2010)

Implicit Diversity More focus on diversity issues. (2006) More content on the role of religion in

practice as an issue of culture of diversity. (2009)

Implicit Field 600 hours in the field is a lot- now that it is over, it’s not so bad. (2005)

Several saw the value of a community practicum project as part of the field sequence. (2006)

600 hours for field is appropriate – “we really need it.” (2008)

Valued course work – “Field is the icing on the cake.” (2008)

“600 hours of field is priceless.” (2008) I am glad the Program requires more field hours

than the minimum. (2009) Organization of the field was “amazing” – made

it easier – it was “professional and well executed.” (2010)

Only have field in the senior year; difficult to be in agencies only M, W, F. (2005)

Issues for hospital placements and the pace of the work; possibility of block placements? (2006)

Stress to field instructors the importance of supervision on an ongoing basis. (2006)

For a hospital setting, perhaps one semester on one floor and then change for the second semester – would provide a more concentrated experience. (2008)

Terminate earlier from the field in the spring in order to have time to finish other work. (2008)

M, W, F is hard at some placements –makes a one semester, full time placement have some appeal. (2009)

Allow students in the field to have 10 additional hours in the fall to cover for any problems in spring. (2009)

Careful about too many cases being assigned at the beginning of placement. (2010)

Some time should be available in field for collection of information for the Comprehensive Review and course assignments – perhaps a couple of hours per week. (2010)

Information should be provided to field instructors and higher level supervisors concerning what students should be doing on their own by what point in the semesters: Students doing interviews on their own; “By this time in the semester, the student should…” (2010)

There should be a meeting of the field faculty at the beginning of the spring semester as there was for the fall semester. (2010)

Concerns about placing students in WV DHHR. (2005)

Some sections of the field evaluation have inapplicable sections- i.e. theory application-field faculty don’t focus on this. (2006)

Caseloads are too high at the end of the semester – conflicts with school work. (2007)

There is a discontinuity in the agency experience due to placements being on Monday, Wednesday, and Friday. (2007)

Field only M, W, F – changing the mind set from class to field was challenging. (2008)

Type Topic Positive Suggested Innovations Negative

Explicit Assignments Four process recordings are okay, two each semester.” (2004)

More role play would be useful with hostile clients. (2005)

“A lot of writing; some repetitious assignments.” (2004)

Explicit Community Service Learning

Community Service Learning course should have more than 55 hours in placement. (2010)

Explicit Comp Review Working on the competency review throughout the year is good; however, much needs to be rewritten. (2007)

Doing the comprehensive reviews sections by sections is good – makes demands on seniors to complete in a more timely way. (2009)

Comprehensive Review is thorough and beneficial in understanding placement. (2010)

Concerning the comprehensive review: put section 3 first. (2006)

Would like to have more feedback on the sections of the review – perhaps on sections 5 and 10, as opposed to 6, 7, 8, and 9, which is covered in case presentations. (2007)

Make Comprehensive Review due earlier to get feedback from supervisor – articulate instructions to class earlier – supervisor’s workload. (2010)

“The comprehensive review and the research methods papers compete for my time.” (2004)

Explicit Content In classes, talk about boundaries in relation to colleagues; not just clients. (2009)

Would like a special topics class on domestic violence. (2009)

More content on the DSM would be helpful – basic knowledge and content. (2009)

Would like to see more macro vs. micro content – esp. advocacy and policy. (2009)

Explicit Curriculum Order of curriculum is useful; old books came in handy. (2007)

The curriculum sets you up well to complete the Program. (2009)

Sequencing of courses works. (2010)

Have two semesters of interviewing. (2006)

More information on community organizing. (2006)

Consider requiring Psychological Tests and Measurements course to provide content in interpreting psychological tests. (2006)

“Following the curriculum in order would have provided a better experience.” (2004)

Combined Field Evaluation and Learning Agreement

Field evaluation revised based on new EPAS with input of field faculty

Second revision based on discussion with the Mid-Atlantic Field Consortium

Learning agreement incorporated into evaluation form

Brainstorming sessions with field faculty to devise examples of measurable learning activities

Process unfolded over 2 years

Name of Intern: MB Field Instructor: __ES__

Agency: Children’s Home Semester: Midterm Fall 2011

Directions:The learning plan portion of the agreement is to be completed by the intern and the field instructor. After the intern enters the information, the learning plan is to be signed by the intern, the Field Instructor, and the Field Liaison.

Learning Activity Feedback:The field instructor will provide feedback on each of the learning activities at the mid-term and end of each semester. The field instructor will enter a number in the appropriate column to the right of each activity in accordance with the following scale:4 Student consistently performs according to expectations in performing measurable learning activity 3 Student generally meets expectations in this area in performing learning activity2 Student has not as yet met expectations in this area and still requires guidance and direction. 1 Student has not met the expectations in this area and falls below expected performance0 There has been no opportunity for the student to demonstrate competence in this area by the date of this evaluation

Mid-term Evaluation:For the mid-term evaluation, the field instructor is only required to provide an overall score for each competency. Individual practice behaviors should be scored only to note a particular strength or concern about the intern’s performance.

Final Evaluation:For the final evaluation, the field instructor should rate each practice behavior. The overall final evaluation score for each competency will be the practice behavior score average.

COMPETENCY ONE:Identify as a professional social worker and conduct oneself accordingly

Practice Behavior Learning Agreement :Measurable Learning Activities

Areas of Strength or Concern atMid Term(0-4)

Rate Level of CompetencyFinal(0-4)

1. Advocate for client access to the services of social work.

1.1 Train and recruit foster parents. Also work as a case manager

2. Practice personal reflection and self-correction to assure continual professional development.

1.2 Write daily journals where I will examine my own responses in the work field

3. Attend to professional roles and boundaries.

1.3 Understand my limits as a Social Work intern and gain more knowledge of the rights of foster parents, children, and their biological families

4. Demonstrate professional demeanor in behavior, appearance, and communication.

1.4 Adhere to dress codes, agency policies, confidentiality, attending meetings, arrive on time, and perform my duties as an intern

5. Engage in career-long learning. 1.5 Engage in organization luncheons, meetings, and staff trainings

6. Use supervision and consultation. 1.6 Prepare own agenda for weekly supervision with field instructor

Overall Rating at Mid Term Overall, how has the intern performed in thiscompetency at the mid-term? MB is an eager learner.She demonstrates good work skills and attentivenessto detail.

Mid Term4

Overall Rating at Final Evaluation Overall, how has the intern performed in thiscompetency at the final?

Final Evaluation

Narrative Comments (optional):

COMPETENCY TWO:Apply social work ethical principles to guide professional practice

Practice Behaviors Learning Agreement :Measurable Learning Activities


Rate level of CompetencyFinal(0-4)

Recognize and manage personal values in a way that allows professional values to guide practice.

2.1 Practicing differences in religious practices and not to push my own values on others

Make ethical decisions by applying standards of NASW Code of Ethics

2.2 Read the code of ethics on or before Nov. 1, 2011

Tolerate ambiguity in resolving ethical conflicts 2.3 Gain an understanding of the complex nature of the child welfare system and that there are a lot of gray areas

Apply strategies of ethical reasoning to arrive at principled decisions.

2.4 Apply ethical reasoning to a case study

Overall Competency Rating at Mid Term Overall, how has the intern performed in this competency at the mid-term? MB is demonstrating good active listening skills as she learns policies and procedures. She asked great questions for greater understanding.

Mid Term4

Overall Competency Rating at Final Overall, how has the intern performed in this competency at the final?

Final Total


COMPETENCY THREE:Apply critical thinking to inform and communicate professional judgments



Rate level of CompetencyFinal (0-4)

Distinguish, appraise, and integrate multiple sources of knowledge, including research-based knowledge, and practice wisdom.

3.1 Within my case presentation, use scholarly literature and articles to complete the assessment

Analyze models of assessment, prevention, intervention, and evaluation.

3.2 Research scholarly websites, articles, and complete case study

Demonstrate effective oral and written communication in working with individuals, families, groups, organizations, communities, and colleagues.

3.3 Completion of a process-recording

Overall Competency Rating at Mid Term Overall, how has the intern performed in this competency at the mid-term? This is an area that will develop with more time and experience.

Mid Term3

Overall Competency Rating at Final Overall, how has the intern performed in this competency at the final? Final Evaluation


COMPETENCY FOUR:Engage diversity and difference in practice




14. Recognize the extent to which a culture’s structures and values may oppress, marginalize, alienate, or create or enhance privilege and power.

4.1 Comprehend healthcare with diverse paying such as insurance, self-pay, and medicare/medicaid; realizing those with privilege, power, and higher socio-economics receive more options in ascertaining healthcare

2 3

15. Gain sufficient self-awareness to eliminate the influence of personal biases and values in working with diverse groups.

4.2 Separate personal vs. professional biases, realizing judgements and biases I have; and not allowing them to influence my work

4 4

16. Recognize and communicate their understanding of the importance of difference in shaping life experiences.

4.3 Realize the difference in perspective between myself, colleagues, and clients in ideals of treatment 4 4

17. View themselves as learners and engage those with whom they work as informants.

4.4 Complete at least 20 level ones by end of first semesterComplete at least 1 bio psycho socialComplete 1 patient’s complete treatment from level 1 to discharge by thanksgiving

Total3

Total4

Overall Competence at Mid Term Overall, how has the intern performed in this competency at the mid-term?

Mid Term

Overall Competency at Final Evaluation Overall, how has the intern performed in this competency at the final? 4 Final Evaluation

Narrative Comments (optional): SB is able to independently complete initial screens, discharge planning assessments, and Psychosocial Assessments. He continues to express interest in becoming more independent in his role as an intern.

COMPETENCY TEN: Engage, assess, intervene and evaluate with individuals, families, groups, organizations and communities.




Substantively and affectively prepare for action with individuals, families, groups, organizations, and communities.

10.1 Prior to seeing patients make sure to view meditech info to give myself a good background knowledge

3 4

Use empathy and other interpersonal skills. 10.2 Use Method I skills in communication with clients, especially in starting where the client is at

3 4

Develop a mutually agreed-on focus of work and desired outcomes

10.3 Co-facilitate at least one task-oriented group on the psych floor with Lynda Artusio. N/A N/A

Collect, organize and interpret client data. 10.4 Use meditech and round information to know what is best care for patient 3 4

Assess client strengths and limitations. 10.5 Use admit history and PT/OT/ST recommendations, as well as client’s own reflections to analyze mental, economic, bio/health, and social strengths/limitations

3 4

Develop mutually agreed-on intervention goals and objectives.

10.6 Complete at least 2 discharge planning assessments/summaries that have been agreed upon by patient or patient’s family and involved active dialogue

2 4

Select appropriate intervention strategies. 10.7 Learn to differentiate needs between SNF, ALF, HHC and involvement with Medicare, Self-Pay, Insurance, and Medicaid

2 4

Initiate actions to achieve organizational goals. 10.8N/A N/A

Implement prevention interventions that enhance client capacities.

10.9N/A N/A

Help clients resolve problems. 10.10 Discuss with patient’s recommendations from hospital for discharge, and listen to client’s concerns and needs

2 4

Negotiate, mediate, and advocate for clients. 10.11 Interact in rounds with nursing, PT/OT/ST, hospitalists, etc. to inform what from case management perspective needs to happen

N/A 3

Facilitate transitions and endings. 10.12 Keep community resources like HHC, SNF’s, ALF’s updated when opening a patient or when patient is considering going to a facilityArranging transportation for patient’s when necessary

23

Critically analyze, monitor, and evaluate interventions.

10.13 Using logs and supervision to process the client treatment and discharge planning as needed 3 4

Overall Competency Rating at Mid Term Overall, how has the intern performed in this competency at the mid-term?

Mid Term

Overall Competency Rating at Final Evaluation Overall, how has the intern performed in this competency at the final?

4 Final Evaluation

Narrative Comments (optional): SB will continue to independently assess patient’s needs on admission and follow throughout the hospital course. SB has learned the Meditech database and appropriately demonstrates his knowledge. SB documents accurately after meeting with his patients with minimal assistance. Field Liaison goes over SB’s documentation daily and makes any necessary corrections. SB will be starting to co-facilitate groups in the FMH Behavioral Health Unit on 12/9/11.