Upload
polly-eaton
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Project ACCESSMeasuring Transition-Related Transition Indicators Among High School Students with Disabilities
Christopher Murray, Clayton Rees, Mimi McGrath-Kato, Jeri Dickinson
Oregon Office of Vocational Rehabilitation Services (OVRS) and The University of Oregon
Funded by U. S. Department of Education, Office of Special Education and Rehabilitative Services (OSERS)
Need For Project in Oregon
0
5,000
10,000
15,000
20,000
25,000
Adolescents withDisabilities
Adolescents withDisabilities in YTP
Number of Youth in OR
Other Needs
Majority of students who do participate in YTP receive services in Grades 11 and 12—our goal is to begin developing awareness of transition-related needs in 9th and 10th grades.
Special Educators and Vocational Rehabilitation Counselors both have strengths and unique skills. Our goal is to bring those skills together to provide students with a unique and meaningful experience.
To provide these experiences to a greater number of students—by targeting groups of students rather than individuals.
Overall Model Structure
State TeamInforms Local Team and
Sites of State Level ChangesAssist with Dissemination and Problem Solving
Local TeamProvides Direct Feedback to sites
Provides Feedback to State Team Regarding Implementation
SitesThree High Schools
Two Alternative SchoolsDevelop and Implement Model
Program
Participating School Districts
Springfield Public Schools--Semi-Urban
South Lane School District--Semi Suburban
McKenzie School District—Rural
We are attempting to develop models that can be applied in a variety of settings throughout the state.
Major Program Features To develop and implement a transition-related program for all students with disabilities in five
sites.
Place a VR Counselor in site 1 day per week (Year 1) to work directly with special education teachers developing and implementing model program. Time spent in schools should increase over project.
Teachers and VR counselors plan a curriculum/sequence of experiences for what information should be provided to students at grades 9, 10, 11, and 12 to assist them in making positive post-school transitions.
Working with families of students with disabilities, develop training model and provide parents with training.
This will be developed and implemented gradually to ensure program integrity…….
Year 1 Year 2 Year 3 Year 4 Year 5
Grade 9 Plan & Implement
Implement Implement Implement Implement
Grade 10 Plan & Implement
Implement Implement Implement
Grade 11 Plan & Implement
Implement Implement
Grade 12 Plan & implement
Implement
Step 1: Site-Based Services
Bring teachers and VR Counselors together at each site to develop curriculum that will be delivered to 9th grade students in Year 1.
We will provide “buy out” either through teacher extended day or substitute buy out for teacher planning.
We provide VR with funding to allow VRC’s to engage in the process of planning.
Next year VR counselors will spend one day per week at sites
One Example
Our Ideal Situation
VR Counselors are working with teachers. Providing co-instruction to students on the day when they are
there. We view this as an opportunity for VR counselors to provide their
insights regarding transition needs to students and teachers. We also believe that VR counselors will benefit and learn about
school processes that are designed to support the transition-needs of students.
VR counselors are getting to know students. This will provide counselors with better information for determining
which students could benefit from VR services. Teachers will continue to provide information to students as
much as possible when VR counselors are not in the school. This will help to ensure improvements in the delivery of transition-
related curricula and skills
Assessment process
Interested primarily on effects for students Want to have a standard assessment process across schools Want to have measures that are sensitive to change based on
the types of activities that are taking place in the schools. Want to have an assessment that measures the types of skills
that students will be learning in 9th and 10th grades Want assessments that are reliable and valid.
Meaning (a) internally consistent—reliable range from 0 - .99/1.0 with higher
being better (b) Valid in the sense that the measures are measuring the
constructs they purport to measure AND that the constructs are predictive of important post-school outcomes.
What we measure
Phase 1—baseline assessment of students, 2X per year The Career Locus of Control Scale Vocational Skills Self-Efficacy Problem Solving Inventory-Revised Outcome Expectations Scale Skills Rating System (Student Rating, Grades 7-12) Inventory of Teacher-Student Relationships School Bonding School Engagement (Behavioral Engagement) School Engagement (Cognitive Engagement) Employment Status Type of Employment Earnings
Assessing Functional Skills
Long-term Post-School Outcomes vs. outcomes that you get to see in school—intermediary.
Pathways of Influence
Annual GoalsChild Skills Intermediary Postsecondary
Indicators/ OutcomeFunctional Skill
How do you determine whether an intermediary indicator of skills is a predictor of later outcomes?
Phase 2—School Records, 1 X per year Race Gender Age (DOB) Grade level Disability Type Academic Achieve Grades Attendance Dropout status Diploma Type
Phase 2 Continued Teacher Data
Ratings on amount of curriculum covered, 3X per year Student Attendance in course devoted to ACCESS content
VR Data Ratings on amount of curriculum covered, 3X per year Number of referrals
School Demographics, 1 X per Year
What we measure
Phase 1—baseline assessment of students, 2X per year The Career Locus of Control Scale Vocational Skills Self-Efficacy Problem Solving Inventory-Revised Outcome Expectations Scale Skills Rating System (Student Rating, Grades 7-12) Inventory of Teacher-Student Relationships School Bonding School Engagement (Behavioral Engagement) School Engagement (Cognitive Engagement) Employment Status Type of Employment Earnings
The Career Locus of Control ScaleMillar, R. & Shelvin, M. (2007). The development and factor structure of a
career locus of control scale for use with school pupils. Journal of Career
Development, 33, 224-249.
Contains two factors, 1 = Strongly Disagree to 4 = Strongly Agree
Luck, 5 items Sample: “Luck matters more than careful planning in
determining whether I will be successful getting the job I want.
Internality, 5 items Sample “I believe through my own efforts I will be able
to be anything I choose”
Vocational Skills Self-Efficacy McWhirter, E. H., Raseed, S. & Crothers, M. (2000). The effects of highschool career education on social-cognitive variables. Journal of Counseling Psychology, 47, 330-341.
Contains one factor, 1 = No Confidence to 5 = Complete Confidence
Voc Skills Self-Efficacy, 36 items Samples: “How confident are you that you
could list four jobs in my general area of interest.
How confident are you that you could identify three of my strengths as a person.
Problem Solving Inventory-RevisedJaffe, E. B., & D’Zurilla, T. J. (2003). Adolescent problem solving, parentproblem solving, and externalizing behavior in adolescents. Behavior Therapy,34, 295-311
Contains five factors, 1 = Not At All True to 5 = Extremely True Positive problem orientation, 5 items
Sample: “When I have a problem, I try to see it as a challenge or opportunity to benefit from the problem in some positive way”
Negative problem orientation, 5 items “I feel nervous and unsure of myself when I have an important decision to make”
Rational problem Solving, 5 items “When I have a problem to solve, one of the first things I do is get as many facts
about the problem as possible” Impulsivity/Carelessness, 5 items
“When I am trying to solve a problem, I go with the first good idea that comes to mind”
Avoidance Style, 5 items “When a problem occurs in my life, I put off trying to solve it for as long as
possible”
Career Outcome Expectations ScaleMcWhirter, E. H., Raseed, S. & Crothers, M. (2000). The effects of high
school career education on social-cognitive variables. Journal of
Counseling Psychology, 47, 330-341.
Contains one factor, 1 = Not At All True to 5 = Extremely True
Outcome Expectations, 6 items Sample: “I will be successful in my chosen
career/occupation” Sample: My career planning will lead to a
satisfying career for me.
Social Skills Rating System (Student Rating, Grades 7-12)Gresham, F. M. & Elliot, S.N. (1990). Social Skills Rating System: Manual.
Circle pines, MN: American Guidance Service. Contains 4 Factors: 1 = Never to 3 = Very Often
Cooperation, 10 items Sample: “I listen to adults when they are talking to me”
Assertion, 10 items Sample: “I make friends easy”
Empathy, 10 Items Sample: “I try to understand how my friends feel when they
are angry, upset, or sad” Self-Control, 10 Items
Sample: “I ignore other students when the tease me or call me names”
Other Measures
Teacher-Student Relationships Communication, 8 items Trust, 5 items Alienation, 6 items
School Bonding, 7 items Behavioral Engagement in School, 5 items Cognitive Engagement in School, 10 items
Results: This assessment was administered to 245 students in Grades 9-12 from 5 high schools.
LD30%
OHI1%
ED9%
MMR4%AUT
4%CD1%
DD0%
VI1%
No Disability50%
Gender & Race
51.6 48
0102030405060708090
100
Male Female
Gender
64.3
28 6
116
0102030405060708090
100
Race
White
Black
Asian
Latino
NativeAmericanMultiracial
Other
Other Demographics
0
10
20
30
40
50
60
70
80
90
100
0
10
20
30
40
50
60
70
80
90
9th 10th 11th 12th
Internal Consistency Reliability of measures The Career Locus of Control
Scale Luck alpha = .83 Internality alpha = .76
Vocational Skills Self-Efficacy Alpha = .96
Solving Inventory-Revised PPO = .78, NPO = .76, RPS = .82,
Imp = .77, Avoid = .73 Outcome Expectations Scale
Alpha = .87 Skills Rating System (Student
Rating, Grades 7-12) COOP = .77, Ass = .75, Em = .80,
SC = .78
Inventory of Teacher-Student Relationships Comm = .88, T = .85, Al = .73
School Bonding Alpha = .82
School Engagement (Behavioral Engagement) Alpha = .82
School Engagement (Cognitive Engagement Alpha = .88
Criterion Related Concurrent Validity
Results of ANOVA’s Comparing Students Without Disabilities (ND) and Students With Disabilities (SWD) The Career Locus of Control Scale Significance
Luck ND < SWD, p < .001 Internality --------------
Vocational Skills Self-Efficacy ND > SWD p < .05
Problem Solving Inventory-Revised Positive Problem Orientation --------------- Negative Problem Orientation ND < SWD p < .001 Rational Problem Solving --------------- Impulsive/Careless ND < SWD p < .001 Avoidance ND < SWD p < .001
Outcome Expectations Scale ---------------
Social Skills Rating System (Student Rating, Grades 7-12) Cooperation --------------- Assertion --------------- Empathy --------------- Self-Control ND > SWD p < .05
Results of ANOVA’s Comparing Students Without Disabilities (ND) and Students With Disabilities (SWD)
IT-SR Significance Communication ND < SWD p < .05 Trust -------------- Alienation ND < SWD p < .05
School Bonding --------------
School Engage (Behavioral) --------------
School Engagement (Cognitive) ND > SWD p < .05
Tentative Uses
The underlying psychometric qualities of these instruments appear reliable and some evidence of validity.
Although many of the instruments are published and copyrighted, some are publicly available Vocational Skills Self-Efficacy Career Outcome Expectations Scale
Goals
Goals are tied directly to the results of assessment (or) current levels of performance and they are: Outcome Oriented Specify What, Specify by When, and Specify Measurable Criteria Goals do not specify what services will happen—that’s what the
rest of the IEP is for!
Important Issues In Determining Observable/ Measurable If the assessment contains norms you can look at standard
scores, percentile rank, on some metric (grade equivalency) when setting the goal.
If the assessment does not have norms, as is the case here, may to use another strategy such as finding the range of possible points and then setting a goal for growth.
Goals Continued
Example: By March 1, 2010 Jacob will improve his vocational self-efficacy skills from a score of 68/180 (38%) to 100/180 (55%) as measured by child-report on the Vocational Skills Self-Efficacy Scale.
Weak example: Jacob will improve his Vocational Skills Awareness.
Objectives
Ideally, objectives are tied directly to goals and are short-term monitoring mechanisms that allow you to determine whether the overall goal is being met. These are intermediate performance steps that
can be accomplished in two ways Break down the skills in the objective into discrete
components or skills, OR Describe the amount of progress within specific
segments of the yearly goal using the goal as a guide.
Objectives continued
Steps: 1) Split the year into three (or more) parts2) Take the overall difference in performance
specified in the goal and divide it into three parts. Present level = 68, Goal level = 100100 – 68 = 32, 32/3 = 11
1) Specify and list the performance expected at each objective time point
2) List the measurement that will be used to monitor the progress.
Objectives Continued
By March 1, 2010 Jacob will improve his vocational self-efficacy skills from a score of 68/180 (38%) to 100/180 (55%) as measured by child-report on the Vocational Skills Self-Efficacy Scale.
Ob1: By June 1, 2009 Jacob will improve his vocational self-efficacy skills by 11 points as measured by VSSE
Obj. 2: By November 1, 2009 Jacob will improve his vocational skills self-efficacy by 22 points as measured by VSSE
Obj. 3: By March 1, 2010 Jacob will increase his vocational skills self efficacy score by 32 points as measured by VSSE.
How can you measure progress toward goal? Give the measure multiple times (Practice Effects?) Divide large measures into 3 random parts—
meaning divide everything by three! 36 items / 3 = 12 randomly chosen items.
Now take the old range of scores, 36 to 180 and divide both by 3 to get new range of scores for each of your three tests.
36/3 = 12 (lowest possible), 180/3 = 60 (highest possible) If you wanted an 11 point increase at each objective time
point on the overall measure, divide 11/3 to get objective the new objective growth rate on each abbreviated test, = 3.6 or increase of 4 points at each time point.
Objectives Continued
By March 1, 2010 Jacob will improve his vocational self-efficacy skills from a score of 68/180 (38%) to 100/180 (55%) as measured by child-report on the Vocational Skills Self-Efficacy Scale.
Ob1: By June 1, 2009 Jacob will improve his vocational self-efficacy skills from 23 to 27 overall (4 points) on a 12 randomly selected items on the VSSE
Obj. 2: By November 1, 2009 Jacob will improve his vocational self-efficacy skills from 27 to 31 overall (4 points) on a 12 randomly selected items on the VSSE
Obj. 3: By March 1, 2010 Jacob will improve his vocational self-efficacy skills from 31 to 35 overall (4 points) on a 12 randomly selected items on the VSSE.