JIM MARTIN & AMBER MCCONNELL
UNIVERSITY OF OKLAHOMADEPT OF ED PSYCHOLOGY
ZARROW CENTER
Transition Assessment and Goal Generator (TAGG): A New On-
Line Assessment
Quality Transition Education Produces Greater Outcomes
Transition Education Begins With Transition Assessment
Illustration of two high school graduates. One on large horse and student suited in armor. This student had quality transition education while in school. The other student in on a stick-horse going in the wrong direction. He did not have quality transition education.
Agenda
Purpose of Transition AssessmentUse results to build postsecondary
and annual transition goalsQuestions to Ask to Pick a Transition
AssessmentIs this a good transition assessment
to use?Importance of Annual Transition
GoalsThe TAGG
Purpose of Transition Assessments
Questions Students Need to Use Results to Build Postsecondary Goals
Postsecondary Goal QuestionsUpon graduation from high school:Where do I want to live?Where do I want to work?Where do I want to learn to do the
job I want?
Questions for Annual Transition Goals
What do I need to learn now:To live where I want after graduating
from high school?To do the job I want after graduating
from high school?To learn where I want after
graduating from high school?
A Few Questions to Ask to Help Pick a Transition Assessment
Validity Evidence
Test developer needs to provide basic validity evidence
User needs to examine evidence and assessment to determine if it is appropriate for students and intended purpose
Importance Factor
The more important decision, the more validity results need to be
How valid and reliable are the results used to make transition planning decisions?
Often not easy to find outDuties . . .
Sources of Validity Evidence1. Item Development2. Items Structured and Organized3. Internal Reliability4. Stability of Scores (Test-Retest)5. Fairness Validity6. Match with other similar
assessments: Concurrent Validity7. Purpose: Does it fit for your uses?
DevelopmentHow were the items developed?
Did the authors just sit down and make them up?
Did the authors ask knowledgeable sources?Did the authors use findings from research
with students with disabilities?Are they based on what students need?Are items developed, then clustered into
similar groups?Are cluster definitions defined first then
items developed?
More Item Questions
How are the items structured and organized?Based on common sense categories?Based upon statistical analyses of
user responses to verify theory behind assessment?
ReliabilityDoes evidence of acceptable internal consistency scores exist?Cronbach’s alpha (.7 or higher)
Does evidence exists that when same users first completed the tool, then completed it again several weeks later, the results are similar?Does evidence of good test-retest reliability exist?
Validity
If the assessment fair? Does evidence exist that no major differences exist acrossRaceEthnic groupsGenderSES
Does evidence exist that the assessment results match those from another assessment that examines the same concept?Correlation between Weschler IQ and
Standford Binet IQ results
Importance Factor
The more important decision, the more validity evidence needed
Often not easy to discover validity evidence
Determining validity is up to the test developers and users
Example of What Can Be Learned
The Career Clusters Interest SurveyFreeSupposed to Identify Interests by the
16 career clustersEasy to use and scoreMany educators use it So, study by Prime and Tracey (2010)
. . . . ..
Study Results
The CCIS is reliable, but not validIt measures well time after time, but
The results are very limited in terms of what is being measured
Doesn’t match-up well with another assessment that measures the same concepts
“Its usage could provide a severely restricted set of options . . .”
Process & Decisions None Low Moderate High
Awareness X
Exploration X
PLEP X X
Strengths X X
Needs/Limits X X
Postsecondary Goals X X
Annual Transition Goals X X
Acceptable Degree of ValidityChart of IEP stages by the level of validity evidence – none, low, moderate, or high
TRANSITION ASSESSMENT AND GOAL GENERATOR
DEVELOPED WITH A GRANT FROM THE NATIONAL CENTER FOR
SPECIAL EDUCATION RESEARCH AND ZARROW CENTER FUNDS
The TAGG
TAGG Overview
An on-line transition assessment for secondary-aged students with disabilities, their educators, and parents designed to provide non-academic annual transition goals to enable students to learn skills to increase likelihood of employment and further education
TAGG Will Provide
The TAGG will provide A graphic profileWritten summary to copy and paste
into PLEP section of the IEPListing of relative strengthsListing of relative needsAnnual transition goal suggestions
matched to needs
Annual Transition Goal Questions the TAGG Can Answer
What do I need to learn now:To do the job I want after graduating
from high school?To learn where I want to learn after graduating from high school?
How Were the TAGG Items Developed?
The research team found research studies that identified behaviors of former high school students with disabilities associated with post high school employment or further education
The research team developed 10 construct definitions
Items developed from constructsOver three years, almost 1,500 students, 130
educators, and 800 parents from across the country completed the TAGG.
Initial Structure: Ten Initial Constructs Knowledge of
strengths and limitations
Actions related to strengths and limitations
Disability awarenessEmploymentGoal setting and
attainment
PersistenceProactive
involvementSelf-advocacySupportsUtilization of
resources
Sample Construct Definition
Disability AwarenessSuccessful individuals know they have a disability and can express needs to others in a non-stigmatizing manner. Individuals demonstrate knowledge of the disability and can express positive and negative aspects. They express information such as how the disability affects life and what supports are needed and legally allowed to compensate in various situations. The student needs to be able to place the disability within the context of his or her life and is not defined by the disability.
Year 1 and Efforts to Build TAGG Structure
Users completed the TAGGApplied various factor analyses
statisticsWent from 10 constructs to 8Went from 75 items to 34Construct structure confirmed by
second study with year 2 data
Professional and Family Constructs After Year 1 Analysis
Stayed1.Strengths and
Limitation2.Disability Awareness3.Persistence4.Interacting with Others5.Goal Setting and
Attainment6.Employment7.Student Involvement in
IEP8.Support Community
Dropped1.Action Related to
Strengths and Limitation
2.Utilization of Resources
TAGG-P: (c2=1043.62, df=499, RMSEA=.058, CFI=.92, TLI=.91,
RMSR=.0597)TAGG-F: (c2=862.74, df=499, RMSEA=.057,
CFI=.91, TLI=.90, RMSR=.058)
Student Version Constructs After CFA
After CFA Constructs
1.Strengths and Limitations & Support Community
2.Disability Awareness
3.Persistence
4.Student Involvement in IEP
5.Interacting with Others
6.Goal Setting and Attainment
7.Employment
Dropped Construct1.Action Related to
Strengths and Limitation
2.Utilization of Resources
Combined Constructs3.Strengths and
Limitations
4.Support Community
TAGG-S: (c2=819.00, df=505, RMSEA=.047, CFI=.89, TLI=.88, RMSR=.064)
Internal Reliability
Does evidence exist that across the TAGG various items respond in the same way?Each TAGG version has great overall
internal consistency and satisfactory subscale consistency (ranging from α =. 89 to α =. 95)
Test-Retest Reliability
Does evidence exists that the results of same users completed once, and then again several weeks later, produce similar scores?14 weeks after the first TAGG completion
a large correlation was found with the second administration for all three version .80 for professional TAGG .70 for family TAGG .70 for student TAGG
Fairness Validity Evidence
Do differences exist by gender and racial groups?No overall differenceDifferences with employment construct
Do differences exist by SES?No overall difference by SES
Match Across Versions Validity Evidence
How closely do the different versions assess?TTAGG has medium correlations,
which is excellent for an assessment of this type
Concurrent Validity Studies
Match with TAGG and AIR Self-Determination AssessmentUnderway – data collected
Match with TAGG and Employer Identified Trait AssessmentUnderway – data collected
You Decide . . .Does the TAGG Fit Your Use?
Based on the given validity evidenceDoes the TAGG has ample evidence to
support how you want to use the results to build the PLAAFP and annual transition goals?
On-Line TAGG
The on-line TAGG will go live to the public before the end of this year
It will look something like this: http://zarrowcenter.ou.edu/taggp4/
TAGG Professional Version sample questions for the strengths and limitations construct. 1. The student told someone
what he or she does well.2. The student told someone
what he or she has trouble doing.
Sample Disability Awareness construct questions.1. The student expressed the
type of support or accommodations needed for his or her disability.
Sample items for the persistence construct.1. The student views not giving up
in school as important.2. The student keeps working to
achieve a goal, even when it becomes hard.
Sample items for the Interacting with Others construct.1. The student successfully participates in small groups to complete projects.
Sample items for the goal setting and attainment construct.1. The student creates short-term goals to attain long-term goals.2. The student adjusts plans to attain goals if they do not work.
Sample items for the employment construct.1. The student expresses wanting a job.2. The student had an unpaid job, such as
working for a family member.3. The had a paid job.
Sample items for the involvement in the IEP construct.1. The student told the IEP team
his or her postschool goals2. The student discussed his or
her present level of performance at the IEP meeting.
Sample items for the support community construct.1. The student accepts help
from support people when offered.
2. The student seeks assistance from community agencies.
Disability Awareness Profile
Sample disability awareness profile bar graph of results for the student, parent, and teacher TAGG versions. The graph is present on a 0 to 9 stanine profile.
Combined Score Profile
Sample profile bar graph of overall results for the student, parent, and teacher users. The graph is present on a 0 to 9 stanine profile.
Greatest and Relative Strengths
Summary of greatest strengths and areas of relative strengths, along with area of greatest need by student, family, and teacher user.
Areas of Greatest and Relative Need
Summary of greatest need, areas of relative need, and a summary of the overall TAGG results. Each can be copied and pasted into the IEP.
Summary Statement for IEP
Chad Bailey’s skills were assessed using the TAGG, a norm-referenced assessment with research-based items know to be associated with post-school employment and education. Compared to similar students Chad’s scores are average. Results indicate greatest strengths are in the areas of Goal Setting and Attainment. Chad’s relative strengths include Disability Awareness and Student Involvement in the IEP. Greatest needs are in the area of Strengths and Limitations, with Employment being a relative need.
Suggested Annual Transition Goals
To prepare for success in employment, the student will write an essay describing three situations where the student used his or her strengths with 90% accuracy in the areas of grammar and context by the end of the essay writing unit. Listing of Common Core Standards match to an annual goal. These are presented by grade for difference English benchmarks.
On-Going Studies
Continue additional validity studies, including SES and TAGG results
Follow group of students from high school into adult life
Determine relation between high school TAGG profile and adult outcomes
Establish criterion levels for constructsProduce profile with prioritized annual
transition goals based upon student’s profile and norm group outcomes
TAGG Availability
By the end of this fall semesterCan be found at the Zarrow Center web site
Google Zarrow CenterMinimal Cost
$3 per set (Professional, Student, Parent versions)
Minimum order 5 setsFunds used to keep TAGG on-line and
updated