Upload
dorian-lansing
View
224
Download
0
Tags:
Embed Size (px)
Citation preview
Standardized Scales
Standardization
Use of identical procedures to collect, score, interpret, and report results of a measure
Assures that differences over time or among different people are due to the variable being measured and not to different measurement procedures
What are Standardized Scales? Set of uniform procedures to collect,
score, interpret, and report numerical results
Usually have norms and empirical evidence of reliability and validity
Typically include multiple items aggregated into one or more composite scores
Frequently used to measure constructs
Construct
Complex concept (e.g., intelligence, well-being, depression)
Inferred or derived from a set of interrelated attributes (e.g., behaviors, experiences, subjective states, attitudes) of people, objects, or events
Typically embedded in a theory Oftentimes not directly observable but
measured using multiple indicators
Evaluating and Selecting Standardized Scales Purpose Reference populations and normative
groups Reliability Validity Practical considerations
Purpose
Identify whether or not a client has a significant problem
Measure and monitor your client’s outcomes to determine if your client is making satisfactory progress
Reference Population
Population of people for which a measure is intended and from which a normative group is sampled and norms are created
Normative Group
Representative sample of a reference population, used to estimate norms for that population and, more generally, used to develop and test standardized measures
Also known as a “standardization group” or “standardization sample”Population
Sample
Reliability
Internal consistency reliability (coefficient alpha) (most important)
Interrater rater reliability (sometimes) Test-retest reliability
Validity
Face Content Criterion Construct
Sensitivity to change especially important
Practical Considerations
Time Effort Training Cost Availability Acceptability (e.g., clients, practitioners,
etc.)
Decisions, Decisions…
Who Where When How often to
collect outcomedata
Who
Client Practitioner Relevant others Independent evaluators
Where and When
Private, quiet, physically comfortable location
Complete at about the same time and under the same conditions on a regular basis
How Often
Regular, frequent, pre-designated intervals
Often enough to detect significant changes in the problem, but not so often that it becomes problematic
In general about once per week
Engage and Prepare Clients
Be certain the client understands and accepts the value and purpose of monitoring progress
Discuss confidentiality Present measures with confidence Don’t ask for info the client
can’t provide
Engage and Prepare Clients (cont’d) Be sure the client is prepared Be careful how you respond to
information Use the information that is collected Be careful how you respond to
information Use the information that is collected
Administering, Scoring, and Interpreting Standardized Scales Score, scoring formula, composite score Unidimensional and multidimensional
scales Cut scores Reverse-worded items Reliable change, reliable improvement,
reliable deterioration Clinically significant improvement Expected treatment response
Score
Generic term for a number derived from a measure that represents the quantity or amount of an attribute or observation (e.g., number of times a behavior is observed, value obtained from a standardized scale)
Interpret in context of all available quantitative and qualitative information
Scoring
Procedure by which data from a measure are used to produce a score (e.g., number of times a behavior occurs or value on a standardized scale) or category (e.g., diagnostic category)
Scoring Formula
A mathematical rule by which data from a measure are used to produce a score (e.g., sum or average of responses to items on a multi-item standardized scale)
Item 1 Item 2 Item 3 Score
Composite Score
Score that combines results from two or more related items or other measures using a specified formula (e.g. percentage of items answered correctly on a statistics test)
Score
Item 3
Item 2Item 1
Unidimensional Scale
Scale that measures a single attribute or construct (e.g., depression). (Contrast with multidimensional scale.)
Multidimensional Scale
Scale that measures two or more distinct but related attributes or constructs, and measures of the different attributes or constructs are referred to as “subscales”
Global Distress
SubjectiveWell-Being
Problems &
SymptomsSocial
Functioning
Cut Scores
Specific predetermined numerical values along a continuum of scores Used to separate people into categories
with distinct substantive interpretations (e.g., clinically depressed or not)
Used to make decisions (provide treatment for depression or not)
Only as good as the normative sample(s) on which it is derived
Interpret in context of all available quantitative and qualitative information
Reverse-Worded Item
Item for which smaller numbers indicate a higher score on the measured variable because the item is worded to mean the opposite of the measured variable
Reliable Change
Change in a score from one time to another that is more than expected just from random measurement error Clinical significance.xls
Reliable Improvement
Improvement in a score from one time to another that is more than expected just from random measurement error
Reliable Deterioration
Deterioration in a score from one time to another that is more than expected just from random measurement error
Clinically Significant Improvement
Change that occurs when a client’s measured functioning on a standardized scale is:
In the dysfunctional range before intervention (e.g., greater than 5 on the QIDS-SR)
In the functional range after intervention (e.g., 5 or below on the QIDS-SR)
Change is reliable
Clinically Significant Improvement (cont’d) Interpret in context of all available
quantitative and qualitative information Does not guarantee a meaningful
change in a client’s real-world functioning or quality of life
Only as good as the normative sample(s) on which it is derived
Does not speak to the question of whether it was your intervention or something else that caused the change
Expected Treatment Response Session-by-session progress is
determined in comparison to normative data from ongoing responses to treatment of thousands of clients
Feedback used in real time to monitor client progress and modify services as needed to reduce treatment failures and increase overall effectiveness
Global Rating
Single rating based on a rater’s integration of information about numerous factors (e.g., global rating of change, improvement, or social functioning)
Single-Item Global Standardized Scales Global Assessment of Functioning (GAF) Children’s Global Assessment Schedule
(CGAS) Social and Occupational Functioning
Assessment Scale (SOFAS) Global Assessment of Relational
Functioning (GARF)
Potential Advantages of Standardized Scales Pretested for reliability and validity Structured, so information less likely to
be missed Can be used to compare individual
functioning to normative group functioning
Can be efficient and simple to use
Cautions in the Use of Standardized Scales May not measure concept suggested by
scale name Different measures of the same concept
may not be equivalent Sometimes limited information about
reliability and validity Concepts as measured
may not be completelyrelevant to individualclients
Resources
Compendiums of measures See Appendix B
Web measurement resources See Appendix B