RSCH 6109: Assessment & Evaluation Methods Introduction & Overview “Hot” Topics in Testing Some Basic Terminology Types of Measures

RSCH 6109: Assessment & Evaluation Methods

Introduction & Overview

“Hot” Topics in Testing

Some Basic Terminology

Types of Measures

Assessment is an integral part of the work of counselors.

Effective counselors use assessment methods to help clients:

understand themselves

develop and realize their potential

make decisions

solve problems

Introduction.…

Source: Modified from Drummond (2003)

Information from tests should not be used in isolation.

Effective counselors integrate information from testing with information about clients from other sources:

Key informants

Interviews

Reflective journals and homework

Behavioral anecdotes and observations

Introduction.…


Assessment is a useful tool for counselors because it can:

help clarify the client’s presenting problem / concerns / goals

help identify treatment options

help evaluate services

provide an objective perspective

help build rapport

Introduction.…


Assessment is a useful tool for counselors because it can:

help client’s recognize / accept ownership for a problem or issue

screen clients for referral to other professionals

facilitate communication with the client / help resistant clients engage in the process

Introduction.…


Good assessment practices require both the “art” and the “science” of psychology.

The “science” is involved in the development, validation, and scoring of tests.

The “art” is involved in the interpretation and debriefing processes.

Introduction.…


Testing is an invasion of privacy.

Tests are gender-biased and use inappropriate language, examples and illustrations.

Tests are culturally biased; they are unfair and discriminate against minority groups.

Some Current Testing Controversies.…


Criterion-referenced tests should be used rather than norm-referenced tests.

Coaching will help performance of clients on scholastic aptitude tests.

Counselors should not be testing clients.



Counselors should replace standardized tests with qualitative assessment measures.

Intelligence tests are not measuring the right constructs.

We cannot rely on grades and diplomas; we must have demonstration of competencies on objective tests.



Every school scores above the median on achievement tests. The norms are not valid: The lake Woebegone effect is rampant.

Multiple choice tests need to be replaced by performance assessments.

There is too much emphasis on testing and teachers teach for the tests.



There is too much pressure on students, teachers, and parents because of high stakes testing: It’s now the “in” approach in schools.

Psychological testing of students is intrusive meddling.



“Measurement is limiting the data of any phenomenon so that those data may be interpreted and, ultimately, compared to an acceptable qualitative or quantitative standard.” -P. Leedy (2001)

Some Common Measurement Terminology:

1. Measurement2. Assessment3. Evaluation4. Test or Measure 5. Psychometrics6. Reliability7. Validity8. Formative vs. Summative

Measures9. Criterion- vs. Norm-Referenced

Measures

Terminology

Assessment: A process that integrates test information with information from other sources in decision making

Evaluation: The process of placing judgment on a behavior, product, or process in order to determine the significance, worth or condition of, by careful appraisal and study.

Test or Measure: An evaluative device or procedure in which a sample of behavior is obtained, evaluated, and scored using a standardized process.

Terminology

Psychometrics

The psychological techniques of mental measurement. Literally means “to measure the mind.”

Measurement Reliability & Validity – Good measures yield reliable scores from which we can draw valid conclusions.

Measurement Reliability refers to the amount of error in a score. Error refers to the extraneous factors that effect students’ performance on a test. If we were to test the same student with the same test multiple times, how much would the scores differ?

Measurement Validity refers to the accuracy of the interpretations we draw from a test.

Terminology (cont.)

Evaluation: Formative vs. Summative Measures

Formative measures are used to monitor students’ or clients progress during instruction or therapy.

Summative measures are designed to evaluate students’ or clients performance at the end of instruction or therapy.

Referencing: Criterion- vs. Norm-Referenced Measures

Criterion-referenced measures compare outcomes to a standard.

Norm-referenced measures compare outcomes to a group.

Terminology (cont.)

Common Types: Measures of Optimal Performance

Examples include intelligence, aptitude, and achievement.

Intelligence tests (or tests of intellectual ability) typically measure performance on various verbal and non-verbal tasks. These measures must be administered by a qualified professional in the field of psychology.

Examples: WPPSI-R: Wechsler Preschool and Primary Scale of Intelligence-Revised (ages 3-7) ; WISC-III: Wechsler Intelligence Scale for Children-Revised (ages 5-15); WAIS-III: Wechsler Adult Intelligence Scale (ages 16-89); KAIT: Kaufman Adolescent and Adult Intelligence Test

Types of Measures

Common Types: Measures of Optimal PerformanceExamples include intelligence, aptitude, and

achievement.

An aptitude is an ability that is necessary or facilitative for a particular purpose.

Aptitude tests are typically used to predict performance in a particular area. Provides some information about students’ aptitude at a particular time. No measure of aptitude is a perfect predictor for future performance: Intelligence and aptitude are NOT fixed!

Examples: SAT, ACT, GRE, MCAT, LSAT, Differential Aptitude Test, Academic Aptitude Tests, Allied Health Aptitude Test, School Readiness Test, Algebra Readiness Test.

Types of Measures (cont.)


Examples include intelligence, aptitude, achievement.

Achievement tests measure what students have learned in relation to a relatively defined educational experience. The majority of achievement tests are locally constructed and administered, but many standardized tests are available.



Examples include intelligence, aptitude, achievement.

Special forms of achievement tests include diagnostic tests and performance assessments.

Diagnostic tests are used to identify strengths and weaknesses in relation to a particular subject. Typically include more narrow range of concepts and are criterion referenced. Performance assessments are used to directly assess performance on tasks that have intrinsic value. These tasks are designed to reflect authentic, complex, intrinsic activities.

Examples: NC EOG/EOC Tests, California Achievement Test, Iowa Test of Basic Skills, Metropolitan Achievement Test, Stanford Achievement Test, Wide Range Achievement Test-Revised, Classroom Communication Skills Inventory, Arizona Clinical Interview Rating Scale, Counseling Skills Evaluation, and the Diagnostic Mathematics Inventory.

.


Common Types: Measures of Typical Performance

Typical performance measures include those designed to assess personality traits such as attitudes, interests, and dispositions. These measures are also classified as “affective measures.”

These measures typically include self-report types of instruments and do not require individuals to ‘perform.’ Instead they ask individuals to report their thoughts, attitudes, feelings, behaviors, and traits.

Examples include: MMPI, MBTI, SDS, SII, Index of Learning Styles, Study Habits Inventory, Belief in Just World Scale, Locus of Control, Vocational Interest Inventory, Quality of School Life Scale, Parent Perception Inventory, and the Math Anxiety Classification Scale.


Sample Web Resources1) NC EOG and EOC Testshttp://www.ncpublicschools.org/accountability/testing

2) The Minnesota Multiphasic Personality Inventory-Adolescenthttp://www.pearsonassessments.com/tests/mmpia.htm

3) Wechsler Adult Intelligence Scale (WAIS)http://www.wilderdom.com/personality/intelligenceWAISWISC.html

http://www.ncpublicschools.org/accountability/testing

http://www.pearsonassessments.com/tests/mmpia.htm

http://www.wilderdom.com/personality/intelligenceWAISWISC.html

Documents

RSCH 6109: Assessment & Evaluation Methods Introduction & Overview “Hot” Topics in Testing Some Basic Terminology Types of Measures