Prof Ed 4 - Evaluation

ASSESSMENT OF

LEARNING

PROFESSIONAL EDUCATION

PRINCIPLES OF

ASSESSMENT

BASIC CONCEPTS

– a procedure in which a sample of an

individual's behavior is obtained, evaluated

and scored using standardized procedures

MEASUREMENT

– a set of rules for assigning numbers to

represent objects, traits, attributes, or

behaviors

BASIC CONCEPTS

ASSESSMENT

– any systematic procedures for collecting

information that can be used to make

inferences about the characteristics of people

or objects

EVALUATION

– refers to the process of determining the extent

to which instructional objectives are attained

BASIC CONCEPTS

Qualitative Research

– data typically in narrative form; gathered by

use of observations and interviews; results

contextual - unique to individual and setting

Quantitative research

– data numerical; gathered by quantifying

observations, administering tests and other

instruments; results generalizable – attempts

to find laws, generalizations

ASSESSMENT IN INSTRUCTION

Placement assessment

– done at the beginning of instruction

Formative assessment

– to monitor student progress

Diagnostic assessment

– intensive study of learning difficulties

Summative assessment

– done at the end of instruction for certifying

mastery or assigning grades

PLACEMENT ASSESSMENT

READINESS

CONCERN have prerequisite

skills?

PLACEMENT

CONCERN have achieved

intended

outcomes?

Provide

readiness

experience

Proceed

with the

planned

instruction

Proceed

with the

planned

instruction

Advance

students to

a higher

NO NO YES YES

FORMATIVE

ASSESSMENT

Students achieving

the intended

learning outcomes?

Provide group or

individual

remediation

Provide feedback to

reinforce learning

DIAGNOSTIC

ASSESSMENT

Proceed with the

planned instruction

Proceed with the

planned instruction

YES NO

SUMMATIVE

ASSESSMENT

Students achieved the

intended outcomes of

instruction?

Provide additional

learning experiences

Assign grades or

certify mastery

YES NO

Evaluate the

effectiveness of the

instruction

ASSESSMENT METHODS

Summary of characteristics

TESTING PERFORMANCE ASSESSMENT

SELECTED

RESPONSE

SUPPLY

RESPONSE

RESTRICTED

PERFORMANC

EXTENDED

PERFORMANC

LOW REALISM OF TASKS HIGH

LOW COMPLEXITY OF TASKS HIGH

LOW ASSESSMENT TIME NEEDED HIGH

LOW JUDGMENT IN SCORING HIGH

CHARACTERISTICS OF

ASSESSMENT RESULTS

RELIABILITY

– refers to the stability or consistency of

assessment results

• VALIDITY

– refers to the appropriateness and

meaningfulness of the inferences made from

assessment results

INTERPRETATION OF RESULTS

• Norm-referenced

– to provide relative ranking of students

– for survey testing

– compares performance to that of others

• Criterion-referenced

– a description of the learning tasks a student

can and cannot perform

– for mastery testing

– compares performance to a clearly specified

domain

APPLICATIONS

OF ASSESSMENT

Student Evaluation

Instructional decisions

Selection, placement, and classification decisions

Policy decisions

Counselling and guidance decisions

- Summative/Formative

- Placement/Diagnostic

- College admission/ determining eligibility

- Evaluating curriculum

- Promoting self-understanding

VALIDITY AND

RELIABILITY

VALIDITY AND RELIABILITY

• To what extent will the interpretation of

results be appropriate, meaningful and

useful?

• To what extent will the results be free from

errors?

MEASUREMENT ERROR

• Some degree of measurement error is inherent

in all measurement.

• Measurement error limits the extent to which

test results can be generalized and reduces the

confidence we have in test results.

• Education professionals should identify sources

of measurement error and minimize their impact.

SOURCES OF

MEASUREMENT ERROR

• Content sampling error

– If the items on a test are a good sample of the

domain, the amount of measurement error

due to content sampling will be relatively

small.

• Time sampling error

– reflects random fluctuations in performance

from one situation to another and limits our

ability to generalize test scores across

different situations

SOURCES OF

MEASUREMENT ERROR

• systematic error

– consistent inflation or deflation of the obtained

score (some items have apparent clues or

some tests have shorter time limits)

• random error

– inflation or deflation of scores in an

unpredictable manner (fluctuation in memory,

variation in motivation and concentration,

carelessness in marking answers, luck in

guessing)

VALIDITY

• the representativeness and relevance of the

assessment results to the measurement of a

specific achievement domain

• Validity is threatened when a test measures

either less or more than the construct it is

designed to measure.

• Reliability is a necessary but insufficient

condition for validity.

TYPES: EVIDENCE OF VALIDITY

• Content-related evidence

– How adequately does the sample of

assessment tasks represent the domain of

tasks to be measured?

– adequacy of sampling

– content relevance

• Criterion-related evidence

– How accurately does performance on the

assessment predict future performance

(predictive) or estimate present performance

(concurrent) on some other valued measure

(criterion)?

– ! degree of relationship between two sets of

measures: test scores and criterion to be

estimated or predicted (expressed in

correlation coefficients or expectancy tables

• Construct-related evidence

– How well can performance on the

assessment be explained in terms of

psychological characteristics?

– construct: hypothetical qualities that we

assume to exist in order to explain behavior

– an attempt to account for all possible

influences on the score

• Consequences of using assessment results

– improved motivation? performance? self-

assessment skills?

– encourage independent learning? good study

habits?

– contribute to transfer of learning? positive

attitude toward schoolwork?

RELIABILITY

• the consistency of assessment results from one

measurement to another

• obtaining approximately the same results for

tests administered at different times or with a

different sample of equivalent items or rated by

different raters

• Reliability provides the consistency needed to

obtain validity

RELIABILITY

• R measures provide an estimate of how much

variation to expect under different conditions

– reliability coefficient or standard error of

measurement

– reliability coefficient: also a correlation—

between two sets of measurements from

same procedure

• R score tends to be lower when the test is short,

range of scores is limited, testing conditions are

inadequate and scoring is subjective

METHODS OF

ESTABLISHING RELIABILITY

• Test-retest method

– administer same test twice to a group with a

time interval in between (stability of scores

over time)

• Equivalent-forms method (alternate/parallel)

– administer two equivalent forms of the test in

close succession (consistency of test scores

over different forms of the test)

METHODS OF

ESTABLISHING RELIABILITY

• Test-retest with equivalent forms

– administer two equivalent forms of the test

with a time interval in between

• Internal-consistency method

– administer test once and compute the

consistency of the response within the test

– e.g.: split-half method: results of two splits of

the same test are correlated

ITEM ANALYSIS

ITEM ANALYSIS • After-the-fact analysis of results of tests

• Purpose: improve tests by revising or eliminating ineffective items

• Process of examining class-wide performance of individual items

• Refers to a varied group of statistics computed for each item on a test

- item statistics help determine the role each item plays with respect to the entire test

ITEM ANALYSIS

1.Item Difficulty

2.Discrimination Index

3. Distracter Analysis

ITEM DIFFICULTY

• Proportion of the number of students in the upper and lower groups who answered an item correctly

DIFFICULTY INDEX

p = Number of students responding correctly

Total number of students taking the test

Tell very little of the item’s usefulness in measuring the test’s construct

DIFFICULTY INDEX

DISCRIMINATION INDEX

• Comparison of the proportion of the number of

students in the upper group who got an item

correctly to the proportion of students in the

lower group who got an item right.

• Comparison of how overall high scorers on the

whole test did on one particular item compared

to overall low scorers.

• Dependent on the difficulty of the item

D = PUpper - PLower

D = Discrimination Index

PU = Proportion of students in upper group

who got item right

PL = Proportion of students in lower group

who got item right

• POSITIVE Index Value

- More high scoring students selected correct

answers than low scorers

knowledge resulted in the correct answer

index closer to 1: the stronger the relationship

between overall test performance and

performance on that item

Retain item

• NEGATIVE Index Value

- More low scoring students selected correct

answers than high scorers

poor item validity

upper group misled by item ambiguity

Discard item

EVALUATION CODE FOR AN ITEM ANALYSIS

DISTRACTER ANALYSIS (Analysis of Response Options)

• Allows teachers to examine how many students

in the upper and lower groups selected each option

on a multiple-choice item

examines performance of each distracter

• Indicates extent to which knowledge is related to

response of student

• Measures validity

• ―Working‖ distracters – appear attractive to lower

groups

Have as many plausible distracters

Eliminate distracters not chosen

Item Analysis: Exercise 1

Item Analysis: Exercise 2

DEVELOPING

A CLASSROOM TEST

DEVELOPING A

CLASSROOM TEST

Step 1: Identification and statement of

INSTRUCTIONAL OBJECTIVES with the following characteristics:

• SCOPE

• DOMAIN

• FORMAT

CHARACTERISTICS OF

EDUCATIONAL OBJECTIVES

refers to how broad or specific the objective is.

DOMAIN

cognitive, affective, or psychomotor

FORMAT

behavioral versus non-behavioral

TABLE OF SPECIFICATIONS (TOS)

ensures congruence between classroom instruction and test content

BEHAVIORAL OBJECTIVES

(overt behavior)

specify activities that are observable and

measurable

NONBEHAVIORAL OBJECTIVES

(covert behavior)

specify activities that are unobservable and not directly measurable

KCAASE Knowledge, Comprehension, Application,

Analysis, Synthesis, Evaluation

BLOOMS TAXONOMY OF

K Rote memory

Learning facts

C Summarize,

Interpret , Explain

A Use general

rules & principles

to solve new problems

A Reduction of concepts into parts and show the relationship of parts to the whole

S Creation of new ideas of results from existing concepts

E Use general

rules and principles

to solve new problems

RRVOC Receiving, Responding,

Valuing, Organization,

Characterization by a value

KRATHWOL'S TAXONOMY OF

AFFECTIVE OBJECTIVES

R Being aware of and willing to attend to

something (Awareness, willingness to attend)

R Actively participating in an activity or process

(Acquiescence, satisfaction)

V Assigning value or worth to an activity or idea

(Acceptance, preference, commitment)

O Ideas and values become internalized and organized into one's personal system of values and beliefs

C Individual values are exemplified in a

characteristic set of behaviors and actions

HARROW'S TAXONOMY OF

PSYCHOMOTOR OBJECTIVES

Reflex Movements

Involuntary actions (Reflexes)

Inherent movement patterns (locomotor, manipulative movements)

Involves interpretation of sensory input that in turn guides movements (kinaesthetic, visual,

auditory)

Basic Fundamental Movements

Perceptual Abilities

Physical Abilities

Functional physical chars that serve as the basis for skilled movements

Complex movements that are the result of learning

Nonverbal communication ranging from facial expressions to expressive dance

Skilled Movements

Non-discursive Communication

RB PP SN

Reflex movement, Basic fundamental

Perceptual abilities, Physical abilities

Skilled movements, Non-discursive communication

TYPES OF TEST ITEMS

Selection-type items

– present students with a set of possible

responses from which they are to select the

most appropriate answer

– multiple choice, true-false, matching,

interpretive exercise

Supply-type

– requires students to create and supply their

own answers

– short answer, essay (restricted/extended)

What type of test is used here?

Which one of the following item types is an

example of a supply-type test item?

A. Multiple-choice item.

B. True-false item.

C. Matching item.

D. Short-answer item.

MULTIPLE CHOICE

• Designed to measure from simple to complex

learning outcomes

• Stem: the problem situation or question

– may be a question or incomplete statement or

in best-answer form

• Alternatives: the options or choices

– include the correct answer and distracters

MULTIPLE CHOICE

Single, clearly formulated problem in the stem of

the item

Stem of item written in simple, clear language

Put as much wording as possible in the stem of

the item

State the stem in positive form, whenever

possible

MULTIPLE CHOICE

Emphasize negative wording whenever it is

used in the stem of an item

Make all alternatives grammatically consistent

with the stem of the item and parallel in form

Make the distracters plausible and attractive

Vary the length of the correct answer to

eliminate length as a clue

MULTIPLE CHOICE

Vary the position of the correct answer in

random manner

Make certain each item is independent of

the other items in the test

Use an efficient item format

Follow the normal rules of grammar

MULTIPLE CHOICE

Verbal clues that enable students to select

the correct answer or eliminate a distracter

Alternatives such as ―all of the above‖

(use ―none of the above‖ with caution)

Spot the errors

1. A table of specifications:

a. indicates how a test will improve learning.

b. provides a more balanced sampling of

content.

c. arranges the instructional objectives in order

of importance.

d. specifies the method of scoring to be used

on a test.

Spot the errors

2. The paucity of plausible, but incorrect, statements that can be related to a central idea poses a problem when constructing which one of the following types of test items?

a. Short answer.

b. True-false.

c. Multiple-choice.

d. Essay.

Spot the errors

3. In objective testing, the term objective:

a. refers to the method of identifying learning

outcomes.

b. refers to the method of selecting the test

content.

c. refers to the method of presenting the

problem.

d. refers to the method of scoring the answers.

Spot the errors

4. Which one of the following is not a category in

the revised taxonomy of the cognitive domain?

a. Understand

b. Apply

c. Analyze

d. (distracter needed)

Spot the errors

5. Which of the following is not a desirable practice

when preparing multiple-choice items?

a. Stating the stem in positive form.

b. Using a stem that could function as a short-

answer item.

c. Underlining certain words in the stem for

emphasis.

d. Shortening the stem by lengthening the

alternatives.

Spot the errors

6. The recall of factual information can be

measured best with a:

a. matching item.

b. multiple-choice item.

c. short-answer item.

d. essay question.

Spot the errors

7. Why should negative terms be avoided in

the stem of a multiple-choice item?

a. They may be overlooked.

b. The stem tends to be longer.

c. The construction of alternatives is more

difficult.

d. The scoring is more difficult.

Spot the errors

8. Obtaining a dependable ranking of students is

of major concern when using:

a. norm-referenced summative tests.

b. behavior descriptions.

c. checklists.

d. questionnaires.

Spot the errors

9. One advantage of multiple-choice items

over essay questions is that they:

a. measure more complex outcomes.

b. depend more on recall.

c. require less time to score.

d. provide for a more extensive sampling

of course content.

Spot the errors

10. Which one of the following would you

consult first to locate research articles on

achievement testing?

a. Journal of Educational Psychology

b. Journal of Educational Measurement

c. Journal of Consulting Psychology

d. Review of Educational Research

Spot the errors

11. Which of the following is a category in the

revised taxonomy of the cognitive domain?

a. Critical Thinking.

b. Scientific Thinking.

c. Reasoning Ability.

d. None of the above.

What types of test are here?

T F True-false items are classified as a

supply-type item.

________________________________________

Column A

1. Least used for educational diagnosis

2. Measures great variety of learning outcomes

Column B

A. Multiple-choice item.

B. True-false item.

C. Short-answer item.

TRUE-FALSE ITEMS

• Used when there are only two possible

alternatives

• Used to measure the ability to identify whether

statements of fact are correct

• Variations:

– ask to change the false statements

– cluster-type true-false format

TRUE-FALSE ITEMS

Include only one central idea

Keep statement short and use simple

vocabulary and sentence structure

Word the statement precisely that it can be

unequivocally be judged true or false

Use negative statements sparingly

Attribute statements of opinion unless used to

distinguish facts from opinion

TRUE-FALSE ITEMS

Extraneous clues to the answer

Disproportionate number of true statements

or false statements

True statements being generally longer than

false ones

Use of absolutes: for FALSE—always, never,

all, none, only; for TRUE—usually,

sometimes, may

Discernable pattern of correct answer

Spot the errors

1. The true-false item, which is favored by test

experts, is also called an alternative-response

2. The true-false item is more subject to guessing

but it should be used in place of a multiple-

choice item, if well constructed, when there is a

dearth of distracters that are plausible.

3. Lengthening a test will increase its reliability.

Spot the errors

4. Testing should play a major role in the teaching-

learning process.

5. True-false items are classified as objective

items because students must supply the

answer.

6. A statement of opinion should never be used in

a true-false item.

MATCHING ITEMS

• a variation of multiple-choice form

• eliminate the repetition of the alternative

answers and present the same items in compact

• premises: the series of stems

• responses: the series of alternative answers

MATCHING ITEMS

Include only homogenous material in each

matching item

Keep list of items short and place the brief

responses on the right

Use larger, or smaller, number of responses

than premises, and permit the responses to be

used more than once

Place responses in alphabetical or numerical

MATCHING ITEMS

Specify in the directions the basis for matching

and indicate that each response may be used

once, more than once, or not at all

Put all of the matching item on the same page

SHORT ANSWER ITEMS

• also, completion items

• require the student to supply the

appropriate words, numbers or symbols to

answer a question or complete a

statement

• used for simple recall of knowledge and for

computational problems

SHORT ANSWER ITEMS

State the item so that only a single, brief answer

is possible

Start with a direct question and switch to an

incomplete statement only when greater

conciseness is possible by doing so

Leave only one blank, which should relate to the

main point of the statement

Place blanks at the end of the statement

SHORT ANSWER ITEMS

For numerical answers, indicate the degree of

precision expected and the units in which they

are to be expressed

SHORT ANSWER ITEMS Extraneous clues to the answer

length of the blank

indefinite articles a/an

Spot the errors

1. In terms of type or response, the ______ item

is most like the _____ item.

2. _______ is likely to increase when a test is

lengthened.

3. The supply-items used to measure the ability to

organize and integrate material is called an

ESSAY QUESTIONS

• provide freedom of response: approach, use of

factual information, organization of answer,

emphasis to aspects of the response

• used to measure ability to organize, integrate

and express ideas

• can be restricted response (through special

directions, use of introductory material, specific

form of answer required) or extended response

(limitations may be limited to time and page

limits)

ESSAY QUESTIONS

Use to measure complex learning outcomes

Relate the questions as directly as possible to

the learning outcomes being measured

Formulate questions that present a clear task to

be performed

Provide ample time for answering and suggest a

time limit on each question

SCORING ESSAYS

Evaluate answers in terms of learning outcomes

being measured

Have defined criteria as guide

Evaluate all of the students’ answers to one

question before proceeding to the next

Evaluate without knowing the identity of the

writer

When possible, have two or more raters

SCORING RUBRIC

• structured, unbiased scoring procedures that are used to enhance the reliability of essay tests

• a rating scale—as opposed to a checklist—that is used with performance assessments

• also known as scoring guide, consisting of specific pre-established performance criteria, used in evaluating student work on performance assessments

TYPES OF RUBRICS

Holistic rubric

– requires the teacher to score the overall

process or product as a whole, without

judging the component parts separately

– a single score is assigned based on the

overall quality of the student's response

TYPES OF RUBRICS

Analytic rubric

– the teacher scores separate, individual parts

of the product or performance first, then sums

the individual scores to obtain a total score

– usually needs to specify the value assigned to

each characteristic

• e.g. 15-point essay item (10 points for

content, 3 points for writing style, 2 points for

grammar)

TEMPLATE FOR HOLISTIC RUBRICS

Score Description

5 Demonstrates complete understanding of the

problem. All requirements of task are included in response.

4 Demonstrates considerable understanding of the problem. All requirements of task are included.

3 Demonstrates partial understanding of the problem. Most requirements of task are included.

2 Demonstrates little understanding of the problem. Many requirements of task are missing.

1 Demonstrates no understanding of the problem.

0 No response/task not attempted.

TEMPLATE FOR ANALYTIC RUBRICS

Beginning 1

Developing 2

Accomplished 3

Exemplary 4

Criterion #1

Description

reflecting

beginning level

Description

reflecting

movement toward mastery

Description

reflecting

achievement of mastery level

Description

reflecting

highest level of performance

Criterion #2

Description

reflecting

beginning level

Description

reflecting

Description

reflecting

Description

reflecting

Criterion #3

Description

reflecting

beginning level

Description

reflecting

Description

reflecting

Description

reflecting

Criterion #4 Description

reflecting

beginning level

Description

reflecting

Description

reflecting

Description

reflecting

PERFORMANCE

ASSESSMENTS AND

PORTFOLIOS

PERFORMANCE ASSESSMENT

Requires students to complete a process or produce a product in a context that closely resembles a real-life situation.

A medical student might be required to interview a mock patient.

more closely reflect real-life settings and applications than traditional paper-and-pencil assessment.

Also referred to as authentic assessment or

alternative assessment.

The primary approach to assessment in classes such as art, music, PE, etc.

PERFORMANCE ASSESSMENT:

STRENGTHS

Can measure abilities that are not accessible using other assessment.

Consistent with modern learning theory

May result in better instruction.

Make learning more meaningful

PERFORMANCE ASSESSMENT:

WEAKNESSES

Notorious for producing unreliable scores.

Difficult to make generalizations about the skills and knowledge the student possess.

Time consuming and difficult to construct, administer, and score.

GUIDELINES FOR DEVELOPING

EFFECTIVE PERFORMNCE

ASSESSMENTS

1. Select an appropriate assessment task.

2. Develop instructions that clearly specify

what students are expected to do.

3. Develop procedures for evaluating students’

responses

GUIDELINES FOR SELECTING

PERFORMANCE TASKS

1. Select performance that provide the most direct assessment of the educational objectives you want to measure.

2. Select performance tasks that maximize your ability to generalize the results of the assessment.

3. Select performance tasks that reflect essential skill.

PERFORMANCE TASKS

4. Select performance tasks that focus your evaluation on the process and/or products you are most interested in.

5. Select performance tasks that provide the desired degree of realisms.

6. Select performance tasks that measure skills that are ―teachable‖.

7. Select performance tasks that are fair to all students.

PERFORMANCE TASKS

8. Select performance tasks that can be assessed given the time and resources available.

9. Select performance tasks that can be scored in a reliable manner.

10. Select performance tasks that reflect educational objectives that cannot be measured using more traditional measures.

INSTRUCTIONS FOR

1. Make sure that your instructions clearly specify the types of responses you are expecting.

2. Make sure that your instructions specify any important parameters of the performance task.

3. Make sure that your instructions clearly specify the criteria you will use when evaluating the students’ responses.

4. Have a colleague read and interpret the instructions before you administer the assessment to your students.

INSTRUCTIONS FOR

5. Try out the performance activity with one or a limited number of students before administering it to your whole class to ensure that the instructions are thorough and understandable.

6. Write instructions that students from diverse cultural and ethnic background will interpret in an accurate manner.

PORTFOLIOS

specific type of performance

assessment that involves the systematic

collection of a student's work products

over a specified period of time according

to a specific set of guidelines.

particularly good at reflecting student

achievement and growth over time.

PORTFOLIO: STRENGTHS

Particularly good at reflecting student

achievement and growth over time.

May help motivate students and get them more

involved in the learning process.

Enhances students abilities to evaluate their own

performances and products.

Enhances teachers' communication with both

students and parents.

PORTFOLIO: WEAKNESSES

Scoring in a reliable manner is difficult.

Time-consuming and demanding process.

PORTFOLIOS

1. Decide on the purpose of the portfolio.

Enhance learning, assign grades, or combination

Best work, representative product, growth or learning progress, or evaluation

2. Decide who will select the items to include in the portfolio

Teacher

Student

Collaboration

PORTFOLIOS

3. Establish procedures for evaluating or scoring portfolio.

Specify the evaluation criteria

Provide specific standards

Decide on a holistic or an analytic approach

4. Promote student involvement in the process.

“TEACHING TO THE TEST”

Implies efforts by teachers to prepare students

to perform better on standardized achievement

tests.

COMMON SOURCES OF RATING

HALO EFFECT: the tendency of raters to be

influenced by a single positive or negative trait,

unrelated to the trait or skill being assessed, that

affects their rating of the student’s other

characteristics.

LENIENCY EFFECT: occur because some

teachers tend to give all students good ratings.

SEVERITY ERRORS: occur because some

teachers tend to give all students poor ratings.

CENTRAL TENDENCY ERRORS: occur because

some teachers tend to give all students scores

in the middle range.

PERSONAL BIASES: may corrupt ratings if

teachers have a tendency to let stereotypes

influence their ratings o student’s performance.

LOGICAL ERRORS: occur when a teachers

assumes that the two characteristics are related

and tends to give similar ratings based on this

assumption.

ORDER EFFECTS: changes in scoring that

emerge during the grading process.

INTERPRETING

ASSESSMENT

RESULTS

• Basic mathematics of measurement

–Measure of Central Tendency

–Measure of Variability

–Frequency Distribution

–Symmetry and Skewness

–Percentiles, Deciles, Quartiles

• Meaning of test scores

–Norm-referenced

–Criterion-referenced

WHAT WE WILL COVER?

DO YOU RECALL THESE

STATISTICAL TERMS?

• population – totality of observations

• sample – subset of population

• parameter – numerical value describing population; Greek letter

• statistic – describes a characteristic of a sample; ordinary English letter

To know how accurate

the statistic estimates the parameter, check the distribution of the data.

DESCRIBING A GROUP

AVERAGE

- A measure of the center of a set of data when data are arranged in an increasing or decreasing order of magnitude

or measure of central location

or measure of central tendency

median

WHAT IS THE BEST MEASURE

OF CENTRAL TENDENCY?

Depends on:

- scale of measurement

- shape of distribution

NOMINAL SCALES

Classify people or objects into categories, classes,

or sets.

Gender

Ethnicity

Place of birth

ORDINAL SCALES

Rank people or objects according to the amount of a

characteristic they display or possess

Preference for activity

Graduation class rank

Percentile rank

Numbers used to represent ―more/less than‖

MEDIAN

INTERVAL SCALES

Rank people or objects like an ordinal scale but on

a scale with equal units; has a constant unit of

measurement with an arbitrary zero.

Fahrenheit and Celsius scales

Intelligence scores

Personality test scores

MEAN, MEDIAN, MODE

RATIO SCALES

Have the properties of interval scales plus a true

zero point.

Height/length in inches

Weigh in pounds

Values

MEAN, MEDIAN, MODE

MEASURES OF CENTRAL TENDENCY

MEAN is the arithmetic average of a distribution.

Mean = sum of scores or

Number of scores

MEASURES OF

CENTRAL TENDENCY

When a distribution is skewed,

the influence of unbalanced extreme scores on

the mean tends to undermine its usefulness.

Mean is pulled in the direction of the skew,

Mean lower than the median in a negatively

skewed distributions; higher than the median in

a positively skewed distribution

MEASURES OF

CENTRAL TENDENCY

MEDIAN

- score or potential score that divides a

distribution in half; middle value

- affected by size of sample

- unaffected by extreme scores or outliers

- most frequently occurring score in a

distribution.

- quickest estimate

MEASURES OF

CENTRAL TENDENCY Measure of central tendency is NOT adequate

to describe data

need to know how observations are

spread out of the data

need to know how much scores

deviate around the mean or are

bundled up

MEASURES OF VARIABILITY

- range

- variance

- standard deviation

- coefficient of variation

RANGE is the distance between the smallest and largest

score in a distribution.

VARIANCE is the deviation of an observation from the

STANDARD DEVIATION is a measure of the average

distance that scores vary from the mean of the

distribution.

COEFFICIENT OF VARIATION

- a measure of relative variation

- used since the standard deviation does

not by itself tells much about variability

of a single data

- can be used to compare variability of two

or more sets of data

- equal to the standard deviation as a

percentage of the mean

How to compute for STANDARD DEVIATION

a. Compute the mean

b. Subtract each score in the distribution from

the mean

c. Sum all the squared difference scores

d. Divide this sum by the number of scores to

derive the average of the squared deviations

from the mean = variance

e. Take the square root of the variance

Measures of central tendency and variability are

still not enough to fully identify

important features of the data

In case of large mass of data, characteristics

can be readily assessed by grouping the data

into different classes and then determining the

number of observations that fall in each of the

classes

FREQUENCY DISTRIBUTION

BAR CHART

HISTOGRAM

FREQUENCY POLYGON FREQUENCY OGIVE

Basic Terms

Frequency distribution table

lists categories of scores along with their

corresponding frequencies.

Frequency for a particular category or class

number of original scores that fall into that class.

Classes or categories refer to the groupings of a

frequency table

Basic Terms

Class width

difference between two consecutive lower class

limits or class boundaries

Class limits

smallest or the largest numbers that can actually

belong to different classes

Lower class limits are the smallest numbers that can

actually belong to the different classes.

Upper class limits are the largest numbers that can

actually belong to the different classes.

Class marks are the midpoints of the classes

Basic Terms

Class boundaries

obtained by increasing the upper class

limits and decreasing the lower class

limits by the same amount so that there

are no gaps between consecutive under

classes.

The amount to be added or subtracted is ½

the difference between the upper limit of one

class and the lower limit of the following

class.

Basic Terms

Frequency Distribution:

RELATED TERMS

Grouped

frequency

distribution

A frequency distribution where scores

are grouped into intervals rather than

listed individual values

Class interval A group of scores in a grouped

frequency distribution.

Upper real limit The boundary that separates an

interval from the next highest interval.

Lower real limit The boundary that separates

an interval from the next lowest interval. 140

NORMAL DISTRIBUTION

Data symmetrically distributed on either side of

its midpoint, which is the average value and also

the median and mode.

Tails - areas on either side of the peak

Sometimes called the "bell curve”

since the curve drawn from the normal distribution

looks like you are looking at a bell from the side

Normal Distribution

MEAN, MEDIAN, MODE

CHARACTERISTICS OF THE

NORMAL CURVE

The normal curve is a symmetrical distribution of

scores with an equal number of scores above and

below the midpoint of the abscissa (horizontal axis

of the curve).

Since the distribution of scores is symmetrical the

mean, median, and mode are all at the same point

on the abscissa.

MEAN = MEDIAN = MODE

CHARACTERISTICS OF THE

NORMAL CURVE

If we divide the distribution up into standard

deviation units, a known proportion of scores lies

within each portion of the curve.

Statistical tables exist so that we can find the

proportion of scores above and below any part of

the curve, expressed in standard deviation units.

Z-scores – scores expressed in standard deviation

PERCENTAGES OF CASES UNDER PORTIONS

OF THE NORMAL CURVE

PERCENTAGES OF CASES UNDER PORTIONS OF THE NORMAL CURVE

_____ % of observations lie within

1 S.D. of the mean

2 S. D. of the mean

3 S. D. of the mean

____ % of scores lie between the mean and 1 S.D.

above the mean.

An equal proportion of scores (____%) lie between

the mean and 1 S.D. below the mean.

Approximately two-thirds (____%) of the scores lie

within 1 S. D. of the mean

___ % of the scores lie between one and two

standard deviations above/below the mean

____ % of the scores are within three standard

deviations of the mean.

SYMMETRIC DISTRIBUTION

A distribution is SYMMETRIC

if it can be folded along the vertical axis so that

the two sides coincide

Mean = Median

Skewness = 0

TYPES OF ASYMMETRY

If the curve drawn from a distribution is

asymmetric, then the distribution is

not normal.

There are several types of asymmetry:

modality

peakedness (kurtosis)

spread (skewedness).

If the curve has more than one peak, it is

multimodal, two peaks, it is bimodal.

TYPES OF ASYMMETRY

If the curve is "too peaked" to be normal,

it is leptokurtic.

use mean

If it is "too flat," it is platykurtic .

FORMS OF KURTOSIS

SKEWNESS

If the curve has one tail that is longer than the

other, it is skewed.

If the longer tail is on the left, it is negatively

skewed (too many scores toward the negative

If the longer tail is on the right, it is positively

skewed.

• Positively skewed

- skewed to the left

- more students got _____ marks

• Negatively skewed

- skewed to the right

- more students got _____ marks

Z-SCORES

Shows relative rank

Measures how many standard deviation an

observation is above or below the mean

(Example: In school tests, z-scores show

how student performed in one

class/subject relative to another

class/subject)

Z-SCORES

Z-score = 1

Score is one standard deviation above the mean

Z-score = -1

Score is one standard deviation below the mean

Z-score = 0

Score = mean

The normal curve with Z-scores along the abscissa

looks exactly like the normal curve with standard

deviation units along the abscissa.

Z-SCORES

Where X is the raw score, X bar is the mean, and

S is the standard deviation.

T-SCORES

T-scores are derived scores with a mean of 50

and a standard deviation of 10.

The average T-score for a group of scorers

would be 50.

T-score of 60 = Z-score of 1.

Each of these scores would be one standard

deviation above the mean and would be equal

to or higher than 84.13% of the scores (50.00%

+ 34.13% = 84.13%)

T-SCORES

Where Z is the obtained Z-score

FRACTILES or QUANTILES

• Other measures of location that describe or

locate the position of pieces of data relative

to the entire set of data.

• Values below which a specific fraction or

percentage of observations in a given set

must fall.

PERCENTILES

DECILES

QUARTILES

PERCENTILE RANKS

Percentage or proportion of scores that score

lower than a given score.

Percentile rank of 90 or 90th percentile

90% of the scores are lower than your score

and 10% of the scores are higher.

50th percentile = median

50% of the scores are lower than the

median, and 50% of the scores are higher

than the median. 167

PERCENTILE RANKS

Reported as whole numbers

highest percentile = 99

lowest percentile = 1

A score that is one standard deviation

below the mean would have a percentile

rank of ____

PERCENTILE RANKS

A score that is two standard deviations

below the mean would have a percentile

rank of ____

(0.13 + 2.14 = 2.27)

A score that is three standard deviations

below the mean would be at the 1st

percentile and one that was three

standard deviations above the mean

would be at the 99th percentile

PERCENTILE RANKS

-4 -3 -2 -1 0 1 2 3 4 z-score

10 20 30 40 50 60 70 80 90 T-score

200 300 400 500 600 700 800

SAT score

55 70 85 100 115 130 145 Deviation IQ

(SD = 15)

Percentile 1 5 10

Normal Distribution Illustrating the

Relationship among Standard Scores

0.13% 2.14%

13.59%

34.13% 34.13%

13.59%

2.14% 0.13%

-4s -3s -2s -1s Mean +1s +2s +3s +4s

Number of

CORRELATION COEFFICIENTS

A quantitative measure of the relationship between two

variables.

Examples:

- Pearson product-moment correlation

coefficient

- partial correlation coefficient

A scatterplot is a graph that visually displays the

relationship between two variables.

CORRELATION COEFFICIENTS

Positive correlation coefficient

indicates that an increase in one variable is

associated with an increase on the other

variable.

Negative correlation coefficient

indicates that an increase in one variable is

associated with a decrease on the other

variable.

ASSESSMENT OF

AFFECTIVE LEARNING

BASIC CONCEPTS

Behavior

– everything done that can be observed

Personality

– pattern of enduring distinctive thoughts,

emotions and behaviors that characterize

one’s adaptation to the world

– an enduring personality characteristic that

tends to lead to certain behaviors

ASSESSMENT METHODS

• Projective tests

– use ambiguous stimulus and asks the

examinee to describe or tell a story about it

• Self-report tests

– also called objective tests or inventories;

directly ask people whether items describe

them or not

• Behavior rating

– an inventory that asks an observer to rate the

examinee in a number of dimensions

PROJECTIVE TECHNIQUES

• Projective drawings

– draw-a-person

– house-tree-person

– kinetic family drawing

• Rorschach Inkblot: what do you see?

• Thematic Apperception Test

• Sentence-completion

• Graphology

INKBLOT

THEMATIC APPERCEPTION

• tell

– a story about each of

the pictures

– events leading to the

situation

– characters’ thoughts,

feelings

– the way the situation

turns out

SELF-REPORT TESTS

• have a large number of statements and

limited number of answers (Y/N, T/F,

• used in the identification of traits

• empirically keyed test

– uses items seemingly unrelated to the desired

criterion in order to predict its presence in an

individual

GUIDANCE AND

COUNSELLING

GUIDANCE SERVICES

– Individual Inventory System

– Information Service

– Counselling Service

– Placement Service

– Follow-up Service

– Appraisal/Testing Service

– Referral Service

– Research And Evaluation

REPUBLIC ACT 9258

• Guidance and Counseling Act of 2004

• Guidance and Counseling

– a profession that involves the use of an

integrated approach to the development of

well-functioning individual primarily by helping

him/her to utilize his /her potentials to the

fullest and plan his/her future in accordance

with his/her abilities, interests and needs

REPUBLIC ACT 9258

• Guidance and Counseling: FUNCTIONS

– counseling

– psychological testing, (as to personality,

career interest, study orientation, mental

ability and aptitude)

– research, placement, group process

– teaching and practicing of guidance and

counseling subjects, particularly subjects

given in the licensure examinations, and other

human development services

PRINCIPLES OF GUIDANCE

• Guidance is based on a true concept of the

client.

• Guidance is designed to provide assistance to

the person in crisis in solving it through self-

discovery and self-direction.

• Guidance is helping the client understand

him/herself.

• Guidance is a learning process.

• Guidance is preventive rather than curative.

• Guidance is fundamentally the responsibility of parents in the home and of teachers in the school.

• Specific guidance problems in any age level should be referred to persons trained to deal with particular areas of adjustments.

• Guidance services help the individual to become increasingly competent in self- direction and self-help.

• Continuous or periodic appraisals should be made.

• Test have their place in guidance.

• Every aspect of an individual’s complex personality pattern constitutes a significant factor into his/her total display of attitude and behaviors.

• Although all human beings are similar in many ways, individual differences must be recognized.

• Guidance should be extended to all, not simply to the obviously maladjusted.

ASSIGNING

MARKS/RATINGS

FEEDBACK AND EVALUATION

FORMATIVE EVALUATION

evaluative activities aimed at providing feedback to students (often informal).

SUMMATIVE EVALUATION

determination of the worth, value, quality of an outcome (formal).

FEEDBACK AND EVALUATION

GRADES

formal recognition of a specific level of mastery.

brief summary statements that fail to convey rich details about a student's performance.

REPORTING STUDENT PROGRESS

LETTER GRADES

most popular method of reporting student progress and are used in the majority of schools

NUMERICAL GRADES

attempt to succinctly represent student performance; may provide more precision than letter grades.

VERBAL DESCRIPTORS

replace letter grades by using excellent, above average, satisfactory, needs improvement

PASS-FAIL

given in situations where the learning

objectives are clearly specified but convey

even less information than traditional letter or numerical grades

SUPPLEMENTAL SYSTEMS

replace or supplement traditional marking systems by the use of checklist of specific learning objectives to provide additional information about the students' strengths and weaknesses.

FRAME OF REFERENCE

NORM-REFERENCED

or relative grading involves comparing each student's performance to that of a specific group. Also referred to as ―grading on the curve‖.

CRITERION-REFERENCED

or absolute grading involves comparing a student's performance to a specified level of performance.

INFORMING STUDENTS OF

GRADING SYSTEM

Students have the right to know the procedures

that will be used to determine their grades.

Parents should also be informed in a note or in

person during conferences or visits what is

expected of their children and how they will be

graded.

Prof Ed 4 - Evaluation

Documents

LET Online Reviewer Prof Ed

Prof ed 602 Fame Game

Washington SNAP-Ed Evaluation Guidance for Federal Fiscal … · 2019-10-31 · 4 Evaluation Training: The evaluation team will post evaluation guidance and videos for SNAP-Ed direct

Diseño Prof. Ed. Mus. Sept15

Sept 22 Prof Ed

Online Let Prof. Ed

Prof. ed. legal foundations of education

colegiul-cantacuzino.ro · 2019-10-11 · Firescu Dragos, prof. ed. fizicä si sport Dobricä Cristian, prof. ed. fizicä 9i sport Dobricä Mihaela, prof. ed. fizicä si sport Cälinescu

Prof Ed Module 13

Prof. Dr. Wolfgang Böttcher Evaluation. Prof. Dr. Wolfgang Böttcher Agenda 1)Definition von Evaluation 2) Ansätze/Formen der Evaluation 3) Standards der

Let reviewer prof ed

Prof ed 602 - Educational Technology 1

Pediatric Concussion: ED Evaluation and Beyond

Joint Evaluation of the GEF Small Grants Programme Joint Evaluation … · Evaluation Documents ED-2 GEF Evaluation Office Ethical Guidelines 2008 ED-1 The GEF Evaluation and Monitoring

Chest Pain ED Evaluation

Prof Ed Drill

SNAP-Ed Guide to Program Evaluation - Home :: …€¦ · SNAP-Ed Guide to Program Evaluation ... The selected evaluation resources focus on measuring objectives with factors of dietary

CURRICULUM EVALUATION TEAM - ed

Prof. ed reviewer

Evaluation Rubric - ed