The Concentration Factor

• The definition– It measures how the students’ responses are distributed

on a multiple-choice test.

– In their response to an MC question with research-based distracters, we assume most students will answer using one of the various reasoning models observed in qualitative research.

Type A B C D EI 20 20 20 20 20II 50 5 30 5 10III 100 0 0 0 0

Possible distributions of responses from a 100 students on a single 5-choice question

RandomTwo modelsOne model

• The formulation– The responses of a class on one question can be

represented with a vector: r = (n1, n2, …, nm), where m = total number of choices.

m

1ii Nn)

m

1

N

n(

1m

m

m

1i

2i

C

C has a value between 0 and 1

Combining the C factor with the scores, we can now model the different types of the responses:

– Low Score:Low Concentration –– LL

– Low Score:High Concentration –– LH

– etc.

Score Level C Level0~0.4 L 0~0.2 L

0.4~0.7 M 0.2~0.5 M0.7~1.0 H 0.5~1.0 H

Three level coding scheme to model the responses with “SC”, e.g. LH for low score and high C

Model the student responses with C

• The S-C plot

S-C Boundary

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

S

C

LL

LM

LH

MM

MH

HH

Constraints – C gives the overall concentration including the contribution from the score, which produces constraints on allowed regions.

S

C

%

%

The S-C State Density

N=100, m = 5

– One-Peak: Most of the responses concentrated on one answer. (LH or HH)

– Two-Peak: Most of the responses concentrated on two answers, usually one correct and one incorrect. (LM or MM)

– Non-Peak: Most of the responses somewhat evenly distributed among three or more answers. (LL)

Implications of Response Types

• Implications of the concentration factor (FCI)

Force and Motion Newton’s Third LawAnswer % Type Answer % Type

5-c 58% LM 2-a 66% LH9-c 45% LM 11-d 43% MM18-a 63% LM 13-c 68% LH22-c 66% LH28-d 51% LM

A LH or LM type of response often implies the existence of a common incorrect student model related to the physical context of the question

FCI Question #2 Response Type – LHImagine a head-on collision between a large truck and a small compact car. During the collision:

A the truck exerts a greater amount of force on the car than the car exerts on the truck.

B the car exerts a greater amount of force on the truck than the truck exerts on the car.

C neither exerts a force on the other, the car gets smashed simply because it gets in the way of the truck.

D the truck exerts a force on the car but the car doesn’t exert a force on the truck.

E the truck exerts the same amount of force on the car as the car exerts on the truck.

FCI Question #24 Response Type – LLA rocket drifts sideways in outer space from point "a" to point "b" as shown below. The rocket is subject to no outside forces. Starting at position "b", the rocket's engine is turned on and produces a constant thrust (force on the rocket) at right angles to the line "ab". The constant thrust is maintained until the rocket reaches a point "c" in space.

Which of the paths below best represents the path of the rocket between points "b" and "c"?

• The Concentration of the Incorrect Responses– The unbiased details about the distribution of the

incorrect responses can be obtained by removing the absolute offset created by the score. (there will be no constraints and can be any value between 0 and 1)

)1m

1

S)(N

Sn

(11m

1m

m

1i

22i

Now the score and are two independent variables.

Pre and post data graphical analysis

Tutorial

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

S

D

Pre-data

Post-data

Pre-average

Post-average

a) Traditional

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

S

D

Pre-data

Post-data

Pre-average

Post-average

b)

Overall (Traditional)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

S

Pre-data

Post-data

Pre-average

Post-average

b)Overall (Tutorial)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

S

Pre-data

Post-data

Pre-average

Post-average

a)

S-C plot for the overallresults

S- plot for the overallresults

S- plot for low performance groups (Traditional)

S- plot for low performance groups (Tutorial)

Low-Performance (Traditional)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

S

Pre-data

Post-data

Pre-average

Post-average

2

1322

5

15

18

289

24

Low-Performance (Tutorial)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

S

Pre-data

Post-data

Pre-average

Post-average

2

13

22

5

15

18

289

24

Mid-Performance (Tutorial)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

S

Pre-data

Post-data

Pre-average

Post-average

Mid-Performance (Traditional)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

S

Pre-data

Post-data

Pre-average

Post-average

S- plot for mid performance groups (Tutorial)

S- plot for mid performance groups (Traditional)

Applications

• Evaluate instruments– Compare FCI and FMCE

• Facilitate test design– Conductivity (L. Bao, M. Wittmann, and E. Redish)– Quantum (L. Bao, and E. Redish)– Astronomy (B. Hufnagel)– Wave Test (M. Wittmann)

• Facilitate instruction– Evaluate student model condition– Study the intermediate states in learning

(or bridging process if the instruction is such designed)