Validity and Reliability

Example of Past Paper Answer – 2007 Q6

Definitions – Reliability

Definition: Reliability is the consistency of your measurement, or the degree to which an instrument measures the same way each time it is used under the same condition with the same subjects. In short, it is the repeatability of your measurement. A measure is considered reliable if a person's score on the same test given twice is similar.

Definitions – validity

Truthfulness: Does the test measure what it purports to measure? the extent to which certain inferences can be made from test scores or other measurement. (Mehrens and Lehman, 1987)

The degree to which they accomplish the purpose for which they are being used. (Worthen et al., 1993)

Evidence for valid and reliable

the survey has been carried out by an independent body (MORI)

carried out on behalf of a reputable foundation the sample size was fairly large (c 2000) asks a wide range of perception questions that

gauge public opinion well the survey was carried out over 3 years, so trends

can be identified fairly recent research.

Evidence against valid and reliable

“representative sample of British residents” – no information about sample other than that

percentages – some add up to over 100, some to less, with no explanation offered to explain these anomalies

no information on the details of the survey eg location, social background of the respondents, etc

the unclear instruction about how many to choose from.