9
The readability of selected A-level biology examination papers lane Richardson and Roger Lock This pilot study sets out to determine whether A-level students understand the meaning of questions set in JMB and AEB biology papers Introduction This pilot study was carried out to determine whether the language used in A-level examinations allows students to demonstrate their biological knowledge and understanding. Complex language may act as a barrier between examiner and candidate and is likely to hinder performance. We were also concerned to investigate whether different examination boards phrase their questions in equally complex language. If not, students entering examinations set by one board . Abstract .... ~ .•.. Students' understanding of the language_used in selected A-level biology examination papers was investigated ili three ways. First, a number of potentially ·difficvlt ·words were presented in their original sentence context to students, who were asked to Indicate their level of understanding and to suggest suitable synonyms. . Secondly, question samples from .the Joint Matriculation Board [JMB) and the Associated Examination Board [AEB) A-level biology papers were ahalysed using readability formulae. A third approach sampled questions set by the two Boards ond onalysed the incidence· of Graece-latin words thot were induded ... Results from the first approach suggest that many A-level students ore not confident of the meanings of several words commonly used on A-level biology examination papers. Results from the second approach Indicate that the readability of parts of AEB and JMB papers is· unsuitable for unsupervised reading by 1B-yeor-olds of average ability. The results obtained using the Graece-latin instrument showed that text samples that were less readable also had 0 high incidence of Graece-latin words, suggesting thot these texts may be conceptually difficult and daunting to students. Key words: A-level biology, Exa'r'ination ·pClpers, Readability. may be disadvantaged, compared to students entering the same examination set by another board. Teaching and examining are rather different activi- ties. One aim of teaching may be to broaden the language experience of students. Examinations are not usually a vehicle for further learning: their prim- ary aim is to find out what students know and can do. The language used in examinations, therefore, must aim to communicate with clarity between the ex- aminers and examinees. Examiners must state dis- tinctly what is expected of candidates, and this must be understood by the vast majority of them. The role of the language used in examinations is not language education. Questions should, as far as possible, be phrased in language that the vast majority of candi- dates can understand. It is our experience that some candidates claim that they do not understand A-level biology ques- tions, although they may be able to answer them once they have been 'translated' into simpler English. In some cases, it seems that it is the language in which the questions are phrased that is problematic, rather than the biological subject matter. Although technical language may have a vital role to play in science, students often find it difficult. One reason may be that the language register of school subjects, including science, tends to be rather imper- sonal and uses terms in ways which are not part of the students' everyday language. As this terminology is alien to the student, it can act as a barrier to learning. Barnes (1969) questions the functions of subject specific registers and considers that they are partly due to history and convention, and not simply due to a desire for clear thinking and expression. Teachers tend to be aware of technical terms and these are generally presented to students (Barnes. 1969). However, there is a danger that the teaching of Journal of Biological Education (1993) 27 (3) 205

The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

The readability ofselected A-level biologyexamination papers

lane Richardson and Roger Lock

This pilot study sets out to determine whether A-level students understand the

meaning of questions set in JMB and AEB biology papers

Introduction

This pilot study was carried out to determine whetherthe language used in A-level examinations allowsstudents to demonstrate their biological knowledgeand understanding. Complex language may act as abarrier between examiner and candidate and is likelyto hinder performance. We were also concerned toinvestigate whether different examination boardsphrase their questions in equally complex language. Ifnot, students entering examinations set by one board

. Abstract .... ~ .•..Students' understanding of the language_used in selectedA-level biology examination papers was investigated ilithree ways. First, a number of potentially ·difficvlt ·words

were presented in their original sentence context tostudents, who were asked to Indicate their level ofunderstanding and to suggest suitable synonyms. .

Secondly, question samples from .the Joint MatriculationBoard [JMB) and the Associated Examination Board [AEB)A-level biology papers were ahalysed using readabilityformulae. A third approach sampled questions set by thetwo Boards ond onalysed the incidence· of Graece-latin

words thot were induded ...

Results from the first approach suggest that many A-levelstudents ore not confident of the meanings of severalwords commonly used on A-level biology examination

papers. Results from the second approach Indicate that

the readability of parts of AEB and JMB papers is·unsuitable for unsupervised reading by 1B-yeor-olds of

average ability. The results obtained using theGraece-latin instrument showed that text samples that

were less readable also had 0 high incidence ofGraece-latin words, suggesting thot these texts may be

conceptually difficult and daunting to students.Key words: A-level biology, Exa'r'ination ·pClpers,

Readability.

may be disadvantaged, compared to students enteringthe same examination set by another board.

Teaching and examining are rather different activi­ties. One aim of teaching may be to broaden thelanguage experience of students. Examinations arenot usually a vehicle for further learning: their prim­ary aim is to find out what students know and can do.The language used in examinations, therefore, mustaim to communicate with clarity between the ex­aminers and examinees. Examiners must state dis­tinctly what is expected of candidates, and this mustbe understood by the vast majority of them. The roleof the language used in examinations is not languageeducation. Questions should, as far as possible, bephrased in language that the vast majority of candi­dates can understand.

It is our experience that some candidates claimthat they do not understand A-level biology ques­tions, although they may be able to answer them oncethey have been 'translated' into simpler English. Insome cases, it seems that it is the language in whichthe questions are phrased that is problematic, ratherthan the biological subject matter.

Although technical language may have a vital roleto play in science, students often find it difficult. Onereason may be that the language register of schoolsubjects, including science, tends to be rather imper­sonal and uses terms in ways which are not part of thestudents' everyday language. As this terminology isalien to the student, it can act as a barrier to learning.

Barnes (1969) questions the functions of subjectspecific registers and considers that they are partlydue to history and convention, and not simply due toa desire for clear thinking and expression.

Teachers tend to be aware of technical terms andthese are generally presented to students (Barnes.1969). However, there is a danger that the teaching of

Journal of Biological Education (1993) 27 (3) 205

Page 2: The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

./ ..../~~d~6fwra(Gmm6mm ~\\u\}n\Ri~\\~ytl'5t)\\hM Lock'11 • :"'JIII'.\ •••••••••• II.IalO\

........ ~ ..". Methodsterminology may be over-empha~i2ed, and can takeon a value of its own. This can prevent the teacherfrom noticing poor student comprehension of under­lying concepts (Barnes, 1969).

Many scientific terms are derived from Greek andLatin words. These terms, such as polymorphonu­clear granulocyte, can seem long and difficult andstudents, therefore, may see the concepts that theyrepresent as irrelevant (Evans, 1973; Merzyn, 1987).Such terms may hinder learning and, if used inexamination papers, examination performance.

It is of vital importance that the readability of anywritten material should be matched to the readingabilities of the students to whom it will be presented.This is particularly important when the writtenmaterial is an examination paper. However, it is wellknown that some examination candidates are unable

to understand the questions in front of them (John­stone and Cassels, 1978; Otterburn and Nicholson,1976). Despite this, examiners may seem to take it forgranted that students will understand the words thatthey use in examination papers (Mobley, 1987). If astudent misunderstands a question in an examination,it cannot be rephrased and clarified as it could be in aface-to-face situation. Therefore, a student may give a'correct' answer, but to the 'wrong' question (Mobley,1987). Questions should be understood by all of thecandidates, and examiners should aim to write ques­tions that are free from bias, such as assumed generalknowledge (Mobley, 1987). If this is not the case,some students may be tested, not on their understand­ing and ability in the examination subject, but ontheir familiarity with the particular style of Englishused in examination papers.

Examination questions are usually written in aformal style with frequent use of passive impersonaland imperative verbs (Threadgold, 1982). Althoughthis style oflanguage is conventional in examinations,it is far removed from students' spoken language(Mobley, 1987). However, candidates may becomeused to this style if they are exposed to it frequentlyenough (Threadgold, 1982). They may be taughtspecifically how to cope with this form of language intheir lessons, where it may be referred to as 'examina­tion technique'. In their lessons, students also learnthe language style in which they are expected to writetheir answers. This style should be consistent, so thatstudents know what to expect in questions and whatexaminers expect in answers. Students may havedifficulties if these conventions are not followed.

This study focused on the language used inselected A-level biology examination papers pro­duced by the Associated Examining Board (AEB) andthe Joint Matriculation Board (JMB). The aims wereto discover if both Boards present students withlanguage of the same difficulty; to see if the languageof the papers is consistent within sections of a paperand from year to year; and to see if students areconfident of the meaning of selected words used in A­level biology examination papers.

1. Student perception of word comprehensionTwenty words were selected for study drawn froJMB A-level biology papers set in 1984, 1985 or 198The words were underlined and presented in tloriginal question context to 19 A-level biology stdents in the first year of their course (term 3). Stdents were asked to indicate their level of understan

ing using a four-point Likert Scale. Nine words weselected because they appeared in the Cassels alJohnstone (1978) list of words (marked + in table'The further 11 words were chosen because they wethought to cause students difficulty, and were COlmonly encountered in A-level examination papers.

A disadvantage of this method is that studermay claim, either deliberately or mistakenly,understand words with which they are not convesant. To circumvent this problem, students were alasked to indicate an alternative word or phrase thcould have been used instead of the underlined wor

2. Readability formulae studyReadability formulae require cautious and judiciouse, not least in a study of this type. They give:indication of the age of an average student who couunderstand the material, with an error of at least :f

year for the most valid and reliable tests. In this studwhere the students were not of average ability, bsome of the most able in the age group, further carerequired in application of the formulae and in tinterpretation of the data that they provide.

In selecting which formulae to use, this study wguided by Lunzer and Gardner (1979). After cloconsideration of nine formulae, they chose to use tFlesch grade score because it correlates well wipooled teacher judgements and is quite easy to use.addition, the tool is widely used in the literature alhas been applied in parallel studies to the current 0(Williamson, 1986). Word passages were selectedrandom and the formula was applied but where tquestion style was inappropriate to the Flesch teenique it was not employed, for example multipchoice sections (JMB) and the short question secti,of the AEB paper I.

Although readability formulae are imperfect toothe results obtained from them are more reliable

confirmed by the results of other formulae, especiaif they rely on different parameters in their compu1tion. For this reason, another formula was also useThe Forcast formula was selected as it is valid wi

material aimed at this age group, and can be uswith text that is not written in proper sentences.can, therefore, be used with those sections of tpapers which cannot be analysed using the Flesformula.

Formulae rejected for this study included the Fgraph, Mugford, FOG and SMOG formulae. TFry graph is not easily applied to short sentences wia large number of long words, or to long sentenc

206 Journal of Biological Education (1993) 27 (3)

Page 3: The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

The readability of selected A-level biology examination papers Richardson and Lock

terminology may be over-emphasized, and can takeon a value of its own. This can prevent the teacherfrom noticing poor student comprehension of under­lying concepts (Barnes, 1969).

Many scientific terms are derived from Greek andLatin words. These terms, such as polymorphonu­clear granulocyte, can seem long and difficult andstudents, therefore, may see the concepts that theyrepresent as irrelevant (Evans, 1973; Merzyn, 1987).Such terms may hinder learning and, if used inexamination papers, examination performance.

It is of vital importance that the readability of anywritten material should be matched to the readingabilities of the students to whom it will be presented.This is particularly important when the writtenmaterial is an examination paper. However, it is wellknown that some examination candidates are unable

to understand the questions in front of them (John­stone and Cassels, 1978; Otterburn and Nicholson,1976). Despite this, examiners may seem to take it forgranted that students will understand the words thatthey use in examination papers (Mobley, 1987). If astudent misunderstands a question in an examination,it cannot be rephrased and clarified as it could be in aface-to-face situation. Therefore, a student may give a'correct' answer, but to the 'wrong' question (Mobley,1987). Questions should be understood by all of thecandidates, and examiners should aim to write ques­tions that are free from bias, such as assumed generalknowledge (Mobley, 1987). If this is not the case,some students may be tested, not on their understand­ing and ability in the examination subject, but ontheir familiarity with the particular style of Englishused in examination papers.

Examination questions are usually written in aformal style with frequent use of passive impersonaland imperative verbs (Threadgold, 1982). Althoughthis style of language is conventional in examinations,it is far removed from students' spoken language(Mobley, 1987). However, candidates may becomeused to this style if they are exposed to it frequentlyenough (Threadgold, 1982). They may be taughtspecifically how to cope with this form of language intheir lessons, where it may be referred to as 'examina­tion technique'. In their lessons, students also learnthe language style in which they are expected to writetheir answers. This style should be consistent, so thatstudents know what to expect in questions and whatexaminers expect in answers. Students may havedifficulties if these conventions are not followed.

This study focused on the language used inselected A-level biology examination papers pro­duced by the Associated Examining Board (AEB) andthe Joint Matriculation Board (JMB). The aims wereto discover if both Boards present students withlanguage of the same difficulty; to see if the languageof the papers is consistent within sections of a paperand from year to year; and to see if students areconfident of the meaning of selected words used in A­level biology examination papers.

206 Journol of Biological Educotion (1993) 27 (3)

Methods1. Student perception of word comprehensionTwenty words were selected for study drawn fromJMB A-level biology papers set in 1984, 1985 or 1986.The words were underlined and presented in theoriginal question context to 19 A-level biology stu­dents in the first year of their course (term 3). Stu­dents were asked to indicate their level of understand­

ing using a four-point Likert Scale. Nine words wereselected because they appeared in the Cassels andJohnstone (1978) list of words (marked + in table 1).The further 11 words were chosen because they werethought to cause students difficulty, and were com­monly encountered in A-level examination papers.

A disadvantage of this method is that studentsmay claim, either deliberately or mistakenly, tounderstand words with which they are not conver­sant. To circumvent this problem, students were alsoasked to indicate an alternative word or phrase thatcould have been used instead of the underlined word.

2. Readability formulae studyReadability formulae require cautious and judicioususe, not least in a study of this type. They give anindication of the age of an average student who couldunderstand the material, with an error of at least ± Iyear for the most valid and reliable tests. In this study,where the students were not of average ability, butsome of the most able in the age group, further care isrequired in application of the formulae and in theinterpretation of the data that they provide.

In selecting which formulae to use, this study wasguided by Lunzer and Gardner (1979). After closeconsideration of nine formulae, they chose to use theFlesch grade score because it correlates well withpooled teacher judgements and is quite easy to use. Inaddition, the tool is widely used in the literature andhas been applied in parallel studies to the current one(Williamson, 1986). Word passages were selected atrandom and the formula was applied but where thequestion style was inappropriate to the Flesch tech­nique it was not employed, for example multiple­choice sections (JMB) and the short question sectionof the AEB paper 1.

Although readability formulae are imperfect tools,the results obtained from them are more reliable ifconfirmed by the results of other formulae, especiallyif they rely on different parameters in their computa­tion. For this reason, another formula was also used.The Forcast formula was selected as it is valid withmaterial aimed at this age group, and can be usedwith text that is not written in proper sentences. Itcan, therefore, be used with those sections of thepapers which cannot be analysed using the Fleschformula.

Formulae rejected for this study included the Frygraph, Mugford, FOG and SMOG formulae. TheFry graph is not easily applied to short sentences witha large number of long words, or to long sentences

Page 4: The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

The readability of selected A-level biology examination papers Richardson and Lock

with relatively short words. Both of these sentencetypes occur in examination questions. The Mugfordformula is quite time-consuming to use, requiringfour lists of words to be compiled. The FOG andSMOG formulae both tend to give very high scoreson more difficult reading material, which can makeresults difficult to interpret. Both of these formulaeare easy to use but FOG does not correlate well withpooled teacher judgements, and SMOG has a greaterstandard error than other formulae.

Table 1 Do students think that they understand selected wordsfrom JMB A-level biology examination papers? (N= 19)

- indicates words with a combined C+ D score of less than 79.+ indicates words studied by Cassels and Johnstone (1978).

Level of student understanding- % of studentswho had:No Someidea ideaA B

Understood C+ DperfectlyD

A goodideaC

Principles 5323726 63-

Principal0262153 74-

Mobilized16163242 74-

Concluded0104247 89

Yielded1002163 84

Devise0103753 90

+Factors005347 100

+Valid1652653 79

+Contrast5102658 84

+Efficient5105826 84

Respectively10103742 79

Verify2152647 73-

+Hypothesis5323232 64-

Retard26163226 58-

+Concept0423721 58-

+Rate053758 95

Correlated21262626 52-

+Probability5163247 79

Comparable555832 90

+Effects055837 95

results do seem to suggest that a significant propor­tion of A-level students are not confident of themeanings of a number of words commonly used inJMB A-level biology examination questions. How­ever, because students may think that they under­stand a word that they do not, or may be unsure of aword that they understand perfectly, they were alsoasked to suggest alternative words or phrases thatcould have been used for the 20 selected words. The

results have been tabulated (table 2) according to thestudents' stated level of understanding of the selectedword.

As would be expected, alternative words weremost frequently suggested when students claimed tounderstand words perfectly, and were never suggestedwhen students had 'no idea'. Usually, if studentssuggested an alternative when they indicated onlyhaving 'some idea' of a word's meaning it was broadlycorrect. For example, a student suggested that 'slowdown' could have been used instead of 'retard',although the student claimed to have only 'some idea'of the meaning of that word. In other cases, studentssuggested incorrect alternatives to words which theyclaimed to 'understand perfectly', although this oc­curred more rarely (seven times in total). For ex­ample, 'variable' was offered as an alternative to'comparable' by a student who claimed to understandthis word perfectly. Both of these circumstances couldlower examination performance .. Students 'under­standing' a word incorrectly, or worrying about themeaning of a word (even though they understand it)are likely to make more errors.

1. Student perception of word comprehensionTable I shows the results of student understanding ofthe 20 selected words as used in the same context in

which they were employed in recent JMB A-levelbiology papers. From the table it can be seen that 11of the words were not well understood by 20 per centor more of the students tested; in other words, morethan 20 per cent of the students indicated that theyhad 'no idea', or 'some idea' of the meaning of thewords. In three cases over 20 per cent of the studentshad 'no idea' of the words' meaning. Very few of thewords were well understood by 90 per cent or more ofthe class, although it seems that these are the levels ofword comprehension that examination questionwriters should aim for.

This pilot survey was carried out using an oppor­tunity sample-a single class of A-level students­and therefore it may not be possible to generalizeusing these results. Most of the students were at theend of the first year of a two-year course and so theirlanguage comprehension may not have been at thelevel it would have reached by the time they sat theirexaminations. However, the words tested were non­technical rather than technical, and are unlikely to bespecifically taught in lessons. These words are likelyto be already causing problems if past examinationquestions are used for classwork, homework andtests. Even with the reservations outlined above, these

Results and discussion

3. The Graeco-Latin instrument

The Graeco-Latin (GL) instrument (Corson, 1982)provides an alternative method of measuring textreadability. Corson has noted that the percent inci­dence of GL words seems to correlate with readingages of texts. This may be because GL words are oftenused to represent more abstract ideas than Anglo­Saxon words, and so their incidence gives an indica­tion of the semantic complexity of texts. GL wordsmay also be used in texts, as they have a high statusand make writing appear more academic. For both ofthese reasons texts with a high incidence of GL wordsmay be less readable than texts with a low incidence.In order to determine the percent incidence of GLwords, random passages of 100 consecutive wordswere selected from JMB and AEB papers. In manycases the passages chosen were the same ones ana­lysed using the readability formulae.

Journal of Biological Education (1993) 27 (3) 207

Page 5: The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

The readability of selected A-level biology examination papers Richardson and Lock

Table 2 Alternative words suggested by students to replace selected words in examination questions *(frequency of suggestion in brackets)

Word on

examination paperAlternative words suggested according to level of understanding of original wordB (Some idea) C (Good idea) D (Understands)

Principles

Principal

Mobilized

Concluded

Yielded

Devise

Factors

Valid

Contrast

Efficient

Respectively

Verify

Hypothesis

Retard

Concept

Rate

Correlated

Probability

Comparable

Effects

Main pointsMain objectivesFunctions

Main

Major (2)

Transported

Plan

Descri be

Compared to each other

Account for

Theories

Slow down

Stop

IdeaDifference

Related

Chance (2)

Different, but with certain links

Main points

Main

TransportedMoved aroundUsedSaidUnderstoodDeductedGave

PlanThink of

Conditions

Facts affectingWhat makes it upWhat it consists ofUseful

Differences (3)Differentiate

How well

ApproximatelyOne after another

TogetherProve (3)

Theories (3)ExplanationExamples

Slow down (2)StopReduceIdea

Meaning (3)Idea/theorySpeed (3)Amount

Related (2)

PossibilityLikelyLess frequentSimilar (4)

Changes

Main pointsBasisGeneral idea

Most important ideasBasic ideas behind

Main (4)*MajorChiefAbundant

Transported (3)Moved around (4)Made available to seed

Said (3)Determined (3)Found out

Produced (9)Made (2)Plan and designMake-up (3)Construct (3)Make/designCreate

Think-upPlan/make-upConditions (3)Points (2)

True (9)Correct

Differences (6)DifferentiateDissimulate

CompetentGood, well achieved

RelativelyOrder of words

Firstly 02' secondly glucose (3)Prove (8)Check

Theories (2)IdeasReasons

AssumptionSlow down (3)StopHinder

Idea (2)Meaning

SpeedPace (2)AmountTime taken

Amount of water per unit timeRelated (4)Connected

Chance (6)Likelihood

Similar (3)EquivalentVariable

ConsequenceWhat happensResults

What certain factors have on things

208 Journol of Biological Education (1993) 27 (3)

Page 6: The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

The readability of selected A-level biology examination papers Richardson and Lock

Students did not always suggest alternatives for allwords; sometimes it may be difficult to think of analternative. Students may have misunderstood thealternative rather than the word itself (e.g. 'deducted'was given as an alternative to 'concluded'). However,in general terms, it was reassuring that alternativewords suggested were consistent with the level ofunderstanding indicated. If anything, studentsseemed to underestimate their own level of under­

standing, which may reflect a lack of confidence inusing this type of language.

2. Readability formulae studyThe readability formulae were employed to addressthree main questions concerning the variation withinand between papers of the same Board, the variationbetween papers of the same Board over a period oftime, and the variation between the Boards.

Three JMB examination papers (June 1983, June1985 and June 1987) and three AEB examinationpapers (June 1985, November 1985 and June 1987)were investigated and the data are presented in tables3 and 4 respectively. Data in table 3 show that the

Table 3 The reading ages of samples taken from 1MB A-level biology papers

Flesch formula

Forcast formula

Paper

QuestionNumber ofW ords/ReadingOne-syllableReadingnumber

syllablessentenceeasewordsage

June /983 lA

4182 110/530.58216.8lA

2163 109/753.19115.9

IB (A)

3185 106/632.48017.0

IB(C)

4159 119/755.19215.8

IB (F)

5158 113/654.18716.3IIA

4-6-- 7717.3IIA

16-20-- 7617.4IIA

31-34 8816.2lIB

2-3162106/754.28716.3lIB

5~7160121/753.99315.71983 mean

47.616.5

1983 SO10.30.6

June /985lA

3d177 109/638.68116.9

lA

5153 110/658.89115.9

IB (A)

3178 122/531.58516.5

IB(C)

4176 108/639.79515.5

IB (E)

I146 121/765.810214.8

IB (F)

4-5167105/750.39115.9IIA

6-7 --8716.3IIA

32-34 -9615.4

IIA

37-38 7717.3lIB

2-3156116/1063.19715.3

lIB

4-5163107/753.49215.8

1985 mean

50.216.0

1985 SO

11.70.7

June /987lA

2162 109/855.99016.0lA

6179 122/737.78716.3

IB (A)

5157 113/1062.594/14815.5

IB(C)

I164 105/650.69715.3

IB (F)

2-4189102/935.48216.8IIA

4-5 9615.4

IIA

14-17 9315.7IIA

32-33 8216.8

lIB

1-3164105/752.98516.5lIB

4-6170109/849.28616.41987 mean

49.216.11987 SO

8.90.6

Overall mean

49.016.2Overall SO

10.70.7

Journal of Biological Education (1993) 27 (3)

209

Page 7: The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

The readability of selected A-level biology examination papers Richardson and Lock

Table 4

The reading ages of samples taken from AEB A-level biology papers

Flesch formula

Forcast formula

Paper

QuestionNumber ofWords/ReadingOne-syllableReadingnumber

syllablessentenceeasewordsage

June 1985 I]--4 --8816.2

I8-9 -9016.0

22141 107/67010015.0

251153 108/659103]4.7

35141 107/566108]4.2

34166 114/851.911213.8

1985 mean61.715.0

] 985 SO6.90.9

November 1985I

4-7 10414.6I

1{}-12 9615.42

I]52 131/75510314.72

2167 108/7509615.42

311175120/8458516.53

4-5185110/52884]6.63

5-7156102/657.69715.31985 mean

47.115.51985 SO

10.50.7

June 1987I

1-3 9815.2I

]{}-] 1 8416.6I

17-18 9415.6

2 comp.2182 100/635.98516.5

2]-3172103/949.79115.9

2 data4]39 105/567.9102]4.8

]987 mean51.2]5.8

1987 SO]3.10.7

Overall mean

53.0]5.4Overall SO

12.10.8

reading ages obtained using the Forcast formularanged from 14.8 to 17.4 years. The Reading EaseScores varied from 30.5 to 65.8 which can be con­verted to 18 to 21 and 13 to 14 years respectively.These data seem to confirm the idea that considerablevariation can occur within sections of examinationpapers. The data also confirm that the reading age ofsame samples is appropriate for independent readingby students. It is also seen that the two formulae canpredict different reading ages for the same piece oftext.

Similar features are shown by the results obtainedby analysis of samples taken from AEB examinationpapers (table 4). A wider range of reading ages wasobtained using the Flesch formula from graduate to13 to 14 years, while a similar spread of values wasobtained using the Forcast formula. However, theseresults show a lower range of reading ages, using theForcast formula, than the results obtained from 1MBpapers.

When the data in tables 3 and 4 are inspected toinvestigate whether differences in reading age arefound between papers of the same Board from dif­ferent years, it is seen that a good degree of consist-

21 0 Journal of Biological Education (1993) 27 (3)

ency exists. It is important that the reading age ofpapers remains steady so that candidates know whatto expect. However, language variation within andbetween papers may reflect differences in the conceptsbeing examined in different sections, or papers oryears.

The overall means were computed by averaging allthe data from all questions investigated, and are usedto give some impression of the overall readability of1MB and AEB papers. These data suggest that 1MBpapers have a higher reading age than AEB papers.These differences in readability were significant at the5 per cent level for the Forcast formula (standarderror of difference, z = 3.36, p < 0.05) but not for theFlesch reading ease score (z= - 0.97, p> 0.05).

3. The Graeco-Latin instrumentThe results from this approach are presented in table5. They show a wide variation in the incidence ofGraeco-Latin words in the parts of the examinationpapers sampled. Among the AEB samples the inci­dence varied from 18 to 44 per cent but there wasrather less variation among the 1MB samples (from25 to 43 per cent). The standard deviation of the 1MB

Page 8: The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

Table 5 The incidence of Graeco-Latin words in passages selectedfrom AEB and JMB examination papers

The readability of selected A-level biology examination papers Richardson and Lock

incidence of Graeco-Latin words among text samplesselected from AEB and JMB examination papers.7. The results indicate that the language used onJMB papers is more consistent than on those pro­duced by the AEB.

Summary

The sample of students used in this study was smalland this may limit a generalization of the findings.However, there are seven main points that haveemerged from this study.

samples was less than that of the AEB samples, whichmay suggest that the language used in JMB papers ismore uniform than that used in AEB papers. How­ever, it may also reflect a greater variety of questionstyles used on AEB papers. The mean incidence ofGraeco-Latin words was broadly similar for the twoBoards and not significantly different.

1. A significant proportion of the sample of studentsused in this study are not confident of the meaning ofa number of words commonly used in JMB A-levelbiology examination papers.2. Students in this study tend to underestimate theirlevel of understanding of words used in A-level ex­amination papers.3. Considerable variation in readability occurs withinsections of AEB and JMB examination papers.4. A good degree of consistency exists in the readingages of papers of the same Board from different years.5. JMB papers may be more difficult to read thanthose produced by the AEB.6. There is quite a wide variation in the percent

Board

AEBAEBAEBAEBAEBAEBAEBAEBAEBAEB

JMB~B~B~B~B~B~B~B~B~B

DateNovember 1985

June 1983June] 985June] 985June 1985November 1985November 1985November 1985June ]987June 1987June 1987

June 1982June 1978June 1985June 1985June 1985June 1985June 1987June 1987June 1987June 1987

Paper Question

2 511 1-22 53 5\ 4-62 3-43 41 12 12 4

MeanStandarddeviation

IIB 8IIB 1lA 3DIB 3

lIA 32-34lIB 4-5lA 6IB A5

IIA 32-33lIB 5-7

MeanStandarddeviation

Incidence ofGraecD-Latinwords %

313919282639413644\8

32.1

8.7

323440352525

4038434035.\

6.0

Implications

Biology teachers need to be aware of the words inexamination questions which may be problematic totheir students. Key terms could be explicitly taughtand students could be encouraged to keep a list of themeanings of the terms for reference. It may be usefulto have an English language dictionary available inthe biology laboratory. When choosing an A-levelExamining Board, it may be useful for teachers toconsider how different Boards make different lan­guage demands on candidates. Teachers usually takethe syllabus content and assessment scheme intoaccount when choosing an Examining Board, butlanguage levels are an added factor to consider.Teachers could administer readability tests on exami­nation papers from a shortlist of Boards beforemaking their final decision.

The Examining Boards could reduce the numberof synonyms to be taught if they published a list ofwords with which they expected students to be fami­liar. Alternatively, they could publish lists of termsthat would not be required. The lists could be revisedperiodically, to prevent them from being too restric­tive. Boards could reduce the use of technical terms in

questions, and could place common names or expla­nations in parentheses if necessary. Where possible,Anglo-Saxon words should be used in preference toGreek or Latin derived words. Examining Boardscould formalize procedures for monitoring the reada­bility of the examination papers that they set, perhapsin a similar manner to the trialling that they carry outon multiple-choice questions. If this procedure couldbe agreed by the various Boards, the results could bepublished and used comparatively by teachers.

Boards could test the linguistic abilities of candi­dates by setting comprehension and essay questions.Otherwise, they should ensure that the language oftheir examinations communicates with the candidates

as clearly as possible.

AcknowledgementWe would like to thank Tessa Carrick who was involvedin the early stages of this work.

References

Barnes, D. (1969) Language, the learner and school.London: Penguin.

Cassels, J. and Johnstone, A. (1978) The understanding ofnon-technical words in science. London: The ChemicalSociety Education Division.

Corson, D. J. (1982) The Graeco-Latin instrument: a new

Journal of Biological Education (1993) 27 (3) 211

Page 9: The readability of selected A-level biology examination papersselected A-level biology examination papers pro duced by the Associated Examining Board (AEB) and the Joint Matriculation

The readability of selected A-level biology examination papers Richardson and Lock

measure of semantic complexity in oral and writtenEnglish. Language and Speech, 25(1), 1-10.

Evans, J. D. (1973) Towards a theory of technicalcommunication. School Science Review, 55(191),233-241.

Johnstone, A. and Cassels, J. (1978) What's in a word?New Scientist, 78, 432-434.

Lunzer, E. and Gardner, K. eds. (1979) The effective use ofreading. London: Heinemann.

Merzyn, G. (1987) The language of school science.JllIernational Journal of Science Education, 9(4),483-489.

Mobley, M. (1987) Evaluating curriculum materials: unit 2,readability. York: Longman.

Otterburn, M. K. and Nicholson, A. R. (1976) The

language of CSE mathematics. Mathematics in Schools,5(5), 18-20.

Threadgold, R. (1982) The problem of register in selectedCSE examinations across the curriculum. Reading,16(3),169-179.

Williamson, J. (1986) CSE; an examination of reading.Reading, 20(2), 111-l20.

The authorsJane Richardson is a Science Lecturer at the Bournville

College of Further Education, Bristol Road South,Northfield, Birmingham B3I 2AJ. Roger Lock is a Lecturerin Science Education at the School of Education, Universityof Birmingham, Edgbaston, Birmingham BI52TT.

BIOLOGICAL NOMENCLA TURE* 'I comnwnd the In:-.titute of BJOlogy for thii'

initiatin~ which :-ihould han> a significant im­

pact on the e~tablishnwl1t of a slandardist'dnomenclature in Colleges and Schools.'

.J.R. ~lerchant

Director. Council Polit.', andA.dministration. SERl'

* 'Any authoritatiH' attl'mpt to rationalise scien­tific material which Jl:'nds ihelf to confusion or

contention is to be welcomed and this publica­

tion is particularl .••useful in addn>ssing aspectsof biolog~· which remain ·grey areas· to morepeoplt· than would can" to admit it~·

P.E.O. \\\ml'l'Co-ordinator: :\ational Centre for

School Biotechnology

* '~lany childn>n find learning science hardenough without the added hurden of confusioncaused b~' non·standardi~ed nomenclature andsymbols. It seems to me essential that standar­disation be introdul't'd, It may mean thatcertain schools or teachers ma~' have to changethe habits of a lifetime, but morl' likely Isuspect it will bring order to what has bl>en arather haphazard and 81(Jpp~' system in tht'past. I am sure that. if followed. thl:' rt't'ommt'n·dations will do much to ht'lp avoid unnecessary

ambiguity'..J. Wilkinson

Science Correspondent. BBC

* 'At a time when we are preparing to introduceXational Curriculum Science into our Schools I

welcome this initiative h:,>' the Institute ofBiolog:,>' and the Association for Science Educa­tion to produce recommendations on tht:'nomenclaturt:'. units and symbols commonly

encountt:'red in biological education in schoolsand collt:'ges, I am sure that this publicationwill be useful for teachers and pupils.'

Rt Hun Kenneth Baker .M1;.

The former Secretary of State forF..ducation and Science

* 'Science ha~ it~ o\\"n language and it is essentialto learn and use that language in order tocommunicate sensibl .••.and accurately, :\owhereis this trUl'r than in Biolo~{.v and in the rn'risenaming of animals and plants, Thl' Institute ofBiology's new re('ommendation:, are clear andas simple as pos:'.ible: they will p"o\'e \'aluableto student:-i and staff alike"

Sir (;eorge Porter.President. The Ro .••.al Society

* 'In rapidly ad\'ancing .'-;('ience,,,;, like biology. a

proliferation of technical terms and conventio­nal s~'mbol:-i is almost inevitabl",. An .••.thing that

bring:-; uni!()I'mity to their use and st .••.ling. andwhich t'liminates redundant. inaccurate or out·

moded variants is greatl .••.tn bl:' welcomed. And

that is just \\'hat 'thi:; booklet does:Sir DfI\'id Attenborough. CBE

HOW TO ORDER

This publication is the first tomake recommendations onagreed practice with regard tobiological terms in schools andcolleges.The recommendations, which seta standard for biologicaleducation, have been approvedby:* the former SEC* examinations boardsl

groups* educational publishers* others in the education

sectorThis publication is available for£4.00.(£3.00 to lOB and ASE membersstating membership number - forpersonal, not institutional,orders).

Cheques and postal orders should be made payable to: 'Institute of Biology'.

Send your order to: The Institute of Biology, 20 Queensberry Place, London SW7 2DZ.

Orders must be accompanied by remittance in full

Note: Members' discounts are for members' personal. (not institutional) orders only

If all the goods ordered are free, please enclose a stamped addressed A5 envelope (18 cm x 25 cm)

Name (block capitals) . Address

Member C:=J Membership No. (if known) Non-member [

Total remittance enclosed £.. . Date . Replacement order form required

21 2 Journal of Biological Education (1993) 27 (3)