SUBORDINATING EXAMINATIONS TO RELEVANCE: IMPRESSIONS FROM KENYA

by John Oxenham

ABSTRACT

Examinations are often seen as obstacles to the achievement of the higher goals of education and to making curricula relevant to the great majorities of pupils who enjoy only a basic education and who, in developing countries, are not likely to secure wage/salary employment. The expense of techniques which would both test and promote good education are sometimes thought to make them infeasible.

However, the Kenya Examinations Unit (now Examinations Council) has been gradually making the exam for the Certificate of Primary Education subserve objectives beyond factual recall and the routine, abstract application of simple skills. 'Causal' facts, observation, discrimination, manipulation of data, reasoning, problem-solving are now given substantial emphasis. So also is knowledge useful to young people who will not be able to continue their schooling.

Care is also taken to keep teachers informed of the changes. To, the usual array of training channels is added a Newsletter, which analyses questions and answers and advises on effective instructional approaches.

71

The menace of exams

Examinations are the enemy of education. This has been the judgement of a long line of educators (e.g. Dore, 1976). The influence of exams is particularly pernicious, when they are used to select young people for further education and salaried employment. What they test monopolises instruction and learning; what they do not test is neglected, even discarded. Reflections of this truth are seen in two articles elsewhere in this volume. McNamara is careful to point out that the Secondary Schools Community Extension Project in Papua New Guinea scrupulously conforms to the Grade X examination. Lichtenstein notes that the Junior Certificate exam was altered to help ensure the acceptance of a new science curriculum in Swaziland. Even then, teachers so worried about the exam, that revising for it became their major complaint against the innovation. It is not surprising then that selective examinations are deemed among the chief obstacles to education relevant to that great majority of children who enjoy only primary education or less, or who are the scholastically less able (Sinclair and Lillis, 1980; Cooper, 1981).

Even for the brighter children, however, exams bring distorted education. A large reason is perhaps the clash between the desirable and the feasible. The resources available to pay for exams cannot cope simultaneously with the pressure of rising numbers of candidates and the high expense of conscientious and compre- hensive examination techniques. Thc ensuing compromises have meant that even in 1981 many centralised exams rely overwhelmingly on paper and pencil tests and exclude most psychomotor attainments. Less necessarily, within the domain of paper and pencil they emphasise overwhelmingly factual recall, skills of routine application and 'unreal' problems. By default, they devalue the higher goals of comprehension, manipulating information,

reasoning, problem solving, creativity (Little. 1981 a, b; Lewin, 1980).

This is not to claim that the problems are either unrecognised or allowed to persist untackled. Indeed, the situation is quite to the contrary. Britain's throes at improving examinations are seemingly endless. India has a special unit in its National Council of Educational Research and Training devoted to examination reform. The overthrow of examinations was part of the Cultural Revolu- tion in China and their restoration is part of the post-Mao revolution. Tanzania is experi- menting with continuous assessment. The West African Examinations Council exhibits a gratifying flexibility in its anxiety to support 'relevant' education. The rest of this article will take a cursory look at one other example of attempts to make examinations promote the higher goals of education, namely the reforms in the Certificate of Primary Education (CPE) in Kenya.*

A Case In Kenya

The exam for the CPE is taken at the end of the 7-year primary school course. Its value as a terminal certificate is now almost nil, since even secondary school graduates face high probabilities of unemployment. However, it is critical as the selector for the government's secondary schools. As there are places for only about 10 per cent of the annual primary school graduates, ambitious young people have to score very well on the exam to be sure of one. Less successful pupils can enter the private l-Iarambee secondary schools, but these are considered, with much justification, a poor second best. Not unnaturally, the annual CPE fever is intense. To aid exam success, the bookshops display an abundance of 'crammers'. One publisher said that he sold some half million crammers annually and that his most successful author earned as many shillings in royalties. An analysis of the crammers' con-

* Until August 1980 the examination for the CPE was conducted by the Examinations Section of the Ministry of Education. Secondary school examinations were the responsibility of the East African Examinations Council, which originally catered for Kenya, Tanzania and Uganda, but in 1980 covered only Kenya and Uganda. In March 1980, the Kenyan Government formed its own National Examinations Council. which in August assumed responsibility for both primary and secondary examimnions. The substance of this article is drawn from publications of the Kenya Examinations Cotmcil. It also owes much to conversations with H. C. A. Somerset. until recently head of the Examinations Research and Dcvelopmcnt Unit. Any crrors, however, are my own.

72

tents shows that the authors believe that rote recall, routine application and standardised expression are the keys to success. In this, to judge by some of the mock exams devised by official educators, they arc at one with the teachers.

However, over the past decade, changes have been gradually formed and introduced. Their tendency is to encourage teachers to make their instruction more relevant to life out of school by shaping the exam to emphasise cognitive skills generally useful out of school. It is as well as to note at once that the exam remains confined to pencil and paper and that most of it has to be in the multiple choice format. The exception is the English essay, which will be looked at presently. This means of course that the psycho-motor skills, creativity and problem-solving of the arts and crafts are omitted and by that token devalued as elements of education. This drawback notwithstanding, the CPE exam indicates what can be achieved even within the confines of

paper and pencil, multiple-choice tests.

The levers used by the reforms have been (1) the kind of fact emphasised, when factual recall is desirable, (2) questions which call for skills rather than knowledge and (3) communications with the teachers.

What/acts?

Technical terms as facts have been heavily reduced. Students tend not to be directly asked, e.g. "What is a base? What is an acid?" Instead, definitions are given and the pupils are asked to apply them to a problem, as Question 61 from 1979 General paper illustrates:

61. Substances which turn the juice from red hibiscus flowers green are called bases. Substances which turn the hibiscus juice pink are called acids. Ali tested different substances with the hibiscus juice and recorded his results in the following chart:

lemon solution vinegar milk solution juice o/ ashes ol soap

turns pink yes no yes no no

lurns green no yes no no yes

Using the chart, which one of the following conclusions is correct?

A. Milk and soap solution are both acids.

B. Soap solution is a base and lemon juice is an acid.

C. Ashes solution and vinegar are both bases.

D. Vinegar is an acid and lemon juice is a base.

Clearly, what is being tested is not knowledge of terms but the ability to use information to judge inferences. The terms themselves are accorded their proper value, namely convenient classificatory labels. (Note also the use of a common local material, hibiscus juice, not imported litmus paper.)

In history, the emphasis is shifted from facts of time or person - - when?, who?, - - to facts of cause m why? This approach will not obviate the rote memorisation of, say, the causes of World War I. On the other hand, it will support and promote the educational goal of encouraging pupils to seek relationships between and explanations of events and phenomena. Similar instances could be adduced from the science and geography sections.

These two subjects also offer scope for testing facts from observation and experiment. For example, General paper question 43 enquired

43. Atieno's class visited a small market in the countryside. Which one of the following groups of activities are they likely to have seen there?

73

A. Bicycle mending m Jiko making Furniture making

B. Bicycle mending ~ Cement making Furniture making

C. Soft drink bottling - - Cement making Furniture making

D. Jiko making - - Soft drink bottling Bicycle mending

Not only does this question favour rural candidates, it encourages teachers to take their pupils out of the school to observe what actually is there. It also allows the non-rural or even unobservant rural pupils to use their reasoning on what might be found at a small market.

The Examinations Council also claims to test facts which will be 'terminally relevant', i.e. useful to the pupils who will not be selected for secondary school. The 1979 General Paper has at least 10 substantiations of the claim ( 11 per cent of the questions).

Skills rather than knowledge

Observing, discriminating, manipulating data and reasoning are the skills which the CPE fosters. Good illustrations can be found in the 1980 General Paper's Questions 32 to 38 m 7 out of 90 questions. A map is provided of a locality, showing township, plantations, factories, schools, railway, roads, rivers, bridges and dam. Orientation, scale and symbols are all present. In short, all the neces- sary data are supplied. The 7 questions test whether the pupils can use the data to form inferences from them, e.g. if a person is walking

from one point on the map to another at a certain time, his shadow will fall in which direction? A large town lies off the map: in which direction is it most likely to be? (Only one tarmac road leaves the map.)

Simpler, but equally useful, is question 65: Hadija's mother could not read the instructions on a packet of tablets given to her at the health clinic. Hadija made a chart to help her mother remember how many tablets to take and at what times she should take them. The instructions were:

'Take ONE tablet early in the morning and TWO late at night"

Which of the following charts did Hadija make to CORRECTLY show the instructions?

There follow four charts with the sun and moon in different positions, from which the pupil has to reason out the correct one.

More complex and entirely pertinent to a predominantly agrarian economy is the information offered for answering questions 86 and 87:

Scientists at Sangalo in Western Kenya carried out experiments to find out how far apart maize seeds should be planted in the rows to get the heaviest crop. The results they obtained with three different types of maize (Hybrid 613, Hybrid 511 and Katumani B) were as follows

Distance seeds planted apart

15 cm

30 cm

45 cm

Weight oJ crop per hectare with

Bybrid 613

2900 kg

3100 kg

3200 kg

Hybrid 511

3000 kg

3600 kg

4500 kg

Katumani B

4600 kg

3400 kg

2500 kg

86. Judging from these results, which one of the following would be the best advice to give to farmers in Sangalo to get the heaviest crop? (4 choices)

87. Judging from the same results which one of the following is CORRECT? A. When seeds were planted 30 cm apart, Katumani B gave the best yield.

74

B. At each planting distance, Hybrid 613 gave the poorest yield. C. With Hybrid 511 maize, the best yield was obtained when the seeds were spaced 45 cm apart. D. All three types of maize gave their poorest yields when the seeds were planted 15 cm apart.

Further exemplification of the intent of the exam is not necessary. The emphasis on intelligent, useful application and reasoning is evident. However, a word is necessary on the English essay, which is the one 'open-ended' exercise in the whole exam.

The Examination Council naturally wishes to promote the correct and fluent use of English. It also wishes to foster liveliness, originality and imagination w especially in order to combat the 'standardised' essays peddled by the crammers. Accordingly around 60 per cent of the English essay mark is distributed between grammar, handwriting, spelling and punctuation. The remaining 40 per cent is awarded for the more subjective qualities. The assessment of this section is based broadly on example and consensus. Experience indicates which schools are likely to produce candidates high on the qualities desired. The scripts from those schools are swiftly scrutinised. Promising specimens are extracted, copied, distributed to the teams of cxaminers (all teachers or inspectors) and analysed. A consensus judgment is made on the best scripts, which then set the tone for the marking scheme. In this way, the higher goals of education are concretely operationalised and rewarded substantially.

Communication with Teachers

Implicit in the last paragraph is the involvement of teachers in defining what it means to satisfy the higher goals of expression in English. It reflects the faithfulness of the Examinations Council to the wisdom of encouraging and making space for the school teachers themselves to influence the develop- ment of the examinations n but not wholly to dominate it! Using teachers as examiners, inviting teachers to submit questions for the exams, running seminars and workshops for them are all part of the repertory for com- municating, diffusing, monitoring reactions to

a n d adjusting the changes. One unusual step has been to publish a crammer for the science section! Perhaps the most interesting and universal n not to say imitable w measure has been the introduction of C. P. E. News- letter: a copy of it goes to every primary school.

Each school of course knows how it has done in the last C.P.E. exam. It also knows its position relative to other schools, for the District Education Officers publish lists of order-of-merit giving the mean standard score of each school on each paper. But the teachers do not know precisely why their pupils have done well or badly. The Newsletter sets out to explain this. Not only does it give all the correct answers, it also explains the reasoning behind a great many of them, shows why the wrong answers were wrong and suggets why pupils might have selected the wrong answers. In effect, it is a training resource. It enables teachers not simply to understand the exam, but also to improve their own ability in responding to it.

It goes further and explicitly advises teachers on what teaching and even what kind of mock exams will produce better results. A kind of Judo trick is being used: exam fever is exploited to promote better education. Of particular interest is the note in the 1980 Newsletter on the comparative performances of boys and girls in the science paper. The overall gap in the rural schools was 7.7 per cent, with the boys scoring a mean of 49.9 per cent and the girls 42.2 per cent. The Newsletter points out first that this gap was wider than those in the other subjects. But it was only about half as wide on the questions which tested reasoning, whereas on observation and experi- ment, it was nearly twice as wide. Therefore,

"Make sure that when you are doing practical work in science, the girls take just as active a part in the observations and experiments as the boys. If you find that girls tend to be passive when working with boys, you may find it useful from time to time to arrange your groups so that boys and girls work separately" p. 39-40

While this technique of analysis is suitable

75

for the multiple-choice questions, it is not wholly appropriate for the English essay. A full demonstration seems the best answer, so seven essays, the three best and the four worst, are this year being published to all the schools, with notes to explain the examiner's assessment. The emphasis is on common errors so that the standard of English is improved and, more enticingly, on the qualities which attract extra marks m so that ambitious teachers and pupils go in quest of them.

The Newsletter and published essay are necessary for at least four reasons. First, the new philosophy for the exam required changes in teaching. But, second, with a large teaching force, formal in-service training would not suffice to cover all the teachers involved or to keep them abreast of annual advances. Third, many teachers were unsettled by the changes

m "How can children of 12 years be expected to reason?", demanded one lady teacher, when she herself got the answer to a science ques- tion wrong. They needed to be reassured that the changes were well within their own and their pupils' capacities. Fourth, without such full guidance, a great many teachers would not know how to change their teaching, so that the lag between changes in the exam and changes in teaching would have been inordinate.

How Effective? Data to judge in detail the effectiveness of

the changes are not to hand. All that is possible here is to set the average subject marks of the 1978 CPE exam against those of the 1979 exam, as a crude initial index of how teachers and pupils are coping.

Average Marks m ~-

1978 CPE 1979 CPE Difference

Mathematics 51.3 43.7 - 7.6

English (multiple-choice) 51.6 42.1 - 9.5

History 40.0 37.4 - 2.6

Geography 40.8 50.5 + 9.7

Science 39.5 46.7 + 7.2

The picture seems inconsistent: two large drops, two large gains and one small drop. Clearly, much more information is needed for a useful assessment.

However, casual empiricism suggests that teachers and inspectors are being influenced in the desired direction - - despite protests from some quarters. The papers for the mock exams which schools and even districts organise, reflect efforts to devise questions along the lines pursued by the Examinations Council.

While we wait for more patient analyses of the short and long term effects of the changes, we can at least take heart from Kenya's experience of the following facts:

- - exams, even in the multiple-choice format, can indeed be made to subserve the

higher goals of education, and to transcend factual recall and routine application;

exam problems, both simple and complex, can be easily related to concrete situations in real life; hence, skills and knowledge useful to school leavers as well as to continuing students can be incorporated;

at least some teachers and inspectors do indeed respond to changes in examinations by changing the content and style of their teaching, revision and testing;

timely information and guidance to teachers will be used at least by some to improve instruction.

76

NECESSITE D'ADAPTER LES EXAMENS AU MODE DE VIE DES CANDIDATS: IMPRESSIONS DU KENYA

Aperfu

Les examens sont souvent consid6r6s comme des barri~res entravant la route qui m~ne aux objectifs sup6rieurs de l'enseignement, ainsi qu'aux programmes scolaires adapt6s ~ des 61~ves dont la grande majorit~ ne poss~de qu'une instruction rudimentaire, peu propre ~ leur garantir un emploi r 6 m u n 6 r 6 - une fois leur scolarit6 termin~e w dans les pays en voie de d6veloppement. Les techniques susceptibles d'examiner les candidats et d'assurer un enseignement efflcaee repr6senteraient parfois des d6penses telles qu'elles en seraient prohibitives.

Cependant, le Service des Examens au Kenya (h pr6sent d6nomm6 Commission des Examens) a progressivement modifi6 l'examen du Certificat d'Etudes Primaires, afin de l'adapter ~ des objectifs d~passant la seule connaissance de quelques faits retenus, ou des d6monstrations traditionnelles et abstraites de compEtences 616mentaires. Agents causatifs, observation, discernement, utilisation des donn6es, raisonnement, solution des probl~mes, prennent actuellement une importance nouvelle; et l'on met en relief telles connaissances susceptibles d'&re utiles aux jeunes dont la seolarit6 risque d'&re 6court6e.

Les enseignants sont tenus au eourant, bien entendu, des modifications qui les concernent. Un Bulletin vient eompl6ter les diverses possibilit6s dont ils disposent pour parfaire leur formation; il analyse questions et r6ponses, et propose des conseils quant aux m6thodes p&lagogiques les plus efficaces.

Subordinando los ex~imenes a la pertinencia: Impresiones de Kenia.

A menudo, los exfimenes se consideran como obstficulos para la consecuci6n de los altos objetivos de la educaci6n y para conseguir, tambi6n, que los programas sean pertinentes para la gran mayorla de los alumnos que s61o disfrutan de una educaci6n bfisica y que, en los paises en vias de desarroUo, no es probable que alcancen un empleo remunerado. A veces se piensa que el coste de las t6cnicas que podrian ensayar y fomentar la mejora de la educaci6n hace que la aplicaci6n de estas t6cnicas sea impracticable.

Sin embargo, la Unidad de Ex,4menes de Kenia (ahora con el nombre de Consejo de Ex~imenes) ha hecho que gradualmente el examen para la obten ci6n del Certificado de Educaci6n Primaria se subordine a objetivos que estfin lejos de la memorizaci6n de heehos y de la aplicaci6n abstracta y rutinaria de simples habilidades. Ahora se presta especial atenei6n a hechos "causales", a la observaci6n, al discernimiento, al manejo de irr[ormaci6n, al razonamiento y a la resoluci6n de casos problem~iticos. Y de la misma manera a conocimientos fitiles para los j6venes que no podr~in continuar su escolarizaci6n.

Se cuida igualmente de mantener informados a los profesores de los cambios introducidos. A la larga, y corriente, eolecci6n de canales de entrenamiento, se afiade una hoja informativa que analiza preguntas y respuestas y aconseja sobre el tratamiento educativo mils efectivo.

77

REFERENCES

Cooper, G., 1981, How Can You Evaluate In An Exam The Fact That A Boy Has Learnt To Control His Temper?, p. 13, The Guardian, September 8, 1981.

Dore, R., 1976, The Diploma Disease, London Allen & Unwin.

Kenya National Examinations Council, 1980, C.P.E. Newsletter, Nairobi.

Lewin, K., 1980, Science Education in Malaysia and Sri Lanka - - Curriculum Development and Curriculum Evaluation, 1970-1978, University of Sussex D.Phil. Thesis, unpubl.

Little, A., 1981a, Content Analyses of Some Public Examinations in The Paper Qualification Syndrome and The Unemployment of School Leavers, Addis Ababa, International Labour Organisation Jobs And Skills Programme For Africa (Vol I, Kenya, Somalia, Tanzania. Zambia, Vol II, Gambia, Ghana, Liberia, Sierra Leone).

Little, A., 1981b, The Role of Examinations in Promoting the "Syndrome", Report of a Seminar on The Paper Qualification Syndrome and the Unemployment of School-Leavers, Addis Ababa, ILO/JASPA.

Sinclair, M. & Lillis, K., 1980, School and Community in the Third World, London, Croom Helm and Institute of Development Studies.

78