The impact of portable computers on pupils' attitudes to study

Journal of Computer Assisted Learning (1993) 9,130-141

The impact of portaMe computers on pupils' attitudes to study H. Morrison, J. Gardner, C. Reilly and H. McNally School of Education, The Queen's University of Belfast

Abstract Over two hundred pupils in nine schools were provided with a personal portable computer for a whole school year. One aspect of the research was to assess the impact which high access to information technology (IT) had on the pupils' attitudes to their schools and to the core disciplines: mathematics, English and science. The attitudes of the experimental and control groups (those with and those without portables) to the disciplines and to their schools were measured using 25-item semantic differential and 28-item Likert instruments respectively. Factor analysis revealed that positive impacts of high access to IT upon the attitudes of pupils was relatively marginal and confined to instances where the pupils' process-based IT work transferred to the content domains in their disciplines. An interpretation of these affective findings, informed by recent National Curriculum research, is offered.

Keywords: Attitudes; English; Mathematics; Portable computers; School; Science.

Introduction

Sponsored by the Department of Education for Northern Ireland, the research set out to evaluate a number of issues relating to the r81e and impact of information technology, and in particular the up-and-coming generations of portable computers, in education. The project was broad ranging in terms of its context and research objectives and involved an investigation of the impact of the machines on the pupils' cognitive and affective development (for details of the former see Gardner et al., 1993a) along with issues related to the operationalization of their usage. This paper sets out the results and analysis relating to the investigation of the affective impacts.

The project involved nine schools: one special education school, one primary school, six non-selective secondary schools and one selective

Invited Paper

Correspondence: Dr. J. Gardner, School of Education, The Queen's University of Belfast, Belfast BT7 INN, Northern Ireland. 130

The impact of portable computers on pupils' attitudes 131

(grammar*) school. In each school one whole class was supplied with portables with, in most cases, at least two extra machines for the teachers. While the pupils and their various teachers were encouraged to use the machines throughout their cumculum, their participation was specifically monitored in a focus discipline: English, mathematics or science. The schools and teachers were made ready for the project during the period April 1 to June 30,1991 and the fieldwork was carried out between September 1991 and the end of June 1992. Details of the experimental and control classes (235 and 191 pupils respectively), the types of machines and software used are also available (Gardner ef al., 1993a, b). Each 'experimental' pupil owned a personal portable computer throughout their school year.

Methodology

This paper outlines the quantitative methods and results in relation to the impact of personal access to portable computers on pupils' attitudes towards English, mathematics and science, and towards school in general. The affective research hypothesis is summarized thus:

the experimental group will be significantly better disposed (at the 5% level) to the t h m disciplines and school, than the control group.

For the cognitive aspects of the research, the pupils had been matched for age, gender and ability in experimental/control class groups (with ability being measured using National Foundation for Educational Research general reasoning ability instruments). However, for the assessment of attitudes the experimental and control groups were combined to make two large groups. The two primary classes were excluded as it was considered that the structure of the primary curriculum would make it difficult for pupils to reliably distinguish betwccn the disciplines.

Affective measures A 25-item, five-point semantic differential questionnaire, which evaluates attitude to teaching and learning in each of the core disciplines, was constructed with bipolar statements which had been identified from a careful study of the pupils' project diaries. Examples from the instrument are:

I find (discipline) boring I find (discipline) interesting

I really become involved in Its always the same dull stuff (discipline) - the lessons over before I know it

every day in (discipline)

The same scale was used for each of the three disciplines with the words maths, science and English placed appropriately. In addition, a 28-item

Northern Ireland, in common with several other parts of the LJK, retains selectiveentry grammar schools. Pupils are awarded places according to their performance in selection tests administered at ll-years of age.

132 H. Morrison, et al.

Likert scale was used to measure pupil attitude to school life in general. Pupils had to reply on a five point scale to statements such as:

My teachers do their best for me My school sees me as an important person.

Results

The data from the two types of attitude measure: attitude to disciplines and attitudes to school, were factor analysed and the disposition of the experimental and control pupils to the various discipline and school factors were compared. The results of the factor analyses are presented below.

Attitude to disciplines Four factors were identified from the 'attitudes to disciplines' data and the details of these are laid out below. Note that for brevity only those items with a factor loading of > 0.5 are presented and those items which loaded on the factor in only one or two of the disciplines are not listed. The order in the listing of the factors is according to their loading on the particular factor in mathematics. No items failed to load in any of the scales.

Table 1. Attitude to disciplines: enjoyment

Item Semantic Differential Items Factor Loadings No. (mathematics, science, English)

~~

24 Every (discipline) class is the same/l feel I'm always learning something new in (discipline) 0.76 0.69 0.73

3 I find (discipline) boring/[ find (discipline) interesting 0.73 0.61 0.80 5 I enjoy (discipline)/ I detest (discipline) 0.72 0.62 0.83

23 (Discipline) doesn't excite me/I find (discipline) exciting 0.69 0.65 0.74 22 I really become involved in (discipline) - the lessons over

before I know it/Its always the same dull stuff every day in (discipline) 0.67 0.69 0.74

Explained variance (96) Reliability (a)

24.4 30.6 27.5 0.7404 0.8912 0.8913

The items listed in Table 1 appear to reflect the pupils' appreciation or xnse of enjoyment of the discipline. It will be denoted as the 'enjoyment' factor and a positive disposition is taken to indicate that the pupils enjoyed their studies in the discipline.

The factor detailed in Table 2 appears to reflect the pupils' perception of the relevance of their discipline to the real world. It will be denoted as the

* The responses of pupils to the semantic differential attitude questionnaires (for each of the core disciplines) were subjected to principal components extraction with orthogonal (varimax) rotation. Factors were extracted using Cattell's scree test (Cattell, 1966) with minimum acceptable loadings o f f 0.3 used in interpreting factors. Analysis of variance (ANOVA) was performed on the regressed factor scores arising from the factor analysis of the questionnaire data. The same process was carried out in analysing the attitudes to school questionnaire.


'relevance' factor and a positive disposition is taken to indicate that the pupils' view their studies as being relevant to the real world.

Table 2. Attitude to disciplines: relevance ~~~~ ~

Item Semantic Differential Items No.

Factor Loadings (mathematics, science, English)

19

9

11

14

It isn't clear how (discipline) helps you in real life

The skills I learn in (discipline) are useless/The skills I learn in (discipline) are valuable 0.67 0.67 0.40 (Discipline) can only be used in a small number of areas of life/(Discipline) can be used everywhere in life 0.61 0.78 0.66 The things I learn in (discipline) are relevant/The things I learn in (discipline) are irrelevant 0.57 0.51 0.61

/(Discipline) is a very practical subject 0.72 0.54 0.68

Explained variance (%) Reliability (a)

6.9 10.5 10.6 0.7105 0.8682 0.7561

Table 3. Attitude to disciplines: performance

Item Scmantic Differential Items Factor Loadings No. (mathematics, science, English)

2 I am bad at (discipline)/I am good at (discipline) 0.76 0.72 0.59

8 (Discipline) confuses me/(Discipline) makes

7

4 I find (discipline) easy/[ find (discipline) difficult 0.72 0.64 0.66

things clear to me 0.68 0.53 0.39 I am satisfied with my marks in (discipline)/I am disappointed with my marks in (discipline) 0.67 0.74 0.66


9.5 6.4 6.0 0.5280 0.8417 0.6825

The factor detailed in Table 3 appears to reflect the pupils' perceptions of their own performance or achievement in the discipline. It will be denoted as the 'performance' factor and a positive disposition is taken to indicate that the pupil is satisfied with his or her performance in the discipline. Table 4. Attitude to disciplines: responsible approach

Item Semantic Differential Items No.

Factor Loadings (mathematics, science, English)

18

25

If 1 am to be good at (discipline) I have to do a lot of work/] can be good at (discipline) with very little work 0.66 0.73 0.68 You need to be an organized worker to be good at (discipline)/You can be good at (discipline) and just work from time to time 0.62 0.64 0.52 I have to work fast to get things done in (discipline) lessons/I can work slowly and still keep up in (discipline) lessons 0.54 0.60 0.63

20


5.7 4.7 5.1 0.4785 0.6743 0.4424

134 H. Morrison, et a!.

It is not easy to interpret the factor detailed in Table 4 but it is reasonable to concludc that a pupil whose factor score indicates a positive disposition to it sees the conventional dutiful, conscientious and hardworking approach to the discipline to be the proper and most rewarding approach. This factor will be denoted as the 'responsible approach' factor and a positive disposition is taken to indicate that the pupil accepts the importance of a responsible approach to his or her work in the discipline.

Attitudes to schml Attitudes to various aspects of school were also assessed. Four factors were identified and the items relating to them are set out below. All of the items with loadings of 0.40 or more are presented for each factor. No items failed to load.

Table 5. Attitudes to school: work in class

Item No. Likert Item Factor Loadings

7

24

9 25 15

22 2

18 19

I think if you don't do your homework you're letting yourself down in front of the whole class 0.65 I like to sit in a quiet classroom listening to the teacher explaining the subject 0.61

0.60 My teachers value what I say in class 0.54 If I'm behind with coursework, my teachers understand that I'll do my best to catch up 0.51 I find that time flies in class 0.46 My teachers do their best for me 0.41

My teachers encourage me to think for myself 0.40

Sometimes I don't want to stop work when the bell rings

I cannot do my best because my classes are too noisy -0.41

Factor 1 explained 26.2% of the variance with reliability 0.7903. It is difficult to interpret but it does appear to reflect the pupils' perceptions of their work in the classroom. It will be denoted as 'work in class' and a positive disposition is taken to indicate that the pupils considered their work in class to be important.

Table 6. Attitudes to school: school


4 17 26 16 10

I enpy going to school I do not like school I like most of the teachers in my school I am proud of my school I get on well with my teachers

0.76 0.79

0.57 0.46

0.59

Factor 2 explained 6.6% of the variance with reliability 0.7583. This factor appears to reflect the pupils' perception of the school as an institution and will be denoted as the 'school' factor. A positive disposition to the factor is taken to indicate that the pupils had a positive perception of their school.


Table 7. Attitudes to school: security

Item No. Likert Item

23 1

12 My school sees me as an important person 0.54 27

14

Factor Loadings

In my classes, pupils behave well most of the time In my classes I feel we are like one big happy family

If I was in trouble I would turn to my teachers for advice

When I get something right the teacher doesn't say well done!

0.67 0.62

0.52

0.42 25 My teachers value what I say in class 0.44

Factor 3 explained 5.5% of the variance with reliability 0.7866. This factor reflects an emphasis on the security and ambience of the school and classroom. It will be denoted as the 'security' factor and a positive disposition is taken to indicate that the pupils felt at home and secure in class.

Table 8. Attitudes to school: teachers


21 I think most teachers are lazy 0.64 3

11 8 5

13 18

My teachers don't really understand the subjects they teach In class I don't get the chance to say what I think very often I get confused by the things teachers ask me to do

Doing work in class is a waste of time I cannot do my best because my classes are too noisy

0.61 0.54 0.52 0.50 0.47 0.47

"lie teachers in my school do not like the pupils

Factor 4 explained 5.3% of the variance with reliability 0.7330. This factor appears to reflect the pupils' perception of their teachers and will be denoted as the 'teachers' factor. A positive disposition is taken to indicate that the pupils had a positive perception of their teachers.

ANOVA results

The dispositions to the various discipline and school factors were compared for the whole experimental group vs the whole control group and comparisons were also made between the experimental boys, experimental girls, control boys and control girls. The results of most interest to the research were the comparisons of the whole groups, experimental boys vs control boys, and experimental girls vs control girls.

It should be noted that the comparisons have been made using regressed factor scores with mean = zero and standard deviation = unity. This explor- atory factor analytic approach ensures the highest correlation between factors and factor scores. The results are presented in Tables 9 - 14 below in the form of the percentage of pupils having a positive disposition to each factor, i.e. those with negative factor scores, since strong disposition was coded 1 and strong disagreement was coded 5. The significance of any difference between the various groups of pupils is also provided in conventional probability form, quoting the p value. INS' is used to indicate that any


difference was not significant. Pairs which were significantly different are given in bold type. In reading and comparing the values in the tables the reader should not be tempted to consider large differences as being necessarily significant. The figures reflect the percentage of the group who were positively disposed to the factor in question while the significance of comparisons between groups is based upon analysis of variance of regressed factor scores.

Enjoyment of discipline Table 9 shows that overall a greater proportion of the control pupils enjoyed mathematics and science than was the case for the experimental groups; while in English the opposite effect is observed. The main contribution to the effect in mathematics comes from the boys' groupings where the experimental group appears to have enjoyed mathematics significantly less than the control group (the girls' groupings actually indicate a greater proportion of experimental girls enjoying mathema tics than their control counterparts).

Table 9. Percentages of pupils who enpy their discipline

All pupils Subject Ctrl Expl

9 0 %

--Boys - c2 EF'

-Girls - Ctrl Expl

% %

Mathematics 59.5 55.6 NS 73.5 58.0 pd.05 45.9 53.4 NS Science 58.6 50.6 NS 60.8 55.4 NS 56.7 46.1 NS English 52.5 58.9 p=0.0500 50.7 63.5 NS 54.0 54.4 NS

Relevance of discipline Table 10 demonstrates that, in contrast to the control groups, a greater proportion of the experimental pupils in all of the groupings felt that their subject was relevant.

Table 10. Percentages of pupils perceiving the discipline to be relevant

All pupils Subject Ctrl Expl

70 7%

--Bovs - Ctrl Exbl

% %

--Girls - Ctrl Expl

% %

Mathematics 53.6 65.1 p=O.OlOO 56.6 64.2 NS 50.6 65.9 NS Science 55.6 58.1 NS 62.0 62.7 NS 50.0 53.9 NS English 51.2 61.1 NS 49.3 63.0 NS 52.9 62.2 NS

Perception of peqormance in each discipline Table 11 shows that a greater proportion of pupils in the experimental groups expressed satisfaction with their performance in science (line 2) than those in control groups but the effect was reversed for English and mathematics. There appears to be a curious gender difference here as more of the experimental boys in both these latter cases (and significantly at p<0.05 in the case of mathematics) were satisfied with their performance than was the case for the experimental girls.


Table 11. Percentages of pupils perceiving themselves to be performing well at their discipline

All pupils --Boys - -Girls - Subject Ctrl Expl Ctrl Expl Ctrl Expl

Yo % 70 % % 70

Mathematics 54.2 50.9 NS 55.4 63.0 NS 52.9 39.8 NS Science 45.6 58.1 p=0.0390 54.4 65.1 NS 37.8 51.7 NS English 51.9 47.4 NS 38.7 51.8 NS 63.2 43.3 NS

Responsible approach to the discipline Table 12 reveals that a greater proportion of the pupils in the experimental groups, in all of the groupings, were aware of the importance of adopting a responsible attitude to their learning than was the case with control pupils. It would seem reasonable to expect a Hawthorne effect whereby the pupils with the portables might be more 'responsible' as a result of having been given a special project with personal responsibility for expensive equipment.

Table 12. Percentages of pupils with a responsible approach to their discipline

All pupils -Boys - -Girls - Subject Ctrl Expl Ctrl Expl Ctrl Expl

% To % % % %

Mathematics 49.4 55.0 NS 49.4 56.8 NS 49.4 53.4 NS Science 40.8 63.4 p=O.O009 39.2 61.4 NS 42.2 65.2 p4.05 English 47.5 53.1 NS 49.3 54.1 NS 46.0 52.2 NS

Attitudes to schml Table 13 reveals that the experimental pupils perceived their classwork as important to a significantly greater extent than did the control pupils. Furthermore, girls with access to portable computers are significantly better disposed to their school than boys with similar access.

Table 13. factors

Percentages of pupils with positive disposition to attitudes to school

AU pupils -Boys - -Girls - Subject Ctrl Expl Ctrl Expl Ctrl Expl

% % % % % 70

Work in dass 48.1 55.2 p=0.0170 56.0 65.4 p4.05 41.1 45.2. NS School 53.1 56.4 NS 49.3 49.4 NS 56.3 63.1 NS Security 52.5 50.9 NS 53.3 53.1 NS 51.7 48.8 NS Teachers 50.0 58.8 NS 40.0 58.0 NS 58.6 59.5 NS

this value for the experimental girls is significantly different from that of the experimental boys: ~ 4 . 0 5 )


Suinrna y In comparison to control pupils, a greater proportion of experimental pupils:

expressed the view that they enjoyed English; expressed the view that they did not enjoy mathematics and science (experimental girls showed the opposite effect in mathematics); in all of the disciplines, considered their discipline to be relevant; in mathematics and English expressed dissatisfaction with their performance (experimental boys showed the opposite effect); in science expressed satisfaction with their performance; considered that having a responsible approach to work was important; considered their work in class to be important; had a positive perception of their schools; felt less at home or secure in class; had a positive perception of their teachers.

An interpretation of the research findings

The control and experimental groups were matched for age, ability, gender mix and were being exposed to the same cumculum. Clearly they had differcnt teachers, and teaching factors have not been controlled, but one major and indisputable difference between the groups was the full-time personal access to portable computers which the experimental groups had for one school year. The question is: to what extent did this access to the computcrs have a bearing on the pupils' attitudes to their work and school?

Some of these results can be more readily explained than others. For example, the pupils with portable machines had been specially chosen by thc schools and one might reasonably expect them to be more positive about their school and about their teachers. Similarly this may have made them more aware of the importance of a responsible approach to their work and that their work in class was important. It might also be argued that the very use of computers in their lessons would have contributed to their perception of their disciplines as being relevant, since their lessons were using the latest in computing technology ie. they were more up-to-date than they would have been otherwise. Explanations are not readily available when one considers the results relating to enjoyment, performance and security. It would appear that more fundamental systematic effects are in action here, including, perhaps, the nature of the IT activities and gender effects. It is beyond the scope of this paper to develop the gender aspects but an interpretation of the effects relating to the nature of the IT usage is offered.

In contrast to the findings of Mevarech & Rich (19851, there was little significant affective improvement overall (in the statistical sense) when these pupils were given access to information technology. However, what distinguishes the current study from the drill and practice or content focus in the Mevarech and Rich investigation, is that the vast majority of activities in which pupils in the portable classes were engaged, were of a process type. In


addressing the IT-related statements of attainmeni, pupils used their portable computers to enhance their mathematical problem-solving skills through LOGO, to conduct investigations in science and to edit and redraft in English. It is instructive to examine recent evidence, gathered from National Curriculum classrooms, that teachers experience considerable insecurity when teaching and assessing process skills in mathematics and science.

In their December 1992 evaluation of National Curriculum assessment, the School Examinations and Assessment Council (SEAC 1992) observed, in the case of the science process attainment target, that:

some science teachers were still struggling with the concept of an investigation and appropriate mcthods of assessment; many process activities may be occurring as set-piece investigations rather than an activity perceived by pupils as normal classroom practice; activities leading to teacher assessment of process did not have learning as a high priority.

The SEAC evaluation identifies similar concerns in mathematics classrooms although experience with GCSE investigations did result in more appropriate activities. The findings point to a clear distinction being made in schools between the assessment of process skills and knowledge or coliteizt skills in mathematics and science. The report found little or no evidence that mathema tics or science teachers actually assessed content while children were in the act of carrying out (process based) investigations. It would appear that while science and mathematics teachers have little difficulty in designing pencil and paper tests to assess knowledge, they are considerably less at home with the assessment of process. English teachers, on the other hand, are happy with process assessment criteria against which measures such as quality of writing, organization and fluency may be scored. Essays are appraised holistically and the atomisation of the curriculum through statements of attainment such as those employed in mathematics and science is viewed as inappropriate in a creative subject such as English.

The teachers of experimental classes were charged with investigating whether portable computers were capable of addressing the IT-related Statements of attainment in their subject and the vast majority of these have a process focus. The evidence from the SEAC study indicates that process work in the classroom may not be perceived as normal class work or may not have learning as its primary emphasis. In the current study it could be argued that teacher insecurity with process activities (such as conjecturing, generalising, synthesizing and so on) may have been transmitted to the experi-mental pupils in science and mathematics lessons, causing them to feel insecure in class and to enjoy their work less. For all three groupings

* For non-UK readers, the statutory curricula of England, Wales and Northern Ireland (but not Scotland) require that pupils in compulsory education (approx. 5-16 years) follow subject courses which are defined by programmes of study. Their achievement in each subject is assessed through an elaborate pseudecriterion referenced system of attainment targets (e.g. the attainment target Number in mathematics covers number skills) which are subdivided into spedic statements of attainment which define the performance expected of pupils at ten separate levels.


(boys, girls and all pupils) the proportion of experimental pupils who enjoyed their discipline exceeded the proportion of control pupils only in the case of English (where process is a much more integrated concept in curriculum terms). In science the position was reversed for all three groupings, and the boys in mathematics classes with access to portable computers enjoyed mathematics significantly less than the boys in control classrooms.

Only in the case of science did greater proportions of experimental than control pupils feel they were doing well while in mathematics and English they were less convinced. These perceptions of their performance were accurate since they can be demonstrated to be in keeping with their achievement scores (Gardner, ef d., 1993a) which show that only the objective scores (spellings and so on) in English and the science scores of the experimental pupils, were consistently higher than their counterparts in control classes. While it is clear the tests did not measure process skills learning, it can be argued that the process work did have impact on the content work in science (and marginal impact in English), as assessed in the tests. (To some extent the effect in science is to be expected as the science testing included graphical interpretation which had formed part of the science work using the portables with sensor equipment.) There is the strong indication here that the use of IT in process context is not translating to content learning, except - as in the case of science - where the curriculum defines IT requirements in content- based statements of attainment.

IT in its cross-curricular framework (the cummunicatim, information handling, modelling and so on strands of the UKIT curriculum) is clearly a process-based concept and IT-related activities in mathematics, science and English, at present, tend to have a process based emphasis - with much less classroom activity, and therefore less tangible impact, in content learning contexts. If the curriculum continues to have the effect of causing process learning to be separated from content learning in the classroom, and if teachers are not comfortable with the separation as the SEAC work appears to indicate, then IT may also be suffering from the same sense of insecurity and uncertainty. In order to make the r6le of IT more effective the curriculum needs to ensure that IT process-based work is squarely integrated into content activity in the various curricula.

References

Cattell, R.B. (1966) The scree test for the number of factors. Journal of Multivariate Behavioral Research, 1,245-276.

Gardner, J., Morrison, H. & Jarman, R. (1993a) The impact of high access to computers on learning. Journal of Computer Assisted Learning, 9,2,2-16.

Gardner, J., Morrison, H., Jarman, R., Reilly, C. & McNally, H. (1993b) Pupils' Learning and Access to lnformation Technology: The Project. National Council for Educational Technology, London.

Mevarech, Z.R. & Rich, Y. (1985) Effects of computer-assisted mathematics instruction on disadvantaged pupils' cognitive and affective development. journal of Educational Research, 79,1,5-11.

SEAC (1992) National curriculum assessment at Key Stage 3: 1992 evaluation. Teacher assessment in mathematics and science at Key Stage 3 . SEAC, London, UK.

Documents

The impact of portable computers on pupils' attitudes to study