HOW EFFECTIVE IS ICDL TRAINING FOR OMANI TEACHERS?

A dissertation submitted to the University of Manchester for the degree of Master of Arts in " Digital Technologies,

Communication and Education" in the Faculty of Humanities

YEAR OF SUBMISSION: 2009

FAHAD KHALIFA HUMAID AL HATMI

SCHOOL OF EDUCATION

UNIVERSITY OF MANCHESTER

List of ContentsList of Tables and Figures. 4 Abstract 5 Declaration 6 Copyright Statement 6 Acknowledgment 7 Chapter 1: Introduction 8 1.1. Background of the Topic 8

1.2. Statement of the Problem 10 1.3. Purpose of the Study 11

1.4. Significance 121.5. Research Questions 12 1.6. Theoretical Model 13 1.7. Scope and Limitations of the Study 15 1.8. Outline of the Study 16

Chapter 2: Literature Review 17 2.1. Introduction 172.2. ICT 17

2.2.1. Defining ICT Literacy 17 2.2.2. ICT in Teaching 19 2.2.3. Teacher Readiness for ICT Instruction 20 2.2.4. Approaches to ICT Education 21 2.2.5. Effectiveness of ICT Training 24

2.2.6. ICT Training Methods 25 2.2.7. Assessing ICT Literacy 26

2.3. ECDL/ICDL 272.3.1. Background 272.3.2. Standardization and Vendor-neutral Approach 272.3.3. Teaching Methods for ECDL/ICDL 282.3.4. Effectiveness of ECDL/ICDL 29

2.3.4.1. Effectiveness in Assessing ICT Literacy 292.3.4.2. Effectiveness for ICT Training 31

2.3.5. Gaps in Existing ECDL/ICDL Literature 332.4. ICT and ECDL/ICDL in Sultanate of Oman 33

2.4.1. ICT in the Sultanate of Oman 332.4.2. ECDL/ICDL in the Sultanate of Oman 34

2.5 Conclusion 35Chapter 3: Methodology 36

3.1. Introduction 363.2. Justification of the Methodology 363.3. Merging Quantitative and Qualitative Data 393.4. Quantitative Phase of the Study 39

3.4.1. Survey Questionnaire Methodology 403.4.2. Quantitative Sampling 403.4.3. Quantitative Instruments 41

2

3.4.4. Data Collection Procedures 423.4.5. Data Analysis 423.4.6. Reliability and Validity 44

3.5. Qualitative Phase of the Study 443.5.1. Sampling 453.5.2. Instruments 463.5.3. Data Collection Procedures 473.5.4. Data Analysis 483.5.5. Trustworthiness 49

3.6. Limitations of the Methodology 503.7. Summary 51

Chapter 4: Findings 524.1. Quantitative Findings 52

4.1.1. Survey Questionnaire 524.1.2Pre- and Post- Test of Access Class under Observation 59

4.2. Qualitative Findings 604.2.1. Descriptive Statistics: Motivation, Expectations, 60

and Skills Gained4.2.2. Descriptive Statistics: Deficiencies in the Training 624.2.3. Descriptive Statistics: Suitability of Training for 62 Teachers, And Recommended Improvements4.2.4. Observation of Access Class 634.2.5. Interviews with Trainers 674.2.6. Interviews with Students of Access Class 68

Chapter 5: Discussion 71Chapter 6: Conclusion 78Chapter 7: Recommendations 81

7.1. Recommendations for Improvement 817.1.1. Make Training Relevant to Educators 817.1.2. Training Length 817.1.3. Training Time 827.1.4. Training Materials 827.1.5. Trainer’s Training 827.1.6. Phased Approach to ICT Training 82

7.2. Recommendations for Future Research 837.2.1. ICT Competency Differences for Teachers in 83 Different Subject Areas7.2.2. The Effect of Lack of and Poorly Maintained 83 Equipment

Chapter 8: References 85Appendix A: Survey Questionnaire 92Appendix B: Structured Observation Form 97Appendix C: Teacher Interviews 100Appendix D: Trainer Interviews 101Appendix E. Participant Affirmation Letter 102

3

List of Tables

Table 1. Research Overview 38Table 2. Quantitative and Qualitative Processes. 38Table 3. Mean and Standard Deviation, Part 2 of the Survey 53 QuestionnaireTable 4 Means and F statistic for Gender 56Table 5 Statistically Significant Differences in Responses in Subject 57 Taught Data Grouping Table 6 Means of the Subject Taught Groups with Statistically 58 Significant DifferencesTable 7 Skills Expected to Improve form ICDL Training as Ranked1 60 by RespondentsTable 8 Skill gained in ICDL Training most Likely to Transfer 61 to the Classroom SettingsTable 9. Pre-Test Post-Test Scores 64

Word Count: 19, 535

4

Abstract

The increasing importance of information and communication technology

(ICT) in business and research challenges the education system to produce graduates

with strong ICT skills (Randall & Zirkle, 2005). Instruction in ICT is particularly

significant for developing nations to ensure that the workforce can compete in the

global environment. One approach to ensure that teachers have the necessary

fundamental skills to support ICT instruction in primary and secondary schools is to

require International Computer Driving License (ICDL) training and certification.

This approach, however, does not appear to provide the full range of ICT training

required to help teachers understand how to use ICT in the classroom to improve

learning outcomes. This concurrent mixed methods study examines the effectiveness

of ICDL training among Omani teachers. Results suggest that while ICDL is effective

at teaching basic skills in computer and application use, the training is not effective in

training educators on critical assessment of technology or in how to use technology in

the classroom. Based on the results of survey questionnaires, interviews, and

structured and unstructured observations, recommendations are made for how ICDL

training might be improved and for how ICT training in general can better meet the

needs of the Omani teachers.

5

DECLARATION I declare that no portion of the work referred to in the dissertation has been submitted in support of an application for another degree or qualification of this or any other university or other institute of learning.

Signed ……………………………………………………………………………………… (Candidate)

Date ………………………………………………………………………………………

COPYRIGHT STATEMENT

1. Copyright in text of this dissertation rests with the author. Copies (by any process) either in full, or of extracts, may be made only in accordance with instructions given by the author. Details may be obtained from the appropriate Graduate Office. This page must form part of any such copies made. Further copies (by any process) of copies made in accordance with such instructions may not be made without the permission (in writing) of the author.

2. The ownership of any intellectual property rights which may be described in this dissertation is vested in the University of Manchester, subject to any prior agreement to the contrary, and may not be made available for use by third parties without the written permission of the University, which will prescribe the terms and conditions of any such agreement.

3. Further information on the conditions under which disclosures and exploitation may take place is available from the Head of the School of Education.

Signed ……………………………………………………………………………………… (Candidate)

Date ………………………………………………………………………………………

6

Acknowledgment

I offer my most sincere gratitude to all the individuals who supported me in

bringing this study to a successful conclusion. First, I wish to acknowledge the

immense help of Dr. Andrew Whitworth whose supervision was always there when

needed. Second, the success of this study would not have been possible without my

parents’ prayers which gave me spiritual support. Finally, my thanks are extended to

those who helped me in collecting my data.

7

Chapter 1: Introduction

1.1. Background of the Topic

The increasing importance of information and communication technology

(ICT) in business and research challenges the education system to produce graduates

with strong ICT skills (Randall & Zirkle, 2005). Instruction in ICT is particularly

significant for developing nations to ensure that the workforce can compete in the

global environment. The education system in many nations, however, has been slow

to incorporate ICT training into the classroom curriculum (Mioduser, et al., 2003).

Several factors account for insufficient ICT educational opportunities. Many

classrooms are without sufficient physical resources to support ICT instruction.

Teachers may not have sufficient knowledge of ICT if they have not received training

(Gordon, 2007). Teachers may have sufficient ICT training and skills, but may not

have the knowledge necessary to incorporate ICT into classroom teaching at the

primary and secondary level (Law & Sim, 2008). Teachers must also have sufficient

confidence in their ICT skills to use them in the classroom environment, which may

require repeated training or sufficient time to practise skills outside the classroom

(Gordon, 2007). When teachers receive in-service training in a subjects related to

ICT, they assume the role of the student, and their attitudes as learners influence the

effectiveness of the training. (Preston, Danby, & Wegrif, 2005).

One approach to ensure that teachers have the necessary fundamental skills to

support ICT instruction in primary and secondary schools is to require International

Computer Driving License (ICDL) training and certification. The ICDL, which is

known as the European Computer Driving License (ECDL) within the European

8

Union, is a certification attesting to basic proficiency in the use of certain types of

software or computer systems (Csapo, 2002). The ICDL/ECDL is the world’s largest

computer certification programme, with more than 9 million candidates for

certification in 148 countries (EDDL Foundation, 2009). The ICDL/ECDL

Foundations responsible for oversight of the programme recommend that individuals

take specific training prior to certification testing. Developed by private organizations

and delivered through courseware that is often combined with tutorial services, the

training consists of seven modules. These modules include: 1) basic concepts of

information technology; 2) using the computer and managing files; 3) word

processing; 4) spreadsheets; 5) database; 6) presentation; and 7) information and

communication. While the courseware conforms to the ICDL/ECDL specifications

and is generally considered standardised, there remains some variability in the content

and in the manner of presentation of the material. In some learning environments, the

courseware is presented and a tutor is available, while other learning environments

provide students only with the courseware (Calzarossa, et al., 2005). In addition, the

ICDL/ECDL instructional modules are intended to be vendor neutral, offering general

information about ICT that can be used with different types of software (Randall &

Zirkle, 2005).

In the Sultanate of Oman, the Ministry of Education urges all teachers to

receive ICDL training and to apply for certification regardless of the subject they

teach (Ministry of Education, n.d.). The Ministry of Education recognises the need to

ensure teachers have fundamental ICT skills in order to facilitate integrating ICT into

classroom teaching. The ministry of Education also considers the ICDL training an

9

effective in-service training approach that allows teachers to obtain fundamental ICT

skills. The ICDL programme in Oman is considered a form of in-service training for

teachers. The ICDL programme in Oman is administered by the ICDL GCC

Foundation, which has been appointed by the ICDL Foundation as the sole operator

of the ICDL in the Gulf Cooperation Council nations, which include the Sultanate of

Oman. The ICDL GCC Foundation operates four training and testing centres in the

Sultanate.

1.2. Statement of the Problem

The specific problem investigated in this study is that Omani teachers do not

effectively use ICT skills in the classroom to enhance and improve student learning.

A prerequisite for using ICT in classroom teaching is sufficient proficiency with the

technical skills related to the operation of computers and software applications

(Rottman, 2004). In addition, teachers must have sufficient confidence in their ICT

skills and an understanding of appropriate methods to integrate ICT skills into

classroom teaching (Jung, 2005). Previous research has not fully determined that the

ICDL programme is effective for teaching the basic skills necessary for operating the

specific computer systems and software applications necessary to obtain certification

(Perez & Murray, 2007). The outcome of the ICDL instruction programme may

depend on factors such as design of the training modules, the specific methods used

to provide instruction, and the previous ICT knowledge of the individual (Dillon &

Tearle, 1999). Previous research is also inconclusive in findings concerning the

effectiveness of the ICDL programme for teaching the higher order or critical

thinking skills necessary to use ICT in applied academic or business situations

10

(Whitworth, 2005). This previous research suggests that the knowledge of

fundamental ICT skills differs from the knowledge necessary to apply the skills in

professional applications. The specific ICT needs of the education profession,

requires both fundamental ICT operating skills and the ability to integrate the skills

into practical classroom applications. .Previous research has not examined generic

ICDL training courseware effectiveness and effectiveness of teaching programmes

for meeting these needs. Because of the inconclusive findings of previous researchers,

uncertainty exists about the effectiveness of ICDL training for teaching fundamental

ICT skills to teachers necessary for supporting their ability to use ICT in classroom

teaching.

1.3. Purpose of the Study

The purpose of this concurrent mixed methods study was to examine the

effectiveness of ICDL training among Omani teachers. The quantitative phase of the

study examined the relationship between the independent variable of ICDL training

and the dependent variables related to improved ICT skills and the attitude toward the

use of ICT for classroom teaching, Data was collected from a population of Omani

teachers that have completed the ICDL in-service training. The qualitative phase of

the study examined the effectiveness of ICDL training by collecting data using a pre-

and-post test, interviews, and observations. The population for the qualitative phase

was Omani teachers and trainers in a single classroom teaching the ICDL database

module using Access. For the quantitative phase of the study, statistical data is

provided from a larger sampling of teachers. The quantitative data was combined

11

with the qualitative data to assess the effectiveness of ICDL training among Omani

teachers.

1.4. Significance of the Study

The findings of the study may have significance for teachers taking the ICDL

course as in-service training programme by identifying the effectiveness of the

training for meeting individual ICT objectives. The findings of the study may also

have significance for the education profession by identifying the extent that ICDL

training produces the desired outcomes of improvement in basic ICT skills and the

critical thinking skills necessary to use ICT in practical educational situations. As a

result, the findings may also have direct significance for the Ministry of Education in

Oman by assessing the effectiveness of the ICDL programme in the Sultanate. The

findings may also have importance for the organisations involved with developing

courseware and providing training by identifying the effectiveness of current ICDL

training for meeting the needs of the education profession.

1.5. Research Questions

There is a primary research question and four sub-questions that the research

attempts to answer.

Primary Question: Does ICDL provide the best approach for Oman with respect to

training its teachers to use technology effectively? The research question examines

whether the ICDL training is effective for providing teachers with the ICT skills

necessary in the academic and educational environment (Young, 2004). For the

purposes of the study, effectiveness is defined as a positive change in ICT proficiency

and skills in the specific subject matter areas addressed by the ICDL modules as well

12

as a positive change in the critical thinking skills necessary to apply the knowledge in

a professional teaching environment.

The sub-questions were derived from the main research question. They

investigate aspects of the ICDL training programme necessary to answer the main

research question of the study. The sub-questions are as follows:

1. Which ICT competencies were improved and which were not improved

through the ICDL training?

2. Does ICDL training influence the attitudes of teachers towards incorporating

ICT into classroom teaching methods?

3. Are there differences in ICT competencies gained through ICDL training that

are based on the mediating factors of gender, number of years teaching, and

class being taught?

4. Are there other approaches that could fill the gap in ICDL training to provide

the full range of ICT competencies required for Omani teachers to be more

effective in their classrooms?

1.6. Theoretical Model

The theoretical framework for the research is based on the Kirkpatrick

evaluation model developed to assess ICT training among teachers as described by

Wu, et al. (2004). This model defines four levels of outcome evaluation: 1)

evaluation-reaction, 2) evaluation-learning, 3) evaluation-behaviour, and 4)

evaluation-results. The first level examines participant reaction to the training using a

closed-end questionnaire. The second level measures what content was learned from

the training and pre and post testing is one method of taking this measurement. The

13

goal of level three measurement is to determine if the training results in behaviour

changes and looks at whether the knowledge gained is transferred to the workplace.

Kirkpatrick suggests interviews as an effective means of making this measurement.

Level four examines the bottom line results of the training (e.g., how did the training

affect ROI, learning outcomes, or any other bottom-line desired result). Level 4 is

difficult to measure because there are many other factors influencing the bottom line

result.

From the perspective of Kirkpatrick’s model as described by Wu, et al.

(2004), ICT training is effective if it provides both the basic skills necessary to use

ICT systems and the critical thinking skills necessary to apply the knowledge to

practical or academic situations. An underlying assumption in the evaluation model is

that ICT training not only should include technical skills necessary to operate

software and hardware, but also knowledge about the appropriate uses of the ICT

system to obtain and manipulate information or to apply ICT in professional

situations. The evaluation model is concerned with technical proficiency, which is a

necessary pre-requisite for the higher order skills necessary to obtain and manipulate

information. It also examines other factors such as the effect of the learning on

changing attitudes towards ICT and the willingness and ability of the learner to apply

ICT in a wide range of professional or academic tasks. In the context of this study, the

model examines the extent that the ICDL standardised training provides teachers with

fundamental ICT skills. It also evaluates the effect of the training on the attitude of

teachers towards the use of ICT in general and the use of ICT in classroom

instruction. Based on the theoretical model, the ICT or ICDL training would not be

14

considered effective unless it meets the criteria of providing both basic ICT skills as

well as critical thinking skills necessary to apply the knowledge in a professional

teaching environment.

1.7. Scope and Limitations of the Study

The scope of the study is restricted to an investigation of ICDL training

among Omani teachers. The scope of the quantitative phase of the study is limited to

Omani teachers that have completed ICDL training. The scope of the qualitative

phase of the study is limited to Omani teachers in the process of taking the ICDL

training course in the database module using Microsoft Access. As a result, the study

examines only the ICDL training courses as provided by the ICDL GCC Foundation

in the training centres in the Sultanate of Oman. The scope of the study is further

limited to an investigation of the ICDL training programme and does not include

direct assessment or observation of the way in which teachers completing the

programme use or apply the knowledge in classroom teaching. The limitations on the

scope of the study are necessary to reduce the effect of confounding variables that

could influence the findings from the use of courseware and teaching methods offered

by different training organisations.

The scope of the study creates a limitation for the study in the ability to

generalise the findings concerning the effectiveness of ICDL training beyond the

population of Omani teachers. While the quantitative component increases the ability

to generalise the findings, the data is based on the use of a specific type of courseware

and instructional approach used by the ICDL GCC Foundation. Another limitation of

the study is the possibility of researcher bias influencing the findings. In the

15

qualitative phase of the study, the researcher interacts with the data collection process

and the weight placed on the data during analysis. The use of the mixed method

approach with a quantitative component, however, reduces the influence of researcher

bias on the findings.

1.8. Outline of the Study

The second chapter of the study contains a review of literature related to the

topics of ICT and ICDL training and effectiveness, and issues related to teacher ICT

readiness. Following the literature review is a methodology chapter, which details the

specific research design and procedures used in the quantitative and qualitative

phases of the study. The fourth chapter presents the findings from the statistical

analysis of the quantitative data and the analysis of the qualitative data, and includes a

discussion of the findings. It combines the data to form a case study of the

effectiveness of ICDL training among Omani teachers. The final chapters contain a

summary of the research, conclusions linking the findings to the literature, and

recommendations for future research.

16

Chapter 2: Literature Review

2.1. Introduction

ICT in education and the effectiveness of EDCL/ICDL are examined in the

literature through both quantitative and qualitative studies. The literature generally

assumes the theoretical perspective that instructors must have both fundamental and

critical thinking competencies with ICT to incorporate technology into classroom

instruction. The literature review is divided into sections consisting of: a) ICT skills

education, b) ECDL/ICD education, and c) ECDL/ICDL effectiveness. Literature

focusing on ICT and the ICDL programme in the Sultanate of Oman is also included.

2.2. ICT

2.2.1. Defining ICT Literacy

No standard] definition of ICT literacy exists with institutions establishing

local objectives for ICT (Kennewell, 2002). Gordon (2007) noted that ICT represent

core skills and transferable skills necessary to use information technology. The skills

are core because they require knowledge about the fundamental processes necessary

to input, manipulate, and extract information from an information technology system.

The skills are transferable because they allow an individual to use the core skills on

any type of information technology system. ICT in education plays a critical role in

developing human capital and in increasing the standard of living (ECDL/ICDL,

2008). Kennewell (2002) suggests that ICT competency can be defined in terms of

specific skills such as the ability to recognise when using an ICT system is required,

the ability to use the functions of an ICT system, and the ability to understand the full

range of applications available to solve specific problems. Both Kennewell’s and

17

Gordon’s definitions suggest that ICT literacy encompasses both computer literacy

and information literacy. Computer literacy is defined as an understanding of the

basic function of a computer and understanding the use of some software applications

Information literacy is an ability to know when information is needed as well as the

ability to locate, analyze, and effectively apply the information to a specific

circumstance (Easton, Easton & Addo, 2006). Rockman (2004) indicated that ICT

literacy involves the ability to use technology to collect and manipulate information,

which is a specialised application of information literacy theory. Whitworth (2009),

however, suggests that people may not be “mentally or cognitively free” (p. 304) to

define their information needs and might not understand the conflicts between what

information they need and what limitations exist in getting the needed information.

Whitworth also introduces media literacy and states that a critical understanding of

the medium might be what actually defines literacy, not just the ability to use and

operate the medium. Garnett (2008) defines ICT literacy a little differently. He offers

that ICT literacy is the ability to use a computer to access resources on the World

Wide Web and to use email as a communication tool. Garnett lists six literacies that

he feels are interrelated: 1) computer literacy, 2) ICT literacy, 3) information literacy,

4) system literacy, 5) e-learning literacy, and 6) e-government literacy. ICT literacy

includes not only having basic computer skills, but also having the ability to

recognise the type of information necessary to solve a problem, having the knowledge

about methods to access the information, understanding limits to accessing the

information, and having the capacity to apply critical analysis to the information.

Garnett’s six literacies suggest that the definition of ICT literacy might be further

18

expanded to include system literacy and Internet/Web literacy, and Whitworth might

add media literacy. Clearly, the definition of ICT literacy is still being developed and

the lack of a standard definition contributes to the difficulty in establishing objectives

for ICT education.

2.2.2. ICT in Teaching

A review of research examining the use of ICT in teaching identified two

categories of barriers to the use of ICT in the classroom consisting of teacher

knowledge and behavioural issues and school-related issues (Becta, 2004). Barriers

related to teachers include insufficient knowledge and confidence to use ICT in the

classroom, the belief that ICT does not enhance learning, and difficulties with

classroom management when using computers. Barriers related to the school include

insufficient or obsolete ICT equipment, lack of technical or administrative support,

and the failure to differentiate training for teachers with varying ICT skills. An

investigation of the factors influencing the use of ICT in classrooms among primary

teachers in five European nations determined that the teacher-related factors were

more significant than school-related or structural factors. If teachers did not believe

they had sufficient ICT knowledge or skills, they were reluctant to incorporate ICT

technology into classroom teaching methods. A qualitative study by Zakopoulos

(2005) also identified similar behavioural and structural barriers to incorporating ICT

into classroom teaching among primary teachers in the United Kingdom. The

behavioural barriers included lack of time to prepare computer activities related to the

curriculum and insufficient training and knowledge in ICT. The structural barriers

19

included insufficient technical support and restrictions contained in the curriculum

related to the type of learning activities used in the classroom.

Lau and Sim’s (2008) investigation of the factors influencing adoption of ICT

by secondary school teachers in Malaysia found that prior training and knowledge of

ICT was critical for the development of a positive attitude towards ICT and increased

use in the classroom. While most teachers participating in the study had a generally

positive attitude towards ICT, they often did not incorporate ICT instruction in their

teaching. Significant reasons for not incorporating ICT into teaching were limited

knowledge of methods to integrate ICT into teaching and limited knowledge about

how to make full use of ICT. These findings suggest that ICT training for teachers

should extend beyond instruction in fundamental technical skills and include

instruction in the use of ICT in the classroom. An additional finding of the study was

that lack of technical support was a reason for teachers not using ICT in the

classroom, because teachers lacked the competencies to make technical adjustments

or repairs to IT equipment. This finding is similar to a structural barrier to ICT

training of insufficient technical support identified by Becta (2004). The findings of

Smeets and Mooij (2001) also indicated that teacher confidence in their knowledge

and use of ICT influences the way in which ICT is integrated into classroom teaching.

2.2.3. Teacher Readiness for ICT Instruction

Teacher readiness for ICT instruction is a critical issue for ICT literacy, with

numerous studies indicating that primary and secondary teachers often lack sufficient

competencies to incorporate ICT instruction in the classroom (Law & Sim, 2008).

Gordon (2007) indicated that official programmes to encourage teachers to

20

incorporate ICT into teaching had little effect unless the teachers had the necessary

skills and incentives to use ICT. Teachers differ from other ICT users because they

are expected to maintain professional status with students and colleagues and must

learn new computer skills and often use the skills before they have had adequate time

to consolidate the learning (Preston, Danby, & Wegrif, 2005). Teachers should have

sufficient competencies in the technical skills to focus on using ICT for more

advanced academic applications rather than focusing on fundamental software

operations. In addition, ICT competencies for teachers should extend well beyond the

fundamental skills necessary to use IT systems or software applications to include

sufficient information literacy to instruct students in critical evaluation of information

sources and uses of information to solve problems (Rockman, 2004). The readiness of

teachers to teach ICT is an issue separate from structural factors such as the

sufficiency of computers in the classroom to support ICT instruction (Zakopoulos,

2005).

2.2.4. Approaches to ICT Education

Reffell and Whitworth (2002) noted that the approach to ICT teaching in

universities is to focus on the skills necessary to operate the equipment rather than the

skills necessary to evaluate the effectiveness and relevance of ICT outputs. In

addition, the skills necessary for performing different ICT tasks are highly varied and

do not involve understanding the methods of providing inputs to automated software

programmes. In general, ICT education does not focus on providing students with the

full range of skills necessary to perform ICT functions. The lack of integration in ICT

21

education results in gaps in student knowledge. Student often develop strong skills in

some areas of ICT knowledge while lacking skills in other areas.

A subsequent assessment by Whitworth (2005) indicates that universities in

the United Kingdom traditionally approach ICT instruction as a service subject where

the expectation is that students will attain proficiency in operational skills necessary

to use the technology. In the context of higher education, service subjects are

accorded low status. Although ICT education is critical for success in both academic

and post-academic endeavours, it nonetheless suffers from low prestige as a service

subject. Expertise among teachers varies for instructing students in ICT. This leads to

poor outcomes such as students rejecting the use of technology in the classroom. The

failure to provide appropriate ICT training does not permit university students to

build on their ICT knowledge and to transfer the knowledge to uses in different

academic courses (Rockman, 2004).

Tanner (2002) indicated that ICT instruction in schools at the secondary level

should have multiple subjective and objective goals. The subjective goals include

instilling confidence in ICT users and a sufficient sense of self-efficacy to allow

students to explore ICT without tutorial guidance. The objective goals include the

mastery of the basic skills necessary to use ICT systems, which are foundations for

the subsequent development of higher order skills necessary for full ICT capabilities.

Tanner (2002: 7) further defines higher order ICT skills as recognizing when the use

of ICT might be appropriate, planning how to approach a problem using ICT, making

and testing hypotheses, monitoring progress in a task and evaluating the result,

reflecting on the effect of using ICT in a particular situation. The higher order skills

22

are necessary to use ICT effectively for teaching academic subjects and for

maximising the value of the ICT systems in business applications. Rockman (2004)

suggested that the objective of ICT instruction in secondary schools should be to

provide students with sufficient information literacy in addition to the fundamental

skills necessary to use ICT systems. Kennewell (2002) indicated that the specific

curriculum used in the secondary schools for ICT education should incorporate

teaching to achieve specific objectives such as recognition of the right time to use the

ICT system to solve a problem. Although Tanner’s (2002), Kennewell’s (2002) and

Rockman’s (2004) discussion of ICT in the secondary school identifies the general

theoretical principles underlying the development of curriculum, it does not provide

specific information to teachers about instructional content and the type of

fundamental tasks students must learn to support subsequent development of higher

order skills.

Jung (2005) developed a theoretical model of teacher training for ICT

intended to address the issues of skills knowledge, confidence, and understanding of

the methods to use ICT in the classroom. The model has two learning continuums.

The first continuum is bounded at one extreme by learning how to use ICT, which

progresses to the other boundary of learning by using ICT. Learning how to use ICT

involves the fundamental skills necessary to obtain and extract information from ICT

systems. Learning by using ICT involves using ICT to solve problems and to acquire

knowledge useful in general academic endeavours. The second continuum is bounded

at one extreme by core technologies and at the other extreme by complementary

technologies. At the core technology extreme, the teacher must obtain skills with

23

basic ICT hardware and software. The eventual progression is towards the ability to

engage in a learning curve necessary to use all types of ICT. While this framework is

an ideal learning pattern for teachers responsible for integrating ICT into the

classroom, the theory has not been tested in practical situations involving instruction

for teachers.

2.2.5. Effectiveness of ICT Training

The effectiveness of ICT training depends on variables such as the methods

used in instruction, the content of the instructional materials, and the attitudes of the

students towards technology (Preston, Danby, & Wegrif, 2005). Ideally, ICT training

should take different types of learning styles into consideration and compensate for

personal characteristics such as low self-efficacy with the use of computers and

technology systems. Poulter and McMenemy (2003) conducted a survey to evaluate

the effectiveness of advanced ICT training modules at the university level. The

findings of the survey indicated that ICT training is most effective when students

have basic computer skills and can focus on learning higher order skills such as

troubleshooting. The findings also indicated that the ability of tutors to communicate

with students in the learning process was an important factor for positive learning

outcomes.

In qualitative research examining the effectiveness of ICT training using CD-

ROMs with a modular learning approach to provide instruction to users outside the

educational environment, Dillon and Tearle (1999) found that outcomes depended on

the content of the training modules, the methods of presentation, and the degree of

prior computer knowledge of the learner. Maximum effectiveness in terms of

24

knowledge acquisition occurred when the content of the training was related to a

specific objective the learner perceived as useful, such as a task to which the

knowledge could be applied. This finding conforms to the general theories of adult

learning in which the learner is motivated to obtain knowledge for specific purposes.

The findings also indicated that the learners had variable motivations depending on

the content of the modules. These findings imply that the most effective ICT training

provides motivation to learners by demonstrating the utility of the ICT knowledge

rather than treating the learning solely as an academic exercise.

Stanley and Tanner (2002) recommended a series of assessments to

determine the effectiveness of ICT training. An initial baseline assessment determines

the existing skill levels and knowledge of students. Teachers should also make

continuous formative assessments to determine whether the approach for learning is

having the desired effect of imparting the desired level of knowledge to students.

Teachers should also assess the outcome the ICT training in terms of both functional

skills and the ability to apply the knowledge in other academic areas. This structured

approach emphasises assessment theory, but does not provide information about the

specific tools that should be used to perform the assessment. As a result, considerable

variability could exist in the methods used to assess outcomes and the effectiveness of

2.2.6. ICT training methods.

According to Smeets and Mooij (2001), ICT instruction in secondary schools

is most effective when the teacher adopts a pupil-centered approach in which the

curriculum and rate of learning is designed to meet the specific needs of the student.

In theory, ICT instruction allows curriculum design to have sufficient variation to

25

meet the needs of students with different learning styles by creating a multimedia and

multisensory learning environment. In addition, ICT approaches allow students to

progress at their own rate, which may be slower or swifter than the median. A survey

conducted by Smeets and Mooij (2001), however, found that teachers most often use

ICT as an adjunct to traditional teaching methods. As a result, the ICT learning tool is

linked to an existing teaching approach rather than designing the curriculum around

the ICT capabilities. The findings of Costa and Peralta (2006), however, determined

that an additional factor influencing the effective use of ICT in primary schools was

the orientation of the educational philosophy of the teacher. ICT use in the classroom

was more common and perceived as more effective when the teacher had a

constructivist rather than a traditional pedagogic orientation toward teaching.

Teachers using a constructivist paradigm felt more comfortable incorporating

emerging technologies into classroom teaching because of the perception that they

made learning more relevant from the perspective of the students.

2.2.7. Assessing ICT Literacy

Some evidence exists indicating that the assessment of ICT literacy does not

depend on the type of instrument. Robbins and Zhou (2007) used regression analysis

to compare outcomes from administering the TAIT assessment and the CSP, which is

a component of the ICDL. The study population was university students in a single

institution that had taken a basic computer skills course modelled on ICDL

curriculum. The findings showed no statistically significant differences in outcomes

using the two assessment tools. The research, however, did not control for

demographic variables such as traditional or non-traditional students, age, and

26

previous computer experience. In addition, instruments measuring the perceptions of

students and employers on their ability to use ICT to solve practical or education-

related problems (Dixie & Wesson, 2001) can assess outcomes in ICT education.

2.3. ECDL/ICDL

2.3.1. Background

The ECDL Foundation (2005) provides certification for courseware

developers to ensure that the contents of the training for ECDL are standardised and

prepare students for the certification process. The ECDL Foundation recommends

courseware developers map the required skills to the training components, ensuring

that students are exposed to multiple methods to accomplish the required tasks. The

use of ECDL approved courseware ensures that students learn the tasks necessary for

certification.

2.3.2. Standardization and Vendor-neutral approach

Most of the literature discussing the ECDL/ICDL system adopts a qualitative

and non-empirical approach describing the theoretical benefits of standardization of

content and certification to demonstrate basic computer literacy for employers. Csapo

(2002) indicated that the standardisation of the core curriculum for the ECDL/ICDL

creates a vendor-neutral approach to content prompting software developers to create

course material based on the ECDL/ICDL standards. The vendor-neutral approach

provides a range of basic or introductory computer skills not specific to any

organization or technology platform (Randall & Zirkle, 2005). The ECDL/ICDL

content may also meet the objectives for teaching the basic skills associated with ICT

in secondary schools. According to Kennewell (2002), the objective of ICT in

27

secondary school should be to provide the knowledge and skills necessary to use

common computer tools found in business and learning other subjects such as word

processing and spreadsheets. The literature implies that the primary benefit from

ECDL/ICDL is the standardization of basic computer literacy teaching objectives

(Randall & Zirkle, 2005).

2.3.3. Teaching Methods for ECDL/ICDL

Research examining the use of ECDL/ICD has determined that it forms the

core curriculum for developing ICT literacy in many secondary schools and

universities. Calzarossa, et al., (2005) used a survey questionnaire to collect data

about the methods used by 46 Italian universities to provide instruction to students in

programs leading to ECDL certification. The findings indicated that 40 universities

used classroom teaching or blended classroom and self-learning as the primary mode

of instruction, with courses offered in all seven of the ECDL modules at 28 of the

institutions. The remaining six universities used the self-learning instructional

approach, with a tutor assigned to each student to provide assistance as necessary.

The research identified practices among universities and did not examine the

effectiveness of different approaches based on ECDL certification outcomes.

In Ireland, the universities offer a mixed teaching approach in which students

can receive training by combining self-instruction courseware with occasional tutor

support or can opt for full self-instruction through automated courseware (Fallon,

n.d.). The variability in the approaches to ICT training at the university level

corresponds to the findings of Kiridis, Drossos, and Tsakiridou (2006) who found

substantial differences in the approaches used for ICT education in Greek secondary

28

schools. The variability was because of the ambivalence of the faculty towards the

need to provide students with formal computer literacy training. These studies,

however, did not evaluate cultural factors in ICT and ECDL training. Using

ECLD/ICDL as the core curriculum for ICT education does not meet the full range of

objectives for ICT education as identified by Tanner (2002) and Reffell and

Whitworth (2002) because it focuses only on basic skills to operate software

programmes.

2.3.4. Effectiveness of ECDL/ICDL

2.3.4.1. Effectiveness in Assessing ICT Literacy

Some of the qualitative literature is critical of the ECDL/ICDL approach to

assessing computer literacy. Young (2004) suggested that the ECDL/ICDL assess

only the knowledge and ability to use specific types of software such as word

processing or spreadsheet programs. It does not assess critical thinking skills or the

ability to use the software to solve research problems. As a result, some organizations

such as the Educational Testing Services in the United States have developed

alternative computer literacy assessments. Whitworth (2005) also noted that the

ECDL focuses primarily on skills valued by business and does not provide students

with ICT skills necessary for the academic environment. At the same time, the

standardised assessment of fundamental ICT skills provided by ECDL overcomes the

problems of developing appropriate measurements to assess outcomes of ICT

instruction (Stanley & Tanner, 2002). These somewhat negative opinions regarding

the effectiveness of the ECDL/ICDL approach to computer literacy may be because

the approach focuses primarily on fundamental skills, which are in the early part of

29

the Jung (2005) learning continuum that involves learning how to use ICT.

Individuals with the fundamental skills may not be sufficiently challenged by the

ECDL/ICDL skills development modules.

Perez and Murray (2006) adopted the position that the ECDL/ICDL

certification should be viewed as an assessment tool to determine if individuals have

the basic skills to operate rudimentary computer programs, but not as an assessment

of computer literacy. From this perspective, computer literacy includes theoretical

knowledge about why processes take place as well as analytic skills to manipulate

processes to achieve specific objectives. The ECDL/ICDL certification process also

addresses only one level of learning in both the four-level learning assessment model

described by Wu, et al. (2004) and the three-level learning assessment model

proposed by Hamtini (2008). The certification does not assess factors such as the

ability to apply knowledge in practical situations, the perceived degree of ease in

interacting with the ICT environment, and the confidence of the learner in using the

knowledge.

Randall and Zirkle (2005) suggested that additional vendor-specific training

might be required after obtaining ECDL/ICDL certification because the certification

does not encompass many task-specific skills. The general implication of the

literature is that the EDCL/ICDL learning modules and certifications address only the

most rudimentary computer skills and does not provide a clear indication of computer

literacy. The effectiveness of the core curriculum in the educational environment may

also depend on factors such as the ICT training of teachers and the ability of teachers

to adapt the teaching to local student needs (Mioduser, 2003). To some degree, these

30

assessments of ECDL/ICDL support the argument of Whitworth (2005) that ICT

training is regarded as a service subject that focuses primarily on teaching operational

skills and not on critical thinking for using skills in applied situations. In addition,

they support the premise advanced by Dillon and Tearle (1999) that the effectiveness

of ICT training in terms of outcomes depends on the content of the ICT training

modules as well as the degree of previous knowledge and experience of the learner.

2.3.4.2. Effectiveness for ICT Training

Dixie and Wesson (2001) conducted a survey among faculty and alumni of

universities in South Africa to assess the perceived effectiveness of the ICDL

program modules used as the foundation for classroom instruction. The findings led

to the conclusion that the ICDL modules cannot be used for complete IT proficiency

education because of the failure to incorporate critical thinking skills and theory into

the instruction. The research, however, did not examine the possibility of outcome

variation in terms of passing the ICDL certification test with the use of different

instructional approaches or techniques. These findings support the conclusions of

Randall and Zirkle (2005) from a review of previous research, which indicated that

vendor-neutral methods of instruction in computer proficiency are appropriate only

for providing basic operational skills. Additional education is necessary to provide

students with theoretical foundations in ICT and more advanced computer skills to

solve complex problems. To some degree, these findings conform to the argument of

Reffell and Whitworth (2002) suggesting that ICT education is multi-faceted, with the

ICDL programme addressing only a single type of knowledge necessary for ICT

proficiency. In addition, standardised ECDL/ICDL training modules to prepare

31

teachers in secondary schools for certification may fail to address the issue of lack of

customised training for teachers that takes existing ICT skills into consideration,

which was identified in the Bectra (2004) survey.

Research conducted by Preston, Danby and Wegrif (2005) found that the

ECDL was highly effective for instructing professional educators in the fundamental

skills necessary to use basic computer programmes. The study used a pre-test and

post-test methodology in which an initial assessment was made of teachers in an

ECDL preparation programme was made using a survey questionnaire. The same

questionnaire was administered to the teachers after the completion of the

programme. The programme was structured to provide both courseware and tutors to

answer questions. The findings indicated that the teachers completing the programme

to prepare for the ECDL believed that the modular approach provided them with the

necessary skills for certification and as a result, use basic ICT in the classroom. The

findings also indicated that success in completing the preparation programme is

linked to support for the teachers from their schools. The findings of this study should

be treated with caution, however, because the research was commissioned by a firm

producing courseware to prepare teachers for the ECDL.

Kriksciuniene, Sarkiunaite, and Sakalauskas (2006) conducted one of the few

empirical evaluations of the ECDL using a return on investment approach. The

methodology examined the time associated with providing training for students to

support their ability to obtain ECDL certification compared to the number of errors

made by the students before and after the training process in routine computer

processes. The findings indicated that the error rate decreased substantially after

32

training, which the authors argued was a proxy measure for higher productivity. The

findings imply that schools, firms or other institutions training personnel to obtain the

ECDL can expect a return on investment in the form of higher productivity that

offsets the cost of the training.

2.3.5. Gaps in Existing ECDL/ICDL Literature

A gap exists in the literature from the lack of prior research investigating the

effectiveness of the ECDL/ICDL in meeting the objectives of providing sufficient

computer skills necessary for business and educational purposes as suggested by

Kennewell (2002). The existing research has investigated methods used to prepare

students for ECDL/ICDL testing (Calzarossa, et al., 2005) and perceptions of

effectiveness (Dixie & Wesson, 2001) in different cultural environments. Fallon (n.d.)

also indicated that there has been insufficient investigation with regard to the

following: 1) the relevance of the ECDL/ICDL instructional models, 2) the cost

effectiveness compared to alternative ICT training approaches, and 3) the

effectiveness in providing students and professionals with the desired skills for

accomplishing necessary tasks. There have been no cross-cultural comparisons of the

ECDL/ICDL and no analysis of the effect of different educational methods on

outcomes.

2.4. ICT and ECDL/ICDL in Sultanate of Oman

2.4.1. ICT in the Sultanate of Oman

In 2006, the Sultanate of Oman established the ICT Authority to promote

growth and development in the nation in the ICT sector, which is intended to remedy

the deficiency of relatively low levels of ICT training (Oxford Business Group,

33

2009). The ICT Authority works in conjunction with the Ministry of Education to

develop educational programmes to ensure that students in Omani schools receive

sufficient basic ICT training that can be applied to academic and business tasks. The

ICT education programme in the Sultanate of Oman is based on an integrated strategy

developed by the Ministry of Education to ensure that students obtain both core skills

and critical thinking abilities when using technology (Ministry of Education, n.d.).

The core skills include the ability to use ICT to obtain information and to perform

routine processes. The critical thinking skills are more complex and involve the

ability to synthesise ideas and construct new knowledge using ICT and the ability to

identify problems in processes and outputs. A particularly important critical thinking

skill is the ability to adapt to change and independently acquire new ICT knowledge.

The educational programme is also intended to increase the general computer literacy

level of the Omani population by offering short skills-development courses open to

the general public (Oxford Business Group, 2009).

2.4. 2. ECDL/ICDL in the Sultanate of Oman

The Sultanate of Oman introduced the ICDL programme to general education

students in grade 11 in 2004 as a mandatory part of the curriculum (Ministry of

Education, n.d.). The purpose of the programme was to ensure students had basic

competencies in skills such as word processing and spreadsheet operations necessary

for the job market. The programme was also intended to make ICDL accreditation

available to all teachers, who were particularly urged to obtain ICDL certification to

ensure they have fundamental ICT skills (Ministry of Education, 2004). To

implement the ICDL programme, 360 schools were designated as ICDL training

34

centres and 12 schools were designated as ICDL testing centres. In 2007, however,

the Sultanate of Oman eliminated the ICDL programme as mandatory for grade 11

replacing it with optional ICT courses for general education students. The ICDL

training remains available for students and teachers at the designated schools and

certification can be obtained through the testing centres. The current approach to ICT

education in the Sultanate of Oman could lead to insufficient range of knowledge,

which is a potential outcome from the lack of integrated ICT education as noted by

Reffell and Whitworth (2002). In addition, there have been no assessments of the

effectiveness of ECDL/ICDL in Oman using approaches such as the four-level ICT

training assessment model recommended by Wu, et al., (2004).

2.5. Conclusion

ICT, literacy, described in terms of both core skills and transferable skills, is

an important element of education from the standpoint of both students and teachers.

ICT education needs to include core skills training, critical thinking skills applied to

ICT selection and use, and the ability to evaluate the outcomes related to the use of

ICT. In general, and specifically in the Sultanate of Oman, ECDL/ICDL programs are

not achieving all of these goals.

35

Chapter 3: Methodology

3.1. Introduction

This study used a concurrent mixed quantitative and qualitative method to

investigate the effectiveness of the ICDL in-service training program for professional

development and ICT competency among teachers in Oman. The quantitative phase

of the study uses a survey questionnaire to collect data about the effectiveness of the

ICDL training programme among teachers in Oman who completed the ICDL

programme. The qualitative phase of the study uses teachers' interviews, classroom

observations, and pre-and post tests to collect data from teachers about the

effectiveness of ICDL training. The use of these mixed methods supported the

development of a case study of ICDL training in a Microsoft Access module.

3.2. Justification of the Methodology

The mixed method approach to research uses a pragmatic research paradigm that

combines the positivist paradigms associated with quantitative and empirical research

with the phenomenological research paradigm associated with quantitative research

(Gliner & Morgan, 2000). The positivist approach to research uses deductive

reasoning to draw conclusions about the subject under investigation from data

collected with objective methods using boundaries established by the researcher. The

quantitative phase of the research is based on the positivist paradigm and has the

advantages of the ability to collect numerical data capable of statistical analysis to

determine the influence of independent variables on the dependent variables.

Quantitative research findings are also capable of generalisation to a larger population

36

if the data was gathered using appropriate sampling procedures. The disadvantage to

the quantitative approach to research is the need for the research to remain confined

within artificial boundaries that may overlook critical factors influencing variables.

The phenomenological approach to research uses inductive reasoning to draw

conclusions about the subject under investigation from data collected with subjective

methods with the subjects of the research establishing the boundaries for the

investigation (Creswell, 2003). The qualitative approach to research allows the

subjects of the research to establish the boundaries for the data collection, with the

researcher interacting with the data during collection and analysis. The advantage of

the qualitative approach is its ability to identify a wider range of factors influencing

the topic under investigation. The disadvantage of the qualitative approach is the

possibility that researcher bias will influence the findings. When the two methods are

used in combination in a mixed approach to research, the advantages of each method

overcomes the disadvantages of the other method to provide a more accurate and

complete presentation of reality (Gorard & Taylor, 2004). The mixed method

approach and methodological triangulation with different approaches to data

collection and analyses provides greater rigour for the findings because of (Tashakori

& Teddie, 2008).

37

Table 1. Research Overview

Table 2. Quantitative and Qualitative Processes

Quantitative Study Process Qualitative Study Process

Invite and send surveys to Omani primary and secondary teachers from Al Dhahera Educational Region who have completed ICDL training to participate in responding to survey

Structured and unstructured observation of 12 student class in Microsoft Access ICDL Training

Completed surveys are returned by participants

Interview 12 students after they complete Microsoft Access ICDL Training (semi-structured interview)

Administer Pre-test to 12 students in Microsoft Access ICDL Training

Interview 2 ICDL trainers during the ICDL training (semi-structured interview)

Administer Post-test to 12 students after completion of Microsoft Access ICDL Training

38

3.3. Merging Quantitative and Qualitative Data

Driscoll, et al. (2007) suggests a method for merging quantitative and

qualitative data. Qualitative data analysis software like NVivo is able to generate

quantitized frequencies of emergent themes in narrative data and can suggest

influential codes. Qualitative data can also be enumerated by the frequency of themes

within the relevant sample or by the percent of themes or categories related to a given

respondent category (e.g., gender). This quantitized qualitative data can then be

effectively compared to the quantitative data captured in the study. The disadvantage

to this method is that the rich content and depth of understanding gained through

qualitative methods can be lost through the quantitization process. One challenge to

merging the two types of data is that qualitative data is usually obtained with smaller

sample sizes than the quantitative data. A sample size that is too small might not

allow quantitization of the qualitative data.

The process for merging qualitative and quantitative data recommended by

Driscoll, et al., (2007) includes the following steps: 1) code open-ended in-depth

interview content with content analysis software, 2) integrate the data using a data

management package (e.g., Microsoft Access), 3) assess combined data, explaining

discrepancies in survey responses by providing contextual data on those survey

responses.

3.4. Quantitative Phase of the Study

Because of the mixed method research design, the procedures used in the

quantitative phase of the study differed from the procedures used in the qualitative

phase. The quantitative phase used a survey questionnaire disseminated to teachers

39

who had completed the in-service ICDL training to obtain data concerning the

independent and dependent variables of the study. The quantitative procedures

required establishing a sampling protocol, developing test instruments and data

collection procedures, and establishing the reliability and validity of the instruments.

3.4.1. Survey Questionnaires Methodology

Survey methodology is an important measurement in social research. Surveys

include any measurement method that is centered on asking participants questions in

a structured format. Surveys are used to reduce bias in research and their standardized

format allows for better generalizability of quantitative results (Trochim, 2009).

Survey questionnaires depend on the participant’s ability to respond, their

understanding of the questions, and their motivation to respond honestly. Survey

questionnaires do not allow researchers to ask probing questions to understand a

participant’s honesty, motivation, or understanding.

3.4.2. Quantitative Sampling

The sampling for the quantitative phase of the study used purposive sampling

which is appropriate when the objective of the sampling is to relate the characteristics

of the respondents to certain characteristics under investigation in the research (May,

2001). Sampling required two approaches to identify the study population. A

directory of Omani primary and secondary teachers from Al Dhahera Educational

Region in Oman was used to identify potential subjects for the survey questionnaire

component of the study, which is the approach to identifying members of the

population recommended by Lodico, Spaulding, and Voegtle (2006). The criteria for

inclusion in the survey questionnaire sample were completion of the ICDL training,

40

and a current teaching position in a primary or secondary schools in Oman. Obtaining

ICDL certification was not a criterion for inclusion because some prospective

participants completing the training course may not yet have qualified for

certification. The individuals in these directories and who met the criteria were

solicited for participation in the study through email contact, with an explanation of

the purpose of the study and a qualifying question for participation as to whether they

have completed ICDL training. The sample consisted of those teachers who agreed to

participate. The sample included 203 teachers who finished the ICDL training in Al

Dhahera Region in Oman. . The procedure resulted in 82 participants that returned

completed surveys.

3.4.3. Quantitative Instruments

A survey test instrument was developed for the study to collect data from

teachers that have completed the ICDL training (see Appendix A). The survey

questionnaire consisted of three sections. The first section used closed-end questions

to obtain data from the respondents concerning motivation for taking the training,

existing ICT competency, use of ICT in the classroom, and ICDL training evaluation.

The second section consisted of a five-point Likert scale asking respondents to rate

their level of agreement with various statements concerning ICDL training ranging

from strongly agree to strongly disagree, which is a scale for assessment

recommended by Bernard (2000). The third section of the questionnaire used open-

ended questions to obtain information about recommended improvements to the

ICDL training course.

41

3.4.4. Data Collection Procedures

The data collection procedures for the survey component of the quantitative

research involved the dissemination of the survey questionnaire by email to the

teachers agreeing to participate in the study. A follow-up request was made to non-

respondents approximately one week after the initial dissemination of the survey

questionnaires. The procedure resulted in the return of 82 usable survey

questionnaires. In order to preserve the confidentiality of the respondents, the survey

questionnaire instruments did not contain information about the identity of the

respondents. While an initial coding was used for follow-up with the survey

questionnaires the coding linking the identity of the respondents with the instruments

was destroyed immediately after the completion of the study. Preserving

confidentiality ensured candour in the responses and maintained the privacy of

respondents (Schutt, 2008).

3.4.5. Data Analysis

Descriptive statistics describe the basic elements of the research data and

provide summaries about the sample and the measurements. Together with graphics

analysis, descriptive statistics form the basis of most quantitative analysis of data

Descriptive statistics allow comparisons of the same data across different groups

Descriptive statistics are helpful in understanding a phenomenon without trying to

find relationships that extend beyond the data itself (Trochim, 2009). Descriptive

statistics were used to describe the respondents in terms of gender, # of years

teaching, subject taught, and prior experience using ICT, showing frequencies and

percentages for each demographic. Additional descriptive statistics were generated

42

to describe the frequencies and percentages for each survey response with regard to

the participants’ motivation to take the ICDL course, skills that they hoped to

improve, and skills that they believe they did in fact improve, current and post-

training ICT skills, reasons for not using ICT in teaching, and new applications that

the teacher might use after training. Additionally, the frequencies and percentages

were generated for questions related to the following: 1) most useful modules, 2)

training aspects that were missing, 3) content that was missing, and 4) optimal

learning style for the participant. Each of these response categories was then

compared based on the demographics of gender, years teaching, class taught, and

years of ICT experience.

Univariate analysis examines, across cases, one variable at a time. Three

characteristics can be looked at: distribution, central tendency, and dispersion :For

this study, the distribution of the variables described above were looked at across the

four demographic factors (gender, years teaching, course taught, and previous ICT

experience) to see the frequencies and percentages of response categories that fall into

each demographic category. Central tendency in this study will look at the mean, of

each demographic and response category. Standard deviation measured dispersion.

The Standard Deviation shows the relationship of values to the mean of the sample.

ANOVA was used to analyse the data from the survey questionnaire to

determine if ICDL training significantly influences the attitude of teachers toward

incorporating ICT into classroom teaching methods. ANOVA was also used to

determine the existence of statistically significant differences in ICT competencies

gained through ICDL training based on the mediating factors of gender, number of

43

years teaching, and class being taught. ANOVA is suitable for examining differences

among groups following a treatment such as ICDL training (Pedhazur & Schmelkin,

1991). For the ANOVA, the alpha level was set at .05, which is suitable for research

involving variables that are difficult to measure with precision (Berry & Sanders,

2000). The survey instrument also generated descriptive statistics.

3.4.6. Reliability and Validity

A pilot test with ten respondents meeting the inclusion criteria for the study

was conducted to establish the reliability of the instrument using a test-and-retest

procedure. The pilot test followed the recommendations of Aaker, Kumar, and Day

(1995) for ensuring that the questions were not ambiguous and that the instrument

produces the same results when used in repeated testing. The test and retest had a

correlation of .83, which is sufficient to establish the reliability of the instrument. The

instrument used for the pre-and-post test was the standardised ICT competency test

used for the ICDL modules. Courseware providers assessed this ICT competency test

for reliability. The external validity of the instruments was established by the use of

random selection methods for participants, which increases the ability to generalise

the findings (Black, 1999). The internal validity of the instruments was established by

a peer review of the ability of the respondents to understand the questions in the

instruments.

3.5. Qualitative Phase of the Study

The qualitative phase of the study involved the development of a case study of

ICDL training for Microsoft Access in a single class of Omani teachers. The data for

the case study consisted of observations of classrooms in which teachers were

44

receiving in-service training in an ICDL module. Additionally, semi-structured

interviews with teachers that had completed ICDL in-service training were used to

collect data for the qualitative phase of the study. . The qualitative phase also

included a pre-and-post test of competencies in Microsoft Access. Interviews are

appropriate for qualitative research because they allow the interviewees to control the

data and direction of the interview (Minichiello, et al. 1995). Interviews can help

measure learning outcomes related to Kirkpatrick’s Level 1, reaction to the learning

(Wu, et al., 2004). Observational methods are suitable to obtain data about the

methods used by teachers in the classroom (Waxman & Padron, 2004). This method

contributes to capturing learning outcomes related to Kirkpatrick’s Level Three,

behaviour changes that result from the learning (Wu, et al.). A pre-and-post test

provided information about the outcomes of the training in the Microsoft Access

class. Use of the pre- and post-test contributes to measuring Kirkpatrick’s Level 2

learning outcome related to content learned (Wu, et al.). By combining the three

methods, the research obtains data about teaching methods and student response to

the methods.

3.5.1. Sampling

Purposive sampling was used for both components of the qualitative phase of

the study, which is appropriate when the objective of the sampling is to relate the

characteristics of the respondents to certain characteristics under investigation in the

research (May 2001). The sampling for the observational and interview component of

the qualitative phase limited the observation to three classes teaching the Microsoft

Access module of the ICDL training. This class was selected for observation because

45

it involves a software application with which many of the teachers may not have had

previous experience. In addition, confining the observations to a single ICDL topic

reduced the possibility that different instructional content could function as a

confounding variable. A specific criterion for the selection of a class to observe was

that it included a minimum of ten students to ensure a sufficient cross-sectional

representation in the class. Because the research design for the qualitative phase

called for interviewing the students of the observed class following its conclusion, the

agreement of the students to participate in the interviews was also a criterion for the

selection of a class to observe. The sampling process resulted in 12 interviews of the

students in the observed Microsoft Access class.

3.5.2. Instruments

The data collection instrument used for the observation component of the

study is a modified form of the classroom observational schedule (COS)

recommended by Waxman and Padron (2004) (See Appendix B). The COS

establishes a structure for observations based on: a) interactions of students with

teachers and peers; b) purpose of interactions; c) settings or situations in which

interactions occur; d) type of material on which students are working; e) specific

activities on which students are working; f) manner or approach used by teachers and

students; g) language used. The approach focuses on the content of the interactions.

The modifications are based on the assumption that in-service teachers are motivated

adult learners. The COS approach focuses on the students for observation rather than

the teacher. This provides greater breadth to the data for determining the effect of the

instruction methods on the behaviours of students in the learning environment. The

46

interviews used with the teachers in the ICDL class under observation employed a

semi-structured conversational format with open-ended questions (see Appendix C).

The interview questions followed the recommendations of Wengraf (2001) for using

questions that employ the language common among the. Both telephone and face-to-

face interviews were also conducted with ICDL trainers.

The qualitative component of the methodology also involved the use of a pre-

and-post test instrument for teachers taking ICDL training in the Microsoft Access

class under observation. The instrument used for the pre-and-post test component of

the quantitative research was the assessment of Access competencies and was a

standardised instrument with the participants certifying they had never previously

taken the Access test.

3.5.3. Data Collection Procedure

In the COS data collection procedure, each student is observed multiple times

during the course of the observational period (Waxman & Padron, 2004). The process

repeats, producing multiple observations of each student. The data collection

procedure also collects information about the teacher and teaching methods through

the observation of interactions with the students. Observational data was collected

about the behaviours of the students and the interactions of the students with the

teacher and peers for each student during each observation period, resulting in

multiple observations per student per class observations. Several observations were

made of the same class to ensure that the data obtained from the observation was

typical, and not anomalous data caused by a temporary external factor such as student

47

weariness. The first observation was unstructured, the second structured using COS,

and the final observation was unstructured.

The interviews were conducted using Skybe and an add-in called HotRecorder

to record the interviews, with each interview lasting approximately 40 minutes. The

interviews were conducted using a conversational style, with the researcher acting as

interviewer in this process. The interviewer could ask follow-up questions to obtain

additional information or clarification of issues raised by the interviewee. The

interviewer also avoided commenting or demonstrating non-verbal reactions to the

responses of the interviewees as recommended by Taylor and Bogdan (1998) to

reduce the possibility of influencing subsequent responses. The interviews were

recorded with the knowledge of the interviewees to facilitate transcription of the data

for analysis. The interviewees were not identified by name during the interviews or in

the interview transcripts to preserve their confidentiality.

The data collection procedure for the pre-and-post test component of the

quantitative phase of the study involved administering the standardised test

instrument to the subjects of the study prior to beginning ICDL in-service training in

Microsoft Access and after the completion of the in-service training on the final day

of the training. The procedure excluded any participants that did not complete the

training. The procedure resulted in 12 completed pre-and-post tests.

3.5.4. Data Analysis

The observational data was analysed by assessing the type and frequency of

behaviours and interactions occurring with each student under observation to identify

critical events. A critical event is a behaviour or interaction that could influence

48

learning activities or outcomes (Wragg, 1999). The specific critical events examined

in the analysis were the differences in learning styles among the students and the way

in which the ICDL courseware and the interactions with the teachers addressed these

differences.

Content analysis was used to analyse the data from the interviews as

recommended by Anastas (1999). This approach examines the data to identify the

general constructs noted by the interviewees, and is referred to as open coding. The

data it then examined several additional times to determine the patterns related to the

general constructs, which is referred to as axial coding. The review of the data

continues with sufficient iterations to ensure that all patterns within the data have

been identified. The outcome of the analysis is presented as an integrated descriptive

narrative of the findings organised around the constructs and patterns identified

within the data. The analysis for the pre-and-post test data used ANOVA, which is

appropriate when the purpose of the analysis is to determine the existence of

statistically significant differences in data means following a treatment such as ICDL

training.

3.5.5. Trustworthiness

The methodology has an inherent limitation from the possibility that

researcher bias influenced the research design, data collection procedures, and

conclusions drawn from the findings of the study. Trustworthiness is the approach

used to establish the reliability and validity of qualitative research, which reduces the

effect of researcher bias. Trustworthiness involves the four elements of credibility,

dependability, applicability, and ability to transfer the data and conclusions (Lincoln

49

& Guba, 1985). Credibility in the qualitative research was established by the

triangulation of observation data, interview data, and pre-and-post test data and by

prolonged engagement with the data collected over an extended period. The use of a

structured and consistent method of collecting the data for the qualitative phase of the

study established dependability by reducing the possibility of researcher bias

influencing data collection techniques. The purposive sampling approach established

the applicability of the data and findings because the data collection occurred using

subjects exposed to the same type of learning environment for the same purpose of

obtaining the ICDL training. The element of transferability is the weakest of the

trustworthiness factors in this methodology. It is similar to external validity in a

quantitative approach and refers to the ability to generalise the findings (Potter,

1996). Because the qualitative component of the research involved a single class with

a limited number of teachers, the ability to transfer the findings to other ICDL

instructional contexts may be limited.

3.6. Limitations of the Methodology

The research design did not include observation of the teachers’ use of

technology in their teaching practices in the classroom. As a result, the data

concerning the teachers’ use of ICT in the classroom following ICDL training is

based on self-reports, which have not been objectively verified. In addition, the

research design did not include pre-ICDL training surveys for teachers to determine if

the training resulted in a change in self-reported skill levels. The Likert survey does

ask the teachers to evaluate their ICT competency, as they perceived it prior to the

50

training and after the training. However, the self-reports are not objectively verified

with observation.

3.7. Summary

The goal of this mixed method research was to add to the body of knowledge

about the effectiveness of the ICDL in-service training program for improving ICT

competency among teachers in Oman. As the literature review reveals, ICDL training

often provides value in teaching core ICT competencies, but the training can be

lacking in providing insight into how teachers might better utilize ICT in the

classroom. This study provides improved understanding of the value of ICDL training

in Oman and provides insight into how the training might be improved. The findings

in this study can be generalised since the same educational policies and training are

found everywhere in Oman because the policies come from the Capital of Oman

(Muscat) to all the educational regions.

51

Chapter 4: Findings

Because the mixed method research design used concurrent quantitative and

qualitative components, the findings are divided into separate sections for each

component. The findings for the quantitative component based on the survey

questionnaire are presented as descriptive statistics from the questions in part 2 of the

survey questionnaire. The information contained in part 3 of the survey questionnaire

is qualitative in nature. The analysis of this data is presented with the findings of the

qualitative component of the study. The qualitative findings also include the analysis

of the observations of the ICDL Access class, the pre-and-post test for the class, and

the interviews with the students and trainers from the class. The results of the pre- and

post-test are reported under the quantitative section below.

4. 1. Quantitative Findings

The quantitative findings of the study are based on the survey questionnaires

from 82 teachers that have completed the ICDL training and the pre-and-post tests of

the Access class under observation.

4.1.1. Survey Questionnaire

The quantitative findings present the analysis of the data from part 2 of the

survey questionnaire. Table 3 reports the mean and standard deviation for each

question in part two of the survey questionnaire.

52

Table 3

Mean and Standard Deviation, Part 2 of the Survey Questionnaire_____________________________________________________________________Question Mean Std. Dev_____________________________________________________________________1. ICDL Training is effective for improving myICT competencies 3.13 1.082. The ICDL training provided me with skillsI will transfer to the classroom 3.08 1.073. The ICLD training provided adequate time tothoroughly learn the information about the software 2.87 .974. The ICDL training taught me to use various ICTsoftware applications to solve future education or research problems 3.01 1.145. The ICDL training increased my understanding of theimportance of ICT in the educational system 3.38 1.156. The ICDL training increased my understanding of theway in which ICT can be integrated into classroominstruction 3.14 1.197. After completing the ICDL training and certification process, I feel more confident integrating ICT into myclassroom teaching 3.17 1.158. The ICDL certification is necessary for careerdevelopment. 3.85 1.079. The ICDL training taught me the full range of capabilities of the software applications covered bythe ICDL certification. 2.64 1.0410. The ICDL training provided all the books, handouts or online materials necessary for learning to use the applications required for certification 1.90 .9811. The ICDL trainer adapted to my needs and addressedany obstacles I experienced during the class. 3.09 .9912. The ICDL trainer provided adequate guidance andassistance to learn the applications required forcertification 3.26 1.0213. The ICDL trainer used teaching methods that werehelpful to me. 3.25 1.0514. In general, the ICDL training was useful for me as a teacher 2.93 1.0415. The ICDL training provides a strong foundation for future ICT education and learning 3.36 1.14____________________________________________________________________

53

The analysis of the means and standard deviation for part 2 of the survey

questionnaire provides an indication of the central tendency of the data, with a mean

of 3.0 suggesting the respondents were neutral towards a proposition. The findings

show that the respondents were neutral toward the proposition in question 1 that the

ICDL training was effective for improving ICT competencies, which had a mean of

3.13. In addition, the standard deviation for this question was relatively high at 1.10,

suggesting substantial variability in the responses to this question. The data also

indicated that the respondents were neutral towards the proposition in question 2

concerning whether the ICDL training provided skills that can be transferred to the

classroom. The findings from questions 4 through 7 are particularly significant for

the attitude of the respondents towards incorporating ICT use into classroom

teaching. While the respondents were neutral towards the statement in question 4 that

the ICDL training taught them to use software applications to solve future educational

problems, the findings for questions 5 and 6 suggested that the training increased

awareness of the importance of ICT in the classroom and slightly improved

understanding of the way that ICT can be integrated into classroom teaching. The

responses to question seven suggest that the ICDL training slightly increased the

confidence of the respondents when using ICT in the classroom.

The findings also showed that the respondents had substantial agreement with

the statement in question 8 concerning the need for ICDL certification in career

development. Another notable finding was the substantial disagreement among the

respondents with question 10 concerning the adequacy of the materials in the ICDL

course, which had a mean of 1.90. The data from question 14 indicates that there was

54

slight disagreement with the proposition that ICDL training was useful for teachers

but there was general agreement with the proposition in question 15 that it provides a

foundation for future ICT learning.

The data from part 2 of the survey questionnaire was used to examine whether

differences exist in ICT competencies gained through ICDL training that are based

on the mediating factors of gender, number of years teaching and class being taught.

The first approach to analysis tested whether a relationship existed between the

number of years teaching and questions 1 and 2 of the survey questionnaire. Question

1 was related to the self-reported improvement in ICT competencies while Question 2

was related to the effectiveness of the ICDL training in providing skills transferrable

to the classroom. Because the data about number of years teaching was sequential, a

regression analysis was performed between the data and the responses to questions 1

and 2. The findings showed a coefficient of correlation of .028 for question 1

and .067 for question 2, with coefficients of determination of .0008 and .004 for the

two questions respectively. This evidence indicates that no significant relationship

exists between the number of years teaching and the effectiveness of ICDL training.

55

Table 4

Means and F statistic for Gender (Questions 1 and 2)_____________________________________________________________________ Mean –Male Mean – Female F-statistic F-critical_____________________________________________________________________

Gender1. ICDL training is effective for 2.97 3.16 .574 3.96improving my ICDL competencies

2. The ICDL training provided me 3.45 3.26 5.44 3.96with skills that I will transfer to the classroom_____________________________________________________________________

The second factor examined was whether gender functions as a moderator for

the findings, with 48.8% of the respondents being male and 51.2% of the respondents

being female. The ANOVA analysis of the data is shown in Table 4 for Questions 1

and 2 of the survey. The results indicate that no statistically significant differences

existed in the means between the male and the female group for questions 1 and 2.

The mean for the male group in question 1 was 2.97 and 3.16 for the female group

while the mean for the male group for question 2 was 3.45 and 3.26 for the female

group. The analysis indicates that the differences in the means were due to normal

statistical variation in the sampling. In addition, no statistically significant differences

were found by gender for the answers to the other survey questions.

56

Table 5

Statistically Significant Differences in Responses in Subject Taught Data Grouping_____________________________________________________________________Question F statistic F Critical_____________________________________________________________________1. ICDL training is effective for 2.96 2.13Improving my ICDL competencies5. The ICDL training increased my understanding of the 4.90 2.13importance of ICT in the educational system7. After completing the ICDL training and certification 4.33 2.13process, I feel more confident integrating ICT into myclassroom teaching10. The ICDL training provided all the books, handouts 3.24 2.13or online materials necessary for learning to use the applications required for certification13. The ICDL trainer used teaching methods that were 2.40 2.13helpful to me.15. The ICDL training provides a strong foundation for 2.85 2.13future ICT education and learning_____________________________________________________________________

The third factor examined was whether the subject matter taught by the

respondents moderated the findings. These results are shown in Table 5. The

respondents indicated they taught the following subjects: English, Social Science,

Arabic, ICT, Physical Education, Mathematics, Science, and Islamic Studies. The

ANOVA analysis of the data from the second part of the survey questionnaire

indicated that statistically significant differences among the groups existed for

questions 1, 5, 7, 10, 13, and 15 of part 2 of the survey questionnaire.

57

Table 6

Means of the Subject Taught Groups with Statistically Significant Differences

_____________________________________________________________________

Question English Social Science Arabic ICT Physical Math Islamic Studies Education Studies___________________________________________________________________________1. 3.06 3.63 2.86 3.41 4.57 2.50 2.25 3.60

5 3.40 4.18 2.86 3.75 4.14 3.50 2.07 4.00

7. 2.93 3.63 2.60 3.58 4.14 3.25 2.25 4.00

10. 1.86 2.18 2.00 1.91 3.14 2.00 1.16 1.60

13. 3.20 3.72 3.20 3.66 3.57 3.50 2.33 3.80

15. 3.53 3.63 3.06 3.75 4.28 3.50 2.41 3.20

An analysis of the means of the responses of the teachers in the eight subject

matter groups provides an indication of the direction of the differences between the

groups and the results are shown in Table 6. The analysis of the responses for

question 1 indicate that the physical education and mathematics teachers did not agree

with the statement that ICDL training is effective for improving my ICT

competencies (mean = 2.50 physical education; mean =2.25 mathematics). The

mathematics and science teachers also had significant disagreement with the

statement in question 5 that the ICDL training increased my understanding of the

importance of ICT in the education system (mean = 2.86 science; mean = 2.07

mathematics). Similarly, the science and mathematics teachers disagreed with the

statement in questions 7 that the ICDL training increased their confidence for

integrating ICT into classroom teaching (mean = 2.60 science; mean =2.25

mathematics). These findings with respect to the lack of effectiveness of ICDL

58

training among science and mathematics teachers was not anticipated because of the

assumption that science and mathematics teachers are conversant with ICT and would

be more likely to appreciate the importance of ICT in education. The pattern of

disagreement or lower agreement with the statements by the science and mathematics

teachers continued in the other questions assessing the effectiveness of the training

for certification and the overall evaluation of the training.

4.1.2 Pre-and-Post Tests of Access Class under Observation

During the qualitative phase of the study, the 12 members of the Access class

under observation took a test prior to the beginning of the class to establish a baseline

for Access skills and another test at the conclusion of the class to determine the effect

of the class for improving Access skills. These results are quantitative and the

analysis method is discussed here in the quantitative phase, even though this data was

part of the qualitative phase of the study. The mean for the group in the pre-test was

40.8 out of a possible 100, with a standard deviation of 12.02. The mean for the group

in the post-test was 70.33 with a standard deviation of 14.80. When the means of the

group were analysed with ANOVA, it produced an F statistic of 28.53 and an F

critical of 4.30. This data indicates that attending the class produced a statistically

significant difference in the mean scores of the study participants. The higher mean of

the post-test indicates that the class was effective for improving the Access skills of

the learners. The increase in standard deviation, however, suggests that the class

produced variable outcomes, with some students benefiting more from the instruction

than others do.

59

4.2 Qualitative Findings

4.2.1 Descriptive Statistics: Motivation, Expectations, and Skills Gained

The first section of Part 1 of the survey questionnaire obtained data on the

motivation and expectations of the respondents for taking the ICDL training.

The findings indicated that 30.4% the respondents were motivated to take the ICDL

training for professional development, while 50% of the respondents took the training

because it was an employment requirement. The remaining 19.5% were motivated by

personal interest. The data also provided information about the ranking of skills that

the respondents hoped to improve through the ICDL training. This section also

identified the priority for the skills the respondents hoped to improve through the

ICDL training, which is presented in Table 7.

Table 7

Skills Expected to Improve form ICDL Training as Ranked by Respondents

_________________________________________________________________1. Improved Computer Skills2. Microsoft Application Skills3. Word Skills4. Access Skills5. Improved Internet Skills6. No Need_____________________________________________________________________

Among the respondents, 41.4% reported no change in their self-rated levels if

ICT competency before and after taking the ICDL training. The group reporting no

change included individuals who rated their pre-training competency at both the high

and low end of a five-point scale, suggesting that the level of pre-existing ICT

knowledge was not a factor influencing the outcome of ICDL training.

60

The data obtained from the first section of the survey questionnaire also asked

respondents to indicate some of the reasons they have not used technology in the

classroom. The data indicates that the two reasons most often cited by the respondents

are personal lack of technology skills and lack of sufficient ICT equipment in the

classroom. Some of the respondents also noted that technology did not enhance

learning. The respondents also prioritized the skills gained in the ICDL training that

they were most likely to transfer to the classroom setting, which would occur despite

the barriers to use of ICT in the classroom identified by the respondents. Table 8

presents the skill gained in the ICDL training that the respondents reported as most

likely to transfer to classroom teaching in order of priority. The priority in Table 8

also reflects the evaluation of the usefulness of the ICDL modules.

Table 8

Skill gained in ICDL Training most Likely to Transfer to the Classroom Settings_____________________________________________________________________1. Presentation (Power Point)2. Spreadsheets3. Basic Concepts of IT4. Information and Communication5. Using the Computer and Managing Files6. Information and Communication_____________________________________________________________________

61

4.2.2 Descriptive Statistics: Deficiencies in the Training

The respondents also indicated that the lack of one-on-one time with the

trainer was a significant deficiency in the ICDL training, which was followed by

insufficient homework. The type of materials most often identified as missing form

the course was paper handouts followed by web tutorials. The preferred leaning style

among the respondents was direct instruction from the trainer. The data suggests that

the respondents would have preferred greater direct instruction from the trainer on a

one-on-one basis, with additional explanatory materials distributed during the ICDL

training.

4.2.3 Descriptive Statistics: Suitability of Training for Teachers, Recommended

Improvements

Part 3 of the survey questionnaire contained open-ended questions obtaining

information about suitability of the ICDL training for teachers and suggestions for

improving the training. The data from this section of the survey questionnaire

indicated that 60% of the respondents believed that ICDL training was important for

teachers, although there was disagreement about the actual effectiveness of the

training experienced by the respondents. Among the respondents who did not believe

that ICDL training was suitable or important for teachers, the reasons cited included

the lack of training concerning methods to incorporate ICDL into classroom teaching

and the lack of training about using the software in general applications not related to

the ICDL test. A common criticism among the group that did not believe the training

was relevant to teachers was the unwillingness or inability of the trainer to discuss

ICT application uses in the classroom.

62

The respondents provided numerous recommendations for improvements to

the ICDL training. Respondents felt that the amount of time for the training in each

module should be increased. They also suggested that an initial assessment be made

of pre-existing ICT knowledge and skills, with the classes divided based on ability.

Respondents felt that the trainers should be knowledgeable in the full range of

applications of the software to the educational environment because all students in the

classes are teachers interested in professional use of ICT skills. The training should

also provide books, handouts, and other materials to supplement the CD testing

software. An additional area of improvement would be adding applications such as

Photoshop or Flash that are outside the ICDL testing, but which represent ICT

applications that teachers might use.

4.2.4 Observation of Access Class

Descriptive statistical data from the pre-and-post tests is included as part of

the observational findings for the Access class.

63

Table 9

Among the individuals taking the class, 8 or 66% passed the ICDL test for the Access

module. The highest rate of failure in the ICDL test was among teachers of Arabic

and Islamic Studies. Three out of four of the Arabic teacher participants failed the

ICDL test and one out of two Islamic Studies teacher participants failed.

The observation of the Access class consisted of unstructured and structured

observations. The unstructured observation occurred with the initial class during

which the learners presumably having no prior knowledge of Access and employed a

COS method. The initial class lasted for 67 minutes. In the first ten minutes, students

listened to direct instruction from the trainer in the basic use of Access. The trainer

used an overhead projector to demonstrate. The students took notes during the direct

instruction period. For the next fifteen minutes, students perform rudimentary tasks

64

with the software such as creating a file name and a table as instructed by the trainer.

The instructor frequently asks general instructional questions with no students able to

provide the answers. When the instructor asked if the learners had any questions, the

same group of learners responded with questions. Forty minutes after the class began;

the instructor asked if any learners required assistance with the assigned task of

creating fields, with three learners asking for assistance. The instructor provided the

one-on-one assistance. At the same time, the other learners were working in pairs

with peers. One hour after the beginning of the class, the trainer moved to the next

task of customising tables, and sent a file to the learners’ computers to allow them to

practice sorting records.

The criteria for the structured observations were based on the data obtained

from the questionnaires and interviews. The observation indicated that 100% of the

students were beginners in Access, with 75% attending the class to obtain

certification while 25 attended the class for professional development. During the

class, the only materials provided consisted of CDs with software applications. No

supplemental materials such as paper handouts, videos, or tutorials were provided to

the students. The central training material was the test software. The instructor used a

projector to provide visual demonstrations to accompany direct instruction. Although

there were more than a sufficient number of computers for students, the test software

did not operate in many of the computers and some were infected with viruses. As a

result, only 11 computers were available for 12 students, forcing two students to share

the same computer. The fact that two students had to share a computer created a risk

to independent learning for these two students because one student was interacting

65

with the software while the other student was merely watching. During the class, the

students did not request additional materials from the instructor.

During the class, the trainer thoroughly covered the topic of certification

requirements, but did not cover topics related to use of ICT in the classroom or using

ICT tools to improve productivity. The trainer provided minimal information about

critical assessment of ICT tools. Although the trainer tried to promote critical

thinking by asking questions, the learners were not able to provide answers to the

questions. As a result, the instructor rather than the learners provided the answer.

During the class, the instructor allowed time for practise of basic tasks with the

instructor and peers providing assistance. In addition, the instructor welcomed

questions, and adapted to the needs and obstacles identified by the students. The

instructor sometimes attempted to engage the learners in discussions about the topic,

engage in subject-related tasks, and managed technical issues that arose. The

instructor did not inform the students of future learning opportunities in ICT and the

students did not inquire about such opportunities. In addition, the materials provided

for instruction did not provide information about future learning opportunities in ICT.

Several unexpected observations were noted. The learners were tired and

distracted during the class, which may have been because of a relatively short amount

of time between finishing work teaching in schools and the ICDL instructional

session. The instructor did not provide any supplementary assignments for the class,

relying solely on the test software for Access. As a result, the students had no

opportunity to extend their knowledge of ICT beyond the information necessary for

ICDL certification. The trainer was also ineffective in managing the class, with at

66

least ten minutes of instructional time lost through side discussions with individual

students. The trainer failed to discuss some of the issues raised by individual learners

with the entire class, confining the discussion to the individual raising the issue.

4.2.5 Interviews with Trainers

The open coding of the data from the two interviews with the ICDL trainers

identified the themes or dimensions of specific training to instruct ICDL courses,

methods of instruction, supplemental materials, and assessment. In the theme of

specific training for ICDL instruction, the interviewees noted they had received no

specialised training for the modules or the subject matter. One trainer had a

Bachelor’s Degree in ICT education while the other trainer had Bachelor’s Degree in

Education with certification in physics and mathematics. Both trainers used direct

instructional methods. There was a difference, however, between the two trainers in

the approach to one-on-one instruction. One trainer considered ICDL students as

adult learners with the motivation and responsibility for acquiring knowledge from

the instructional course. This trainer stated: ‘If the learner does not develop such

skills by practicing and using his mind I as a trainer can do nothing for him.’ The

other trainer viewed the students as adult learners, but was willing to provide one-on-

one assistance. Both learners believed the purpose of the training was specifically to

prepare learners to pass the ICDL certification test. The trainers indicated that

supplemental materials such as paper handouts were distributed at the beginning of

the course and believed the amount of materials were adequate to support learning.

The assessments performed by the trainers are aimed at ensuring the students are

prepared for the ICDL testing. At the same time, the trainers admit that passing the

67

ICDL test does not mean that the skills learned are the same for all students or that

the student will be able to apply the knowledge gained from training in a work related

situation. The trainers also suggest that the learner’s primary concern during class is

passing ICDL certification. One stated, “The learners do not care about them (higher

level skills). Their only concern is to pass the test”

4.2. 6 Interviews with Students of Access Class

The open coding of the interviews with the students in the Access class

identified the themes of insufficient time in the ICDL training, the focus of the ICDL

training on passing the certification test, structural deficiencies in the ICDL

instructional methods, change in attitude towards ICT following the ICDL course,

and increase in confidence for use of ICT in the classroom.

All the interviewees noted a shortcoming in the ICDL modules of insufficient

instructional time to learn the applications. The expectation of many of the

interviewees when taking the course was to have sufficient time during the training

sessions to learn the applications. The interviewees noted that the expectation was

generally not met. This was a particular difficulty for individuals who did not have

sufficient self-efficacy to continue studying the subject on their own. One interviewee

commented that: ‘we did not have enough time for discussions with him [the tutor].’

An issue related to time was the standardised number of hours of instruction for each

module course, although the skills required for some software such as Excel or

Access were more complex than for other types of software such as word processing.

An additional criticism of the interviewees was the strong focus of the training

on the requirements to pass the ICDL certification testing. The training was limited to

68

the specific skills necessary to pass the test. Other applications or uses of the software

not directly related to the certification testing were not considered because of the

limited time allowed for instruction. Possible additional uses of applications included

possible uses of the software for classroom instruction. Several of the interviewees

indicated that they focused on the testing software to a greater extent than the CD

provided as a supplement. One outcome of insufficient time was the need for students

to practice the tests together.

The time of day when the courses were offered presented a structural

deficiency in the training. Courses were offered at the end of the normal workday

when the students were tired. The student’s physical exhaustion seemed to reduce the

rate of learning. Another structural problem suggested by the students was the

mandatory attendance. This seemed to result in students not being motivated to learn

the material. For some students with existing ICT competencies, the material was too

fundamental, while it is too difficult for other students without sufficient basic ICT

skills. Another structural deficiency noted by the interviewees was a lack of

instructional support materials. While the students could access the instructional CD

software, they felt a need for additional books, handouts, and web-tutorials to allow

them to obtain a more comprehensive knowledge of the material. Not all the

interviewees, however, had sufficient ICT skills for independent study. These

findings suggest that the course did not address the needs arising from the variability

of the learners’ existing ICT knowledge. The majority of the interviewees, however,

were satisfied with the direct instructional methods used by the ICDL tutor. Several

interviewees also considered the poor maintenance of computers in the training

69

classroom as a structural deficiency, with the time spent in attempting to operate the

computers detracting from learning time.

Another central theme among the interviewees was a positive change in their

attitudes towards ICT in general because of the ICDL training. The positive change

appeared to be related to increased confidence in the ability to use various software

applications. All the interviewees noted improvement in confidence and skills in

some ICT areas, although most also noted lack of confidence with software such as

Access or Excel that they found particularly difficult. The interviewees further noted

the limitations of the ICDL training for understanding the full range of application of

the software. While the course taught the fundamental skills necessary to use the

software, many of the interviewees remained uncertain about appropriate applications

of the software.

The interviewees indicated that they would be using some of the software

covered in the ICDL training in classroom applications. Power point for presentations

was the most commonly cited classroom use of the knowledge obtained during the

course. One interviewee noted, ‘I am no longer afraid of using PowerPoint

presentations in the classroom.’ Several of the interviewees also indicated that they

would be using spreadsheets to maintain grades and other office applications for

designing lessons. These uses of ICT, however, involved presentation and

administrative tasks, and did not involve fully incorporating ICT into the tasks or

activities required of students. The interviewees noted that insufficient ICT

equipment in the classroom was a structural barrier to using ICT applications in

teaching.

70

Chapter 5: Discussion

ICDL training was evaluated in this study based on Kirkpatrick’s Model of

Training Evaluation, as described in Wu et al. (2004). The following four factors from

the model were used in the evaluation: 1)training outcome with regard to participants’

reaction to the training, 2) participants’ content learning, 3) participants’ behaviour

changes after training, and 4) as best as could be determined how the ICDL training

might impact overall teaching effectiveness related to improved use of ICT in the

classroom.

The quantitative component of the study indicated that the respondents to the

survey questionnaire had only very slight agreement with the statement that ICDL

training is effective for improving ICT competencies. In addition, 4.1.4percentage

reported no change in their ICT competencies because of the ICDL training. The data

obtained from the interviews with the students in the Access class appears to contradict

the findings from the survey questionnaire by indicating that the interviewees believed

that the ICDL training had improved their ICT skills. The quantitative evidence from the

pre-and-post test of the Access class also indicates that the participants experienced

substantial improvement in their ICT skills at least with respect to Access. Even the

individuals failing to pass the test for ICDL certification in Access had significantly

higher test scores after the ICDL training, which is an objective measure of improvement

in ICT competency. The discrepancy between the findings from the survey questionnaire

and the findings from the examination of the Access class may be because of the small

size of the sample for the Access class. The particular individuals included in this

component of the study may have experienced a benefit from ICDL training for

71

improving ICT competencies greater than the general experiences of a larger population.

These results provide a contradictory answer to the research question that asks which ICT

competencies were improved and which were not improved through the ICDL training,

Empirical evidence suggests that the application competency is improved because of the

training. However, the participants’ reaction to the training reflects an overall view that

they did not improve in ICT competencies with the exception of a specific mention of

improved use of PowerPoint. This might reflect a contradiction in how participants were

interpreting ICT competency. . If we refer to Easton, Easton, and Addo (2006), computer

literacy reflects an understanding of the basic function of a computer as well as an

understanding of the use of some software applications. On the other hand, information

literacy reflects the ability to know when information is needed and to have the ability to

locate, analyze, and effectively apply the information to a specific circumstance. The

range of participant interpretation of ICT competency could vary widely across the

spectrum between computer literacy and information literacy.

Similarly, the quantitative component of the study indicated that the respondents

to the survey questionnaire were neutral towards the statement that the ICDL training

provided them with skills they would transfer to the classroom. In the interviews in the

qualitative component of the study, however, the teachers (participants) in the Access

class indicated that they would use some skills in the classroom, and particularly

PowerPoint. The respondents to the survey questionnaires also indicated that PowerPoint

and other presentation software was the element of ICDL training that they would most

likely use in the classroom. . The interviewees in the qualitative component of the

findings noted that the ICDL training improved their confidence, although they remained

72

somewhat uncertain about the specific uses of ICT in the classroom beyond assistance

with presentation and administrative tasks. Although these findings are contradictory,

they nonetheless suggest that in answering the research question regarding how ICDL

training may have influenced the attitude of teachers towards incorporating ICT into

classroom teaching, within the limited scope of presentation and administrative tasks,

attitudes were improved.

The quantitative findings of the study provide evidence that the number of years

teaching and the gender of the teacher do not moderate the effectiveness of the ICDL

training. The subject taught, however, does have some influence on the effectiveness of

the ICDL training. Teachers of science, physical education, and mathematics had

substantially lower levels of agreement with various areas examined in part 2 of the

survey questionnaire. The outcome of the ICDL training for the Access class under

observation also had the highest certification failure rate among teachers of Arabic,

followed by teachers of Islamic studies. These findings suggest that teachers of certain

non-technical subjects may have a lower ICT base knowledge when taking an ICDL

course, which may contribute to a slower pace of learning the material.

Additionally, the differences in the responses to the various questions based on

subject taught may be because of the standardisation of the ICDL training and the failure

to segregate teachers taking the course based on abilities. The teachers of certain subjects

such as physical education may not have had previous training in ICT and enter the ICDL

course as a beginner. These students would require more materials, support, and attention

during the ICDL training. At the other extreme are teachers in subjects such as science

and mathematics that are likely to have substantial ICT knowledge prior to taking the

73

ICDL course. For these individuals, a standardised course may be too fundamental. .In

answering the research question of whether differences in gender, years teaching, and

subject taught would moderate ICDL effectiveness in teaching ICT competency the

answer is yes, but only in regard to subject taught. The issue of differences in the

effectiveness of the ICDL training among teachers of different subjects could be

addressed by segregating teachers into different ICDL classes based on their abilities.

The findings also identified many shortcomings in the ICDL training that might

be improved to increase effectiveness of training for teachers. The suggested

improvements might be considered as alternatives to the current form of ICDL training.

The trainers do not receive specialised instruction concerning the methods for ICDL

training, which might contribute to substantial variability in the manner of ICDL

classroom instruction. At the same time, however, the trainers indicated that they use

direct instruction methods, which conformed to the preferred learning style of the

respondents to the survey questionnaire. The variability in the instructional methods and

approaches among the trainers could account for the finding in the survey questionnaire

that one-on-one instruction and sufficient handouts were lacking form the ICDL training.

The observation of the Access class indicated that the trainer became involved in side

discussions with individual students reducing the amount of time the trainer was able to

devote to instruction or answering general questions applicable to all students.

The duration and the content of the ICDL training were also identified as

significant problems. According to participant responses, the time allotted to each ICDL

training course was insufficient to cover the material, and particularly affected the

modules with more complex content such as Excel or Access. In addition, holding

74

training in the evening created difficulties for some teachers because they were tired at

the end of the workday. Supporting Reffell and Whitworth (2002) trainers indicate that

this ICT teaching focuses on the skills necessary to operate the applications rather than

the skills necessary to evaluate the effectiveness and relevance of ICT output. Trainers

also identified the fact that the ICDL training used standardised content focused solely on

the requirements to pass the ICDL test, and they identified this as an obstacle to

understanding the full range of application use. The teacher participants were often

unable to identify practical classroom applications for much of the ICDL training; At

most, many of the teachers participating in the study indicated they would use

PowerPoint for presentations. The lack of applicability of training to the participant’s

specific educational context may have accounted for the high percentage of respondents

indicating that the ICDL training had not influenced their use of ICT in the classroom.

Smeets and Mooij (2001), show that ICT instruction is most when the curriculum and

rate of learning is designed to meet the specific needs of the student. Both quantitative

and qualitative analysis in this study provides an answer to the final research question

that alternative training methods might better meet the ICT learning needs for teachers

These findings suggest that alternative approaches to teaching ICT competencies

to teachers be considered. One alternative may be to modify the content of the modules to

increase applicability to the professional educational environment. Modifications could

include shifting the focus of the training away from passing the ICDL test and toward

classroom applications of ICT. Modifications could also include the use of trainers with

prior experience in using ICT applications in the classroom. The findings might also

75

suggest that a different training course, in addition to ICDL, could provide training

specific to use of ICT in the classroom.

Becta, (2004), supports the lack of ICT equipment and improper equipment

maintenance identified in this study. The factor of little or poorly maintained equipment

is relevant to the effectiveness of ICDL training as well as to the subsequent use of ICT

in the classroom. The observation of the Access classroom as well as the interviews with

the teachers attending the class indicated that improper equipment maintenance reduced

the ability of all students to use a separate computer. It also caused delays in teaching

while the trainer and the students jointly attempted to ensure that the software could be

used in the computers, many of which had been infected by viruses. In addition to the

specific problems in the ICDL training environment, the respondents to the

questionnaires frequently indicated that insufficient ICT equipment and maintenance was

a barrier to incorporating ICT into classroom teaching.

It appears that the general structure of the ICDL course being geared toward

passing a certification exam sets expectations for both trainers and students as to what the

purpose of the training is. Trainers perceive students as being there to learn how to pass

the test, and students see the training as being designed as a standardized process to

facilitate passing certification tests. This expectation at both ends inhibits the ability of

the training to extend outside of these boundaries.

The primary research question for this study asked whether ICDL provided the

best approach for Oman with respect to training its teachers to use technology effectively

and whether the ICDL training is effective for providing teachers with the ICT skills

necessary in the academic and educational environment. Overall, the study suggests that

76

while ICDL is effective in helping teachers receive certification, the training is not

effective in expanding teacher awareness of how to use ICT in the classroom to improve

learning. This finding supports Perez and Murray (2006) who adopted the position that

the ECDL/ICDL certification should be viewed as an assessment tool to determine if

individuals have the basic skills to operate rudimentary computer programs, but not as an

assessment of ICT literacy. With the exception of increased confidence and

understanding of how to use PowerPoint in the classroom, all aspects of the study show

that ICDL training is limited in the scope of what it teaches.

77

Chapter 6: Conclusion

The rapid growth of technology and Internet use demands that educators improve

ICT competency to improve the use of software applications and ICT technology for

presentation, administration, and enhanced teaching effectiveness. Additionally, the

advanced level of technology use by students suggests that effective teaching might need

to include expanded use of technology in the classroom in order to engage students and

hold their attention. ICDL training for ICT competency was designed as a course to help

students pass certification training. For this purpose, standardization and a narrow focus

were important in the design of the training. However, what educators need in terms of

ICT training includes the expanded ability to make critical assessments of ICT and to

understand how to effectively apply ICT to enhance the educational experience for

students.

This study shows contradictory results with regard to a participant’s belief that

they have improved ICT competency and what empirical test scores show about

improved understanding of the applications that were studied. Definitions play a role here

and the varied interpretations of how ICT competency or literacy is interpreted across

participants probably play a role in explaining the contradictions. It might be important to

educate teachers first on the differences in computer literacy, ICT literacy and

competency, and information literacy. The definitions vary even in the literature review.

There might be value in the education industry creating definitions for each term that are

specifically relevant to educators. The fact that most participants in this study were

neutral as to whether they improved ICT competency in the ICDL course, reflects a lack

78

of clarity in understanding the term. Neutral responses often emerge when there is

ambiguity.

In addition to potential differences in how ICT competency is being interpreted,

there are also variations in ICT competency and ICDL success across educators who

teach different subjects. The study suggests that those who teach in completely non-

technical areas like languages, humanities, or cultural areas have lower starting ICT

competency, and do not appear to learn the required competencies taught for certification

in the ICDL training. It is unknown what contributed to the obstacle in learning for some

students, but it does suggest that ICDL training is not effective for students starting at the

lowest end of the ICT competency spectrum and supports the need for a more

rudimentary beginner’s course that precedes the certification course. Perhaps with the

addition of this rudimentary course, an advanced course could also be developed. ICDL

training could be seen as one phase in the overall process of a more complete ICT

literacy-training program.

Training needs to have clearly stated objectives, and it is clear that the objective

of the ICDL training is to prepare students to pass a certification test in specific software

applications. Use of the ICDL for purposes beyond this is unlikely to be successful. The

question arises then, as to whether ICDL training should be altered, or whether there are

other alternative methods to train educators to apply ICT in their classrooms. The training

of critical assessment and application of ICT to a variety of scenarios requires critical

thinking and problem solving training that is not likely to be well supported in the

standardized format of the ICDL course. Additionally, to add this element to the ICDL

course, takes away from the objective of a narrow focus on passing certification tests.

79

Although students had the opportunity to suggest web-based tutorials or self-paced

training, the preference appeared to be or instructor led training and this should be

considered when moving forward to a more comprehensive training plan. Students did

indicate a willingness or interest in doing homework for the course, which reflects some

interest in self-directed work while still under the supervision of a face-to-face trainer.

Participants in the study demonstrated improved confidence for the application

they learned in the ICDL training. However, it might be that the increased confidence is

not taken advantage of because there is no further training to show educators how to

apply ICT within the classroom. Additionally, the study identified lack of equipment and

poorly maintained hardware as a factor that impedes learning during training. The

importance of hardware and software problems might be underestimated as a factor that

impedes both learning and application of the learning. Difficulty with equipment can

reduce a new learner’s confidence and create a fear of trying to apply ICT in the

classroom.

Improving ICT competency for educators appears to require training beyond the

ICDL. Better definition of ICT competency within the educational field is necessary so

that learning objectives can be defined, and from those objectives, effective training can

be developed.

80

Chapter 7: Recommendations

The study results also allow the recommendation of a number of improvements

for ICT competency training for educators. In the short term, some improvements to the

exiting ICDL training can be recommended.

7.1. Recommendations for Improvement to Training

7.1.1. Make Training Relevant to Educators

Providing the ICDL training with a context of using the application in a classroom

setting would add to the value of the training for educators while still accomplishing the

objective of training for ICDL certification. Assignments, activities, and exercises could

be designed to reflect how an application might be used in a teaching environment.

All students have the right to be taught by teachers with the same competence, for which

standards need to be set. Nowadays, competence not only means subject competence, but

also includes the other skills required for innovative education.

7.1.2. Training Length

While the students feel that the training should be longer, the addition of

homework assignments might add time to the use of the application while still preserving

the allotted time for the existing training. The homework could serve the purpose of

additional use of the application and apply relevant scenarios for educators as homework

assignments. Additionally, trainers need to optimize the classroom training time by

avoiding sidebar conversations and by bringing all questions and issues to the entire class

to expand the learning.

81

7.1.3. Training Time

Normally teachers in Oman take their professional development courses within

the working days. The ICDL training was an exception. The teachers took it after they

finish a full working day, which caused lack of concentration or carelessness. It is

recommended that it be given within the working day as any other professional

development course. The teachers can be released either for a full day or for some time

within the working day.

7.1.4. Training Materials

The students clearly state a desire for and need for additional materials in the form

of handouts and books. To reduce the expense of paper materials, online materials might

be a solution. The students could be provided links to supplemental tutorials, books, and

articles as well as suggestions or ways to continue their learning when the ICDL training

is over. Although the scope of the ICDL training does not include training on how to

apply ICT in the classroom, material in this area could be provided as Web-based content

for future reference.

7.1.5. Trainer’s Training

It is obvious from the interviews with trainers and trainees that the trainers did not

receive any adequate and specific training to teach the ICDL other than passing the ICDL

test, so it is recommended that the trainers should receive adequate training to train ICDL

to others.

7.1.6. Phased Approach to ICT Training

The ICDL should be considered one phase towards ICT literacy in Omani schools.

Teachers completing this programme should be enrolled in more general courses first or

more specific courses aiming at introducing the computer into the classroom.

82

In the long term, the results of this study suggest a phased approach to ICT competency

training with stages for the novice, the beginner, the intermediate, and the advanced

student. The end goal of this phased training would be to enable and empower teachers to

use ICT skills in the classroom to enhance and improve student learning.

7.2. Recommendations for Future Research

Several areas for future research also emerge from the results of this study.

7.2.1. ICT Competency Differences for Teachers in Different Subject Areas

The somewhat unexpected variation in ICT competency and success of learning

between educators in different subject areas suggests a need to understand the causes of

this variation. Understanding the applications of ICT in non-technical courses is

important and understanding the motivation of teachers in non-technical courses to learn

ICT would be important areas for research. Because the participants who failed the post-

test also demonstrated failing scores for the pre-test scores, it suggests a need to further

analyze the specific areas of ICT competency that are inadequate to determine what

needs to be taught in a beginner’s course.

7.2.2. The effect of lack of and poorly maintained equipment

The influence of poorly maintained equipment and lack of equipment on training

success is a significant area for study. The influence of equipment problems for the

students at the low end of ICT competency might be a relevant factor in their success in

that these students are less able to understand or adapt to these types of problems.

Additionally, studies related to the effect of outdated, inadequate hardware and software

on teacher use of ICT in the classroom could reveal interesting results. The focus of most

ICT competency research and research in how ICT is applied in the classroom focuses on

83

the teacher’s knowledge of ICT and the quality of ICT training. Understanding the

influence of hardware and software problems on the willingness of teachers to use ICT in

the classroom might be revealing.

84

Chapter 8: References

Aaker, A., Kumar, D., & Day, G. (1995). Marketing research. New York: John Wiley &

Sons.

Anastas, J.W. (1999). Research design for social work and human services. New York:

Columbia University Press.

Becta. (2004). What the research says about the barriers to the use of ICT in teaching.

Available at:

http://www.mmiweb.org.uk/publications/ict/Research_ClassroomOrg.pdf.

[Accessed 24 July 2009].

Bernard, H.R. (2000). Social research methods. Thousand Oaks CA: Sage Publications.

Berry, W. & Sanders, M. (2000). Understanding multivariate research. Boulder CO:

Westview Press.

Black, T. (1999). Doing quantitative research in the social sciences. Thousand Oaks CA:

Sage Publications.

Colosi, L. & Dunifon, R. (2006). What’s the difference? Post then Pre? & Pre then Post?

Cornell Cooperative Extension.

Calzarossa, M., Ciancarini, P., Maresca, P., Mich, L., & Scarabottolo, N. (2005). The

ECDL programme in Italian universities. Computers & Education, 49(2), 514-

529.

Costa, F. & Peralta, H. (2006). Primary teachers’ competence and confidence regarding

the use of ICT. In Pearson, E. & Bohman, P. (Eds.), Proceedings of world

conference on educational multimedia, hypermedia and telecommunications 2006

(pp. 46-52). Chesapeake, VA: AACE

85

Creswell, J. W. (2003). Research Design: Qualitative, quantitative, and mixed methods

approaches. Thousand Oaks, CA: Sage Publications.

Csapo, N. (2002). Certification of computer literacy. T.H.E. Journal, 30(1), 46-53.

Dillon, P. & Tearle, P. (1999). Ensuring effectiveness of ICT training. Telematics Centre,

School of Education, University of Exeter.

Dixie, C.H. & Wesson, J. (2001). Introductory IT at the tertiary level—is ICDL the

answer? Proceedings of SAICSIT Annual Research Conference. Available at:

http://osprey.unisa.ac.za/saicsit2001/Electronic/paper36.PDF. [Accessed July 7,

2009].

Driscoll, D., Appiah-Yeboah, A., Salibz, P., & Rupert, D. (2007). Merging qualitative

and quantitative data in mixed methods research: How to and why not. Ecological

and Environmental Anthropology, 3(1), 19-28..

Easton, A. Easton, G., & Addo, T. (2006). But I am computer literature: I passed the test.

Journal of College and Teaching, 3(2), 39-44.

ECDL/ICDL digital literacy for education. (2008). Available at:

http://www.pliroforiki.org/joomla/index.php?

option=com_docman&task=doc_download&gid=34&Itemid=57. [Accessed 25

July 2009].

European Computer Driving License Foundation [ECDL]. (2005). Developing

courseware for ECDL/ICDL.

Fallon, H. (n.d.). The use of the European Computer Driving Licence (ECDL) as an IT

training package in Irish universities. Available at:

http://eprints.nuim.ie/11/2/sconul.pdf. [Accessed 24 July 2009].

86

Garnett, F. (2008): Six ICT literacies, working paper.

Gliner, J. & Morgan, G. (2000). Research methods in applied settings. Mahwah NJ:

Lawrence Erlbaum Associates.

Gorard, S. & Taylor, C. (2004). Combining methods in educational and social research.

Maidenhead UK: Open University Press.

Gordon, N. (2007). Experiences of embedding information technology into discipline

based teaching. Italics, 14 (January). Available at:

http://www.ics.heacademy.ac.uk/italics/Vol4-1/gordon.pdf. [Accessed 25 July

2009].

Hamtini, T. (2008). Evaluating e-learning programs. Journal of Computer Science, 4(8),

693-698.

Jung, I. (2005). ICT-pedagogy integration in teacher training: Application cases

worldwide. Education Technology & Society, 8(2), 94-101.

Kennewell, S. (2002). The nature of ICT as a subject. In Kennewell, S. (Ed.) Learning to

teach ICT in the secondary schools (pp. 18-36). London: RoutledgeFalmer.

Kiridis, A., Drossos, V., & Tsakiridou, H. (2006). Teachers facing information and

communication technology. Journal of Technology and Teacher Education,

14(1), 75-93.

Kriksciuniene, D., Sarkiunaite, I. & Sakalauskas, V. (2006). Evaluating return on

computer literacy of the enterprise with ICDL. Innovations in Information

Technology, 1-5.

Lau B. & Sim, C. (2008). Exploring the extent of ICT adoption among secondary school

87

teachers in Malaysia. International Journal of Computing and ICT Research,

2(2), 19-36.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage

Publications.

Lodico, M., Spaulding, D., & Voegtle, K. (2006). Methods in educational research. New

York: Wiley.

May, T. (2001). Social research issues, methods and processes. Philadelphia: Open

University Press.

Minichiello, V., Aroni, R., Timewell, E., & Alexander. L. (1995). In-depth interviewing:

Principles, techniques, analysis. Melbourne: Longman Cheshire.

Ministry of Education, Sultanate of Oman. (2004). Ministry of Education integrates the

ICDL programs. (September 24) Press Release.

Ministry of Education, Sultanate of Oman. (n.d.). ICT and Education in the Sultanate of

Oman. Available at:

http://www.moe.gov.om/portal/SiteBuilder/Sites/EPS/Arabic/MOE/Newest/ICT

%20and%20education.pdf. [Accessed 24 July 2009].

Mioduser, D., Nachmias, R., Tubin, D., & Forkosh-Baruch, A. (2003). Analysis schema

for the study of domains and levels of pedagogical innovation in schools using

ICT. Education and information technologies, 8(1), 23-36.

Oxford Business Group. (2009). The report Oman 2009. Oxford: Oxford Business

Group.

Patton, M. (1990). Qualitative evaluation and research methods (2nd Edition). Newbury

Park: Sage Publications.

88

Pedhazur, L. & Schmelkin, L. (1991). Measurement, design and analysis: An integrated

approach. Mahwah NJ: Lawrence Erlbaum Associates.

Perez, J. & Murray, M. (2006). Journey to the centre of the core: Computers and the

internet in the core curriculum. Issues in Informing Science and Information

Technology, 3, 489-497.

Potter, W.J. (1996). An analysis of thinking and research about qualitative methods.

Mahwah NJ: Lawrence Erlbaum Associates.

Poulter, A. & McMenamy, D. (2003). Beyond ECDL: Basic and Advanced ICT Skills for

the New Librarian. Paper presented at 4th Annual LTSN-ICS Conference, Galway,

pp. 200-202.

Preston, C., Danby, M., Wegerif, R. (2005). ECDL for Educators. BCS/Aston Swann.

Randall, M. & Zirkle, C. (2005). Information technology student-based certification in

formal education settings: Who benefits and what is needed. Journal of

Information Technology Education, 4, 287-306.

Reffell, P. & Whitworth, A. (2002). Information fluency: Critically examining IT

education. New Library World, 103(1182-1183), 427-435.

Rockman, I. (2004). Introduction: The importance of information literacy. Available at:

http://media.wiley.com/product_data/excerpt/78/07879652/0787965278.pdf.

[Accessed 24 July 2009].

Robbins, R. & Zhou, Z. (2007). A comparison of two computer literacy testing

approaches. Issues in information systems, 8(1), 185-191.

Schutt, R. (2008). Investigating the social world. Thousand Oaks CA: Pine Forge Press.

Tashakkori, A. & Teddie, C. (2008). Foundations of mixed methods research. Thousand

89

Oaks CA: Sage Publications.

Smeets, E. & Mooij, T. (2001). Pupil-centred, learning, ICT, and teacher behaviour:

observations in educational practice. British Journal of Educational Technology,

32(4), 403-417.

Stanley, N. & Tanner, H. (2002) Assessing attainment. In Kennewell, S. (Ed.)

Learning to teach ICT in the secondary schools (pp. 109-129). London:

RoutledgeFalmer.

Tanner, H. (2002). The place of ICT in secondary education. In Kennewell, S. (Ed.)

Learning to teach ICT in the secondary schools (pp. 3-17). London:

RoutledgeFalmer.

Taylor, S. & Bogdan, R. (1998). Introduction to qualitative research methods. New

York: John Wiley.

Trochim, W. (2006). Research methods knowledge base: Descriptive statistics. Retrieved

on July 1, 2009 from http://www.socialresearchmethods.net/kb/statdesc.php

Waxman H. &.Huang, S..(1999).. Classroom Observation Research and the Improvement

of Teaching. InNew Directions for Teaching Practice and Research, Waxman, H

& Walberg, H. (Eds). Berkeley, CA: McCutchan.

Wengraf, T. (2001). Qualitative research interviewing. London: Sage.

Waxman, H. & Padron, Y. (2004). The uses of classroom observation schedule to

improve classroom instruction. In Waxman, C., Tharp, R. & Hilberg, S. (Eds.)

Observational research in U.S. classrooms. (pp. 72-87). Cambridge: Cambridge

University Press.

Whitworth, A. (2005). The state of ICT skills education. Italics, 14(January). Available

90

at: http://www.ics.heacademy.ac.uk/italics/Vol4-1/ITALICS_editorial.pdf.

[Accessed 24 July 2009].

Whitworth, A. (2009). Computer and information literacy. In A. Whitworth.

Information Obesity (pp. 75-105). Oxford: Chandos Publishing.

Wragg, E. (1999). An introduction to classroom observation. London: Routledge.

Wu, C., Chen, Y., Lee, G., Ho, R. & Chiou, G. (2004). Evaluation of an inservice teacher

training program ICT Integration. In Cantoni, L. & McLoughlin, C. (Eds.),

Proceedings of World Conference on Educational Multimedia, Hypermedia and

Telecommunications 2004 (pp. 3451-3455). Chesapeake, VA: AACE

Young, J. (2004). Testing service to unveil an assessment of computer and information

literacy. Chronicle of Higher Education, (November 12), 33.

Zakopoulos, V. (2005). An evaluation of the quality of ICT teaching within an ICT-rich

environment: The case of two primary schools. Education and Information

Technologies, 10(4), 323-340.

91

Appendix ASurvey Questionnaire

Gender: Subject Taught: Level Taught: Years of Experience:

Part 1:

Motivation and Expectations

1. What was your motivation for taking the ICDL course?

a) Professional developmentb) Employment requirementc) Personal interestd) Not sure

2. Rank the skills that you hoped to improve through the ICDL training (1 will be the highest need)

o _____No needso _____Word skillso _____Access skillso _____Microsoft application skillso _____Improved computer skillso _____Improved Internet skills

_Others. Please specify: ___________________________

ICT Competency

3. How would you rate your level of ICT competency before starting the ICDL training?

1 2 3 4 5 Incompetent Slightly Neither Competent Highly Competent Competent

4. How would you rate your current level of ICT competency after finishing the ICDL course?

1 2 3 4 5 Incompetent Slightly Neither Competent Highly Competent Competent

92

Use of ICT in the classroom

5. Some of the reasons that I have not used technology in the classroom include: My own lack of technology skills My students’ lack of technology skills I do not think that technology enhances learning I am not sure which applications to use for specific purposes I don’t think my students like me to use technology Others: Please Specify

6. What specific skills gained in ICDL training are you most likely to transfer to the classroom setting?

_____Basic concepts of IT _____Using the computer and managing files _____Word processing _____Spreadsheets _____Database _____Presentation (PowerPoint)_____Information and Communication

7 Did the ICDL training result in your use of new software applications in the classroom?a. yes b. no

If yes, which applications? List them below: __________________________________If no, why not?___________________________________

ICDL Training Evaluation

8. Rate the modules from 1 to 7 according to which ones were the most useful for you. Start with 1 as the most useful and 7 as the least useful. _____ Basic concepts of IT ____ Using the computer and managing files ____ Word processing ____ Spreadsheets ____ Database ____ Presentation ____ Information and Communication

93

9. What learning support from the trainer do you think is missing in the course? (select as many as are applicable)

o One-on-one timeo Discussion timeo Be able to ask questionso Homeworko Practice time in class

Others: Please specify: _____________________________

10. What learning support from the materials do you think is missing in the course? (as many as applicable)

o CD contento Paper Handoutso Online sourceso Web tutorials

Others: Please specify: _____________________________

Favorite learning style:11. What is your favorite method of learning ICT?a. direct trainer's instructionb. Learning from peersc. self studyd. Other. Please specify: ______________________

Part 2: The following section asks you to rate your level of agreement with statements about ICT and ICDL training. A rating of 1 indicates you strongly disagree with the statement while a rating of 5 indicates that you strongly agree with the statement. Please circle the number that corresponds with your level of agreement.A Strongly

DisagreeDisagree Neither

Agree nor Disagree

Agree Strongly Agree

ICT Competencies1. ICDL training is effective for improving my ICT competencies.

1 2 3 4 5

2. The ICDL training provided me with skills that I will transfer to the classroom.

1 2 3 4 5

3. The ICDL training provided adequate time to thoroughly learn the information about the software applications

1 2 3 4 5

ICT Use in Classroom4. The ICDL training taught me to use 1 2 3 4 5

94

various ICT software applications to solve future education or research problems5. The ICDL training increased my understanding of the importance of ICT in the educational system

1 2 3 4 5

6. The ICDL training increased my understanding of the way in which ICT can be integrated into classroom instruction

1 2 3 4 5

7. After completing the ICDL training and certification process, I feel more confident integrating ICT into my classroom teaching

1 2 3 4 5

ICDL effectiveness for Certification 8. The ICDL certification is necessary for career development

1 2 3 4 5

9. The ICDL training taught me the full range of capabilities of the software applications covered by the ICDL certification.

1 2 3 4 5

10. The ICDL training provided all the books, handouts or online materials necessary for learning to use the applications required for certification

1 2 3 4 5

ICDL Training Evaluation11.The ICDL trainer adapted to my needs and addressed any obstacles I experienced during the class.

1 2 3 4 5

12. The ICDL trainer provided adequate guidance and assistance to learn the applications required for certification

1 2 3 4 5

13.The ICDL trainer used teaching methods that were helpful to me.

1 2 3 4 5

14. In general, the ICDL training was useful for me as a teacher

1 2 3 4 5

Future learning opportunities15. The ICDL training provides a strong foundation for future ICT education and learning

1 2 3 4 5

95

Part 3:1. Do you think that the ICDL training course is suitable for you as a teacher? Why?

2. What do you thing the course is missing?

3. How can the course be improved to be more effective?

96

Appendix BStructured Observation Form

Structured Observation Form

ExpectationsThe trainer introduced the purpose of the training to be_______________________________The students could be categorized into the following ICT knowledge level at the beginning of the course:

Beginner Intermediate Proficient# of students # of students # of studentsFrom what you observed in the class, the student’s purpose in being in class could be categorized as:

Certification Professional Development Personal Development# of students # of students # of studentsMaterialsThe materials provided in class included:CDs with software applications

CDs with training materials

Paper Handouts

Videos Tutorials

Each training module included the following:Training Material Comment

Did the students have questions about the materials?Questions about Materials Response from Trainer

Did the students request or express a desire for different or additional materials and what were they?________________________________________________________________________

97

TrainerThe trainer’s credentials and qualifications were:__________________________________________The trainer used the following teaching methods during the course:Lecture Interactive

DiscussionDemonstration Class

ActivitiesVideo Tutorial

Comments______________________________________________________________________________________________________________________________________The trainer’s primary focus during the training appeared to be:

Topic Trainer covered topic thoroughly

Trainer covered topic minimally

Trainer did not cover this topic

Certification RequirementsUse of ICT in the classroomUse of ICT tools to improve learner’s productivityCritical assessment of ICT toolsSecurity basics related to ICT toolsOther:

Comments______________________________________________________________________________________________________________________________________During training, the teacher did the following:

Often Sometimes NeverWelcomed questions and provided adequate responsesEngaged students in discussion about the topicProvided opportunities for students to engage and talk to or work with each otherAdapted to student’s needs and addressed obstaclesEffectively managed technical issues that arose

98

Comments:______________________________________________________________________________________________________________________________________Technical IssuesThe following technical issues arose during the course:Technical Issue Resolution to the problem Impact on student

learning

Comments:______________________________________________________________________________________________________________________________________Future Learning OpportunitiesDid the instructor inform students of future learning opportunities in ICT?___________________________________________________________________________________________________________________________________________

Did the students inquire about future learning opportunities in ICT

Future learning opportunities discussed by trainer or student during the class included:Advanced ICDL courses

Web-based training

Book/paper materials

Vendor Training

Did the students offer information about specific future learning interests?___________________________________________________________________________________________________________Unexpected ObservationsIn addition to the above observations, I observed the following

Observation Comment

Appendix C

99

Teachers' Interviews Questions

1. What did you expect from the ICDL training prior to the ICDL course?

2. A. What went well in the training?

B. What went badly? 3. Which module was

A. the most useful module and why?

B. the least useful module and why?

4. How did you learn? A. How did you interact with the tutor?

B. How did you interact with other learners?

C. Which teaching or learning materials did the tutor use and which did you use?

D. Which teaching methods did the trainer use?

5. Do you think that the training helped you learn ICT independently?

6. Has the training changed your attitudes toward the use of technology and if so, how?

7. What are the areas of weakness in the training and how can they be improved?

8. Has the training changed your confidence in using ICT in the class and if so, how?

9. Has the training helped you to use the skills you learned in unpredictable situations in the future?

10. How are you going to use the skills you learned in the future (in the classroom and life)?

Appendix D

100

Trainer Interview Questions

1. Which kind of training did you obtained before teaching ICDL?

2. What about ICDL itself, did you receive specific training for teaching the ICDL?

3. Which teaching methods do you use in teaching the ICDL?

4. What teaching materials do you use in teaching the ICDL?

5. Do you think the learners use the learning materials and resources as they should be used?

6. Which kind of difficulties do learners face and how do you help them in overcoming those difficulties?

6. Do you give the learners anything to study at home?

7. Do you find it a disadvantage to depend on the software test for assessing your students and if so, why?

8. Do you think that obtaining the ICDL certificate means that the learner is skilled in ICT and if so, why?

9. Did you receive any kind of support from the Ministry of Education or the ICDL organisation to help you in teaching the course?

10. How can the ICDL training course be improved?

Appendix E

101

Affirmation Letter from Participants

I affirm that I did not practice for either the pre- or post-test for the Access ICDL course before the tests were administered by the researcher.

Learner ID: ____________________________________

Signature: ____________________________________

102