___________________________________________________________________________

Implementation of Blackboard 9.1 (learn) on the Police Academy of the

Netherlands

Reinder Vrielink, MSc

Police Academy of the Netherlands

July 22nd, 2016

_________________________________________________________________________

Abstract

This research covers appreciation of the use of Blackboard among students and teachers on

the Police Academy of the Netherlands from 2011 up to and including 2015 with the focus on

the educational implementation of Blackboard 9.1(learn). The Technology Acceptance Model

(TAM) was used as a research method. The results of this research show due to the

educational implementation of Blackboard 9.1, appreciation in the use of Blackboard by

teachers has increased. Appreciation among students has decreased, possibly due to faltering

technology. The results furthermore demonstrate that implementation of ICT in education is

difficult and complex and arriving at a programme approach is therefore recommended.

Highlights

• Due to the educational implementation of Blackboard 9.1(learn), appreciation in the

use of Blackboard by teachers has increased.

• Appreciation among students has decreased, possibly due to faltering technology.

• Usefulness is the strongest predictor of the intention to start using Blackboard, and

enjoyment has a great impact on this as well. Enjoyment and Ease of use positively

effects behavioural intention through usefulness.

• Implementation of ICT in education is difficult and complex and arriving at a

programme approach is therefore recommended.

Keywords: Blackboard; behavioural intention; enjoyment; implementation; usefulness

___________________________________________________________________________ Corresponding author: email: reinder.vrielink@gmail.com:

1

1. Introduction

1.1. The use of Blackboard on the Police Academy of the Netherlands/Previous research

The Police Academy of the Netherlands offers a basic training course (initial) and a further

training course (post initial). The institute covers six schools and nine trainings locations.

The School of Policing is one of the six schools and offers middle vocational education

and training, leading on to all-round Police Officer (initial, level 4). The use of

Information and Communication Technology (ICT) plays an important role in the course.

Education and training is supported by Blackboard (since 2002).

The appreciation of the use of Blackboard by students and teachers is not great. There is

grumbling that the use is difficult and student unfriendly. This is endorsed by previous

research (Vrielink, 2015). This previous research covers appreciation of the use of

Blackboard among students and teachers on the Police Academy of the Netherlands in the

period from 2008 up to and including 2011, using the Technology Acceptance Model

(TAM). From questioning the students and teachers, it became clear that the unclear

structure of Blackboard was a problem. Teachers scored (in 2011) significantly lower at

behavioural intention than students.

In 2013 Blackboard 9.1 (learn) was introduced. The reasons for this included: Blackboard

8 would no longer be supported; Blackboard 9.1(learn) would have a user-friendly

interface; the connection to Single Sign On would be easier, and the design and create a

better search function was simplified. The introduction comes with an implementation

plan (teaching and learning with Blackboard), in which there is specific attention for the

professionalization of management and teachers. The research was repeated at the end of

2014 and at the beginning of 2015, a year after the implementation of Blackboard

9.1(learn).

2

1.2. Review of Literature

Implementation of e-learning

There are relatively few scientific articles on the implementation of e-learning (Cross, J.

& Dublin, L., 2002; Netteland G., Wasson B., and Anders I., 2007).

Attwell (2005) claims that the restricted empirical research in the e-learning field has

mainly focused on the development of technology or product evaluation, and not on “what

works and what does not [work in a workplace environment]”. Rosenberg (2006)

underlined the importance of implementation. “It is not the technology which makes

learning challenging, but the way it is used and implemented. It’s the highway, not the

destination; the means rather than the end”.

The Four in Balance model (Kennisnet, 2008)

Ever since its creation, the Four in Balance model has proven its added value in the

implementation of ICT in Dutch educational institutions. The Four in Balance model

shows how schools can maximise the value of ICT. The model describes the preconditions

(building blocks) required to successfully use ICT in education.

Figure 1 The Four in Balance model

3

These preconditions (building blocks) include;

Vision: About the undeniable importance of leadership and vision during a change process

are many books written, e.g. by Kotter (1996). The view regarding high-quality and efficient

education and the position ICT occupies here. The vision comprises the overarching

ambitions. The direction is known. The organisation is managed on the basis of this vision.

Not only words, but also deeds. Displaying exemplary behaviour, participating in online

activities, support (resources, interventions), staying on course, and not losing sight of the top

of the mountain. Showing how important e-learning is through behaviour (Rubens, 2013).

Expertise: ICT expertise of teachers. Knowledge, skill and attitude in respect of ICT, in

pedagogical-didactic practice (Selim, 2007), working in the school context as well in

individual professional development. Are teachers capable of using technology?

According to Kirschner (2013), teachers become unsure if ICT ‘has’ to be deployed in

education.

• Teachers have no experience with ICT.

• Teachers do not have a good concept of what you can and cannot do with it.

They lack examples from their own training.

• What’s in it for me? If you have to explain why this innovation is required, this

is the deathblow to innovation.

• Teachers often had and still have insufficient facilities to use ICT in education.

• ICT is still too often regarded as an addition, instead of a normal tool to

provide education.

According to Kirschner (2013), the biggest challenge for teachers in this field is to properly

reflect on what type of education should be provided, and what learners have to learn. What

are the learning goals, how can they be achieved, and how do you test? This requires teachers

who are experts in the field of content, didactics and learning technologies. Teachers are

afraid of things they do not know and with which they have no experience. Having to explain

it is the kiss of death for educational innovation. Success factors are the necessary facilities,

with failure factors coming down to not having a clue. For example, if, in the case of

cooperative learning, the assignment is too simple, the student will prefer to do it by

themselves.

4

Content and applications: The information, educational content and software used in an

educational institution, such as:

• Digital learning material. The biggest challenge for teachers is determining

learning goals, including the didactics and technology used. A triangle between

teacher, student, and content. What content, which pedagogical approach and with

what technology. Why and how? (Hudson, 2008). (Hudson, 2008). A teacher is

excellent if they possess superb domain knowledge and understand how didactics

are to be applied. They are then capable of differentiating and providing

customised education (Kirschner, 2013).

• Educational software packages, such as an electronic learning environment,

student tracking system and student administration.

• Timetabling packages, apps, and so on.

Infrastructure: The availability and quality of hardware, networks and connectivity. The

hard ICT side, the infrastructure and systems, the design of the network. Wireless technology,

creating enough exit points, which facilitates working wireless with large groups of learners

as well. Interconnectivity between systems, so that certain data will not need to be entered

again in various systems such as an electronic learning system. Are enough suitable spaces

available in a blended situation in the event of a mixture of face-to-face and online learning?

Rubens (2013) adds the following building blocks to the Four in Balance model:

Curriculum and content development: what impact does e-learning have on the

curriculum? Will the existing curriculum be digitalised or will other learning activities be

developed now we have new technologies at our disposal?

Culture: one of the four Cs (Culture, Champions, Communication and Change) critical for an

implementation’s success in the introduction phase (Rosenberg (2001).

Does the culture focus on collaboration, on online learning together? In a hierarchical

organisation, attention will first have to be paid to cooperation, social learning and

collaborative learning.

Power relations and interests: it will have to be clear beforehand what roles there are during

the implementation. What role does ICT management have? Do we have to support existing

5

processes or do we have to adapt them first? What is the relationship between the ICT

department and the business plan? Who is the owner of the e-learning application? What is

the position of staff? Who directs who? Do staffs direct education or does education direct

staff?

Context: the implementation occurs within a certain context. What is the view in respect of

learning? Is it traditionally focused on courses and training programmes? What is the impact

of various social developments and the network society? What is the incentive to be involved

in e-learning?

1.3. Problem statement/Research question

As described in the introduction, there is little appreciation for the use of Blackboard among

students and teachers. With the introduction of Blackboard 9.1(learn) is the opportunity taken

to the focus on the educational implementation. The introduction comes with an

implementation plan (teaching and learning with Blackboard), in which there is specific

attention for the professionalization of management and teachers.

A project was designed for the introduction of Blackboard 9.1 (learn) steering group and a

project group were established. For the implementation, a leader and two key users were

appointed for every school. The aim of implementation of the latest version of Blackboard is

for application and updating of Blackboard to occur in such a way that the new educational

view is supported in the best possible way, i.e. better use of Blackboard in education with an

improved return on investment. Project teams were established for every school.

In order to realise a better application and use of Blackboard for every school as well as

Policy Academy-wide, the following results are aimed for:

1. The vision of Blackboard’s application in education being translated into objectives

and policy

2. Improved design, functionality and performance of Blackboard

3. Improved content management and functional management

4. Professionalization and ‘change’ for teachers, educational supervisors, experts and

managers

5. Guidance, organisational assurance and support: ownership, roles, responsibilities,

6

anchorage in the HR cycle and HR policy, guaranteeing continuous development.

6. Blackboard 9.1(learn) implemented according to a coordinated development plan

7. Implementation of new functions, in consultation with the schools:

a. Installed module, Blackboard Mobile

b. Installed module, Blackboard Collaborate (optional)

Research question:

Does the suggested project-based approach to the implementation of Blackboard 9.1(learn),

with the formulated results, produce more appreciation of the use of Blackboard among

students and teachers in 2014/2015 compared to the results of the comparative research from

2011 (Vrielink, 2015)?

This research, as well as previous research (Vrielink, 2015), was carried out among students

and teachers of the School of Policing (initial).

7

2. Material and methods

2.1. The Technology Acceptance Model (TAM)

In this paragraph the Technology Acceptance Model (TAM) is described, which Davis (1993)

introduced and which was modified by Yi and Yuong Hwang (2003). The TAM encompasses

a wide range of research objects, a brief list: website usage (Selim, 2003); the use of

Blackboard (Yi ad Yujong Hwang, 2003); ICT acceptance by therapist (Schaper and Pervan,

2004); online banking (Pikkarainen 2004); online shopping (Banus and Baptist, 2005); mobile

telephone (Aversano, 2005), and mobile commerce (Wu & Wang 2005). TAM has proven to

be a powerful model for explaining and predicting usage intentions and acceptance behaviour.

Legris (Legris et al, 2003) concluded that TAM can be regarded a useful model, but it has to

be integrated into a broader one which would include variables related to both human and

social change processes, and to the adoption of the innovation model. TAM does not

explicitly include any social variables. TAM provides a quick and inexpensive way to gather

general information about individuals’ perceptions of a system. TAM provides an information

representation of mechanisms by which design choices influence user acceptance, and should

therefore be helpful in applied contexts for forecasting and evaluating user acceptance of

information technology (Davis, 1993). TAM introduced two new constructs: perceived

usefulness (the belief that using an application will increase one's performance) and perceived

ease of use (the belief that one's use of an application will be free of effort). TAM theorizes

that an individual’s actual system usage is determined by behavioural intention, which is

determined by perceived usefulness and perceived ease of use.

8

Figure 2. Davis’ Technology Acceptance Model (TAM).

Recent findings on intrinsic motivation in social psychology indicate that enjoyment plays an

important role in determining a person’s behaviour. Yi and Yujong Hwang (2003)

investigated this feature (enjoyment) as external variables and their research shows that the

motivational variable enjoyment plays an important role in influencing the individual’s

decision whether or not to use a Web based technology.

According to Yi and Yujong Hwang (2003) the ease of use perceptions are influenced by the

degree to which people perceive using the system to be personally enjoyable. Agarwal and

Karahanna (2000) propose that the individual traits of playfulness and personal

innovativeness are important determinants of cognitive absorption. Performance is reached

when it joins up with enjoyment and learning in the same triangle. When there is enjoyment,

this will positively influence learning, which in its turn will lead to performance and so on.

Most of us know by experience that performance enhances when you are enjoying yourself.

In this manner, you are able to develop competencies (Gallwey, 1999).

external variables

behavioural intention

actual system

use

perceived usefulness

perceived ease of use

9

Criticism

According to Chuttur (2009) TAM has been widely criticised, despite its frequent use, leading

the original proposers to redefine it several times. Criticism of TAM as a "theory" include its

lack of falsifiability, questionable heuristic value, limited explanatory and predictive power,

triviality, and lack of any practical value. Benbasat and Barki (2007) suggest that TAM ‘has

diverted researchers’ attention away from other important research issues and has created an

illusion of progress in knowledge accumulation. Furthermore, the independent attempts by

several researchers to expand TAM in order to adapt it to the constantly changing Information

Systems (IS) environments has led to a state of theoretical chaos and confusion. In general

TAM focuses on the individual 'user' of a computer, with the concept of 'perceived

usefulness', with extension to bring in more and more factors to explain how a user 'perceives'

'usefulness'. TAM ignores the essentially social processes of IS development and

implementation, without question where more technology is actually better, and the social

consequences of IS use.

Despite the criticm above, TAM was chosen in this research because it is used worldwide.

TAM gives easy and quick results, partly because there is only one external variable chosen

(enjoyment) in this research. It was decided to use only one variable because this way the

questionnaire remains short. Experience from previous research shows that a longer

questionnaire consisting of questions where the link with Blackboard is not clear to students is

confusing to students.

Moreover, it is about a comparative research on the appreciation of the use of Blackboard

under different conditions (e.g. improved structure). A previously carried out comparative

research showed that this method is suitable for this (Vrielink, 2006).

2.2. Questionnaire

The preparation of the Blackboard research resulted in constructing a questionnaire in Dutch

for students, the same as the examination of Vrielink in 2008 and 2011 (Vrielink, 2015) is

used, consisting of 14 questions and the opportunity to make comments. All the questionnaire

items used an 11-point Likert-type scale where 0=completely disagree, 5=neither agree nor

disagree, and 10=completely agree.

The questionnaire consisted of three questions to measure the enjoyment construct; four

questions were used to measure the ease of use construct; four questions were used to

10

measure the usefulness construct and three questions were used to measure the behavioural

intention construct.

The actual use of Blackboard was measured through the course statistics of the control panel

by counting the number of times (frequency) a student logged into Blackboard courses in the

period of 3 months.

2.3. Data analysis

The reliability of the questionnaire was measured by examining the internal consistency. ICR

= Internal Consistency Reliability (similar to Cronbach’s alpha). The correlation coefficient

(Pearson’s product-moment correlation coefficient) is measured to show the connection

between the questions.

A T-test was carried out to find if there was a significant difference between the samples. The

T-test is based on an a-select random sample survey and on the assumption that the random

sample survey distribution is normal divided.

The structural model and hypotheses were assessed by examining the significance of the ß-

coefficients and the variance accounted for by the antecedent constructs. The standardized ß

coefficients (the coefficient of the independent variables when all variables are expressed in

standardized form) were presented. Multiple regression analyses were employed to adjust for

the influence of behavioural intention on use the ß-coefficient is measured with use as

dependent variable.

All statistical analyses were performed with the SPSS version 18.0. (SPSS Inc.).

11

3. Results

3.1. Respondents

In 2011, 213 initial students from six locations (average age 24.5, 32% female) and 71

teachers (average age 45.6, s.d. = 8.8, 32% female) completed the questionnaire. In 2015, this

involved 208 students (average age 25.2, 31% female) and 36 teachers (average age 49.7, s.d.

= 7.6, 28% female) respectively.

Not all respondents answered all questions (e.g. name, age, gender), as a result of which N

may deviate from the numbers above in the tables below.

3.2. Reliability of the questionnaire

The reliability of the questionnaire is expressed in ICR (ICR=Cronbach’s alpha). All ICRs

vary between 0.70 and 0.92. Internal consistencies (comparable to Cronbach’s alpha) of 0.70

or higher are considered adequate (Barclay et al., 1995).

3.3. The correlation between the questions

With correlation are the strength and direction of the relationship between two (or more)

interval/ratio variables expressed. Pearson's Product-moment correlations are expressed in

correlation coefficient r. The value of r is always between -1 and + 1. The higher the absolute

value, the stronger the relationship. Pearson’s product-moment correlation coefficient across

the board, r, varies from 0.44 to 0.84. This may be qualified as a high interconnectedness

(Lund and Lund, 2010).

While an ICR of 0.70 or higher is considered adequate, in relation with Pearson r, it can be

concluded that the questionnaire is a good tool for this comparative research.

12

3.4. Comparative research into the scores of students and teachers in 2015 and with

the help of the independent T Test.

Table 1: Comparison of the scores from students in 2015 and 2011. It shows that students in

2015 scored significant lower on the construct behavioural intention than students in 2011

Students 2015

N = 208

Students 2011

N = 213

Construct mean s.d. mean s.d. t Sig.

(2-tailed)

Enjoyment 4.6 2.1 4.8 1.8 - 0.8 ns

Ease of use 5.8 1.8 5.7 1.7 0.3 ns

Usefulness 5.5 1.9 5.8 1.6 - 1.7 ns

Behavioural

intention

4.5 2.0 6.1 1.7 -8.5 P < 0.001

Actual use 138 106 111 81 * *

*Because of a total change in the curriculum is a meaningless comparison. The training

duration has been shortened from four to three years. Also the rhythm changed.

Table 2: Comparison of the scores between students and teachers in 2015. It shows that

teachers scored significantly higher on usefulness and behavioural intention than students.

Students 2015

N = 208

Teachers 2015

N = 36

Construct mean s.d. mean s.d. t Sig.

(2-tailed)

Enjoyment 4.6 2.1 4.9 1.6 -0.7 ns

Ease of use 5.8 1.8 5.5 1.7 1.0 ns

Usefulness 5.5 1.9 6.3 1.7 -2.7 P<0.01

Behavioural

intention

4.5 2.0 5.9 1.7 -4.2 P < 0.001

13

Table 3: Comparison of the scores between teachers in 2015 and in 2011. It shows that

teachers in 2015 scored significantly higher on the items enjoyment and behavioural intention

than teachers in 2011.

Teachers 2015

N = 36

Teachers 2011

N = 71

Construct mean s.d. mean s.d. t Sig.

(2-tailed)

Enjoyment 4.9 1.6 4.0 2.0 2.0 P<0.05

Ease of use 5.5 1.7 5.2 2.0 0.6 ns

Usefulness 6.3 1.7 5.5 2.2 1.8 ns

Behavioural

intention

5.9 1.7 4.9 2.4 2.2 p < 0.001

14

3.5. Regression analysis for testing the suggested model

Figure 4: Regression analysis for testing the suggested model

Figure 4 and table 4 show the results of the regression analysis

6

Usefulness

Behavioural Intention

Ease of Use

Enjoyment

Use

2

5

1

3

4

15

Table 4: Comparison of beta coefficients. Table 4 shows the results of the measured β

coefficients. Behavioural intention has a significant effect on Use. Usefulness has a

significant effect on Behavioural intention and Ease of use has a no significant effect on

Behavioural intention . Enjoyment has a significant effect on Usefulness and Ease of use has a

significant effect on Usefulness. Enjoyment has a significant effect on Ease of use.

2015

N = 137

2011

N = 186

1 0.19* 0.29*

2 0.73*** 0.59*

3 ns ns

4 0.35*** 0.22*

5 0.53*** 0.52**

6 0.53*** 0.58***

|ns = not significant| |* p<0.05 | | ** p<0.01| | *** p<0.001|

3.6. Comments from students in Blackboard research (2015)

Table 5: Comments from students categorised. Table 6 indicates that in 28% of cases students

complain about poor displaying of the schedule. Subjects 1 and 6 are to do with technology.

Comments from students Percentage

1 Display of the schedule is problematic 28

2 No structure, difficult to find things 23

3 Announcements are not always for one’s own group 17

4 Search function is missing 8

5 Teachers make insufficient use of Blackboard 8

6 Slow, freezes, system error 6

7 Good program 6

8 Other 4

16

4. Discussion

The aim of the comparative research was: Does the suggested project-based approach to the

implementation of Blackboard 9.1(learn), with the formulated results, produce more

appreciation of the use of Blackboard among students and teachers in 2014/2015 compared to

the results of the comparative research from 2011 (Vrielink, 2015)?

From table 1 it becomes clear that students from the 2015 population score significantly lower

at behavioural intention than students from the 2011 population. The scores in 2015 could be

attributed to faulty technology (see the comments by students, table 5). There were problems

with the Single Sign On making the applications on the eCampus, including Blackboard. The

problems are now solved.

Table 2 shows that students in 2015 intended to use Blackboard less often than the teachers in

2015! Students in 2015 scored lower on usefulness than their teachers. Teachers in 2015

intend to use Blackboard more often than the teachers from 2011 and they scored better on the

construct enjoyment too (table 3). The implementation of Blackboard 9.1 (learn) had a

positive effect on teachers and a negative effect on students. In General on the goals and

results (paragraph 1.3), they are somewhat vague and may be more specific. In the goals is

involvement of the student in the design and use of Blackboard missed. Teachers and

managers are explicitly mentioned (paragraph 1.3, result 4). Involvement of the student is not

mentioned.

Teachers give scores of 5.9 (11-point scale) for the intention to use Blackboard in 2015. This

is encouraging. Perhaps this concerns a select vanguard of teachers.

The attitude of the teacher plays an important role Albirini (2006). Whether an innovation

succeeds, strongly depends on the question if teachers are willing and able to further develop

their competencies, and if teachers are able to promote the competency development of their

teachers (Miedema and Stam, 2008). This is also in accordance with the observation by

Ertmer (Ertmer et al., 2012) that the personal convictions and attitudes of teachers regarding

the relevance of technology for students were considered to be the biggest impact on their

success.

It appears from table 4 that the construct ‘usefulness’ is the most powerful one for the

intention to use Blackboard. This is in line with the results of the research by Pei-Chen (Pei-

17

Chen et al., 2008) where perceived usefulness was also one of the critical success factors in

perceived satisfaction for students. In addition, the construct ‘enjoyment’ scores high.

Improvements in the use of Blackboard therefore need to be sought in the areas of

‘usefulness’ and ‘enjoyment’. In general, students consider Blackboard to be user-unfriendly,

as a result of which they no longer experience the usefulness and enjoyment, as a result of

which they quit. This corresponds with the findings of Timothy and Jones (2011): “It is

important to ensure that teachers acquire the necessary technical skills for the performance of

their role as a teacher, and it is important for teacher trainers to focus on the development of a

positive perception of usefulness and the extent to which technology is easy to use; both these

elements are strongly influenced by perceived enjoyment.”

It appears from table 4 that Ease of Use does not affect the intention to start using Blackboard.

Only perceived ease of use is not recognised as one of the critical factors affecting learners'

perceived satisfaction. This result is in line with the findings of the studies of online banking

(Pikkarainen 2004), and mobile commerce (Wu & Wang 2005). Perceived ease of use is less

likely to be a determinant of attitude and usage intention. According to of Donghue (2009) the

effect of ease of use on intention to use and actual use was mediated by perceived

usefulness. Ease of use is not the students’ major concern for deciding to use the e-resources

because they had some experience of using e-resources and knew how to use them. In the

study of Okantey and Addo (2016) it was also observed that significant relationship existed

between Perceived Usefulness and e-learning, while the weakest relationship existed between

e-learning and Perceived Ease of Use.

Usefulness is the strongest predictor of the intention to start using Blackboard, and enjoyment

has a great impact on this as well. The results of the study clearly point out the important roles

of enjoyment and usefulness in positively influencing the discussion to use Blackboard and

subsequent actual use. These findings significantly extend prior research on user acceptance

of web-based technology (Yi and Yujong Hwang, 2003; Selim, 2003; Shu-Sheng Liaw, 2008)

and empirically validating the relationship with enjoyment as the external variable.

Enjoyment is a powerful external factor which positively effects behavioural intention

through usefulness. This corresponds with the findings of Yi and Yujong Hwang (2003) and

Goetz et al. (2006). Enjoyment might play a more influential role than ease of use in

determining the usefulness perception within the Web-based IS context.

18

From table 5 – comments by students – it appears that students complain about technology

(schedule, announcements, failure), about the faulty structure and about the fact that teachers

do not work with Blackboard often enough. This possibly explains the low scores from

students. Moreover, table 1 and 5 shows that improve the performance of Blackboard are not

entirely successful. The development of policy at central level (project team) and then hand

over to the schools (key users) fails often because schools (managers) also have their own

agenda. Therefore, members of the project team have to remain present in the

implementation, in order to support managers and key users, answer questions, motivate and

so on. Salmon (2005) notice “the difficulty of truly ‘embedding’ e-learning into everyday

practice including uncertain leadership, lack of true innovation and lack of professional

expertise”.

Technical problems were also observed in the research by Al-Drees (Al-Drees et al., 2015)

which mentions technical problems in relation to Blackboard. The finding of this study

showed a poor utilization of Blackboard learning features. Students faced technical

difficulties while using Blackboard. The findings of this study indicate the need of

compulsory students training of any newly introduced learning management systems

including Blackboard in the educational institute.

In relation to the Four in Balance Model the balance is disturbed. The project approach of the

implementation of Blackboard 9.1 (learn) with project teams for every school, in such a way

that the new educational view is supported in the best possible way, has not led to a better use

of Blackboard in education by students’. This approach goes only on for the teachers’

population. The implementation of learning technology is apparently more complex than it

was thought.

According to the “Four in Balans Monitor 2013” three quarters of the schools in the

Netherlands indicates that they are changing the education and search for the possibilities that

ICT has to offer. The need where the education as a whole stands for is: minimize wrong

choices using knowledge about what works with ICT. The wrong expectation that young

people, many with ICT, also spontaneously have the skills to find information, to understand

and use (Kirschner, 2013). Lack of understanding of what works with ICT and what not,

hampers schools to make good choices.

According to Rubens (2015), the introduction of technology enhanced learning

19

is a complicated process, in which a large number of factors and actors play an important role.

It is primarily a process in which the human factor plays an important role, like dynamics,

interests, relationships and shared leadership. The multitude of actors makes education

difficult to predict and plan an innovation dynamic and fascinating process”.

Rubens (2013) recommends tackling the implementation process on a project basis. This

conflicts with what Assen & van der Pol (2008) argue: “In our experience, what successful

implementations have in common is that they do not work in terms of a project, but in terms

of a multi-year programme. E-learning involves organisational change and this cannot be

achieved with one project.” They also argue that “while it is important to have ICT on board,

e-learning is often overlooked on the strategic ICT agenda. As a result, the development of

internal ICT expertise for e-learning often lags behind and the tasks to be performed by ICT

tend to incur delays”.

According to Jacobs (2013), in the Netherlands, the use of ICT in learning environments often

depends on improvisation, coincidences, individual commitment and isolated projects.

Teachers and education managers often do not have an interconnected vision on the

application and impact of ICT in education. Viewed in light of the intensive use of ICT in

society, education lags behind.

Students abandon Blackboard and they are using other tools like Facebook, WhatsApp and

Twitter. There are becoming more free cloud solutions as alternative for Blackboard. This

allows the Police Academy called for a new dilemma. Continue with Blackboard? The cloud

is safe enough for the Police Academy? The finding of Maleko (Maleko et al., 2014) was that

Facebook attracted more students (over Blackboard) due to its social and community learning

benefits, encouraging students to support one another. Blackboard was viewed as the

authoritative and valid medium for official course material. Social media is not the answer to

every problem, but it’s an undoubtedly a useful and powerful advance in learning’ (Clark,

2011). Moreover, is there a growing gap between learners and teachers in the way they use

the Internet? (Atwell, 2011).

20

5. Conclusions and recommendations

The introduction of Blackboard 9.1 (learn) had a positive impact on the intention of teachers

to use Blackboard. Significant progress may be perceived in the intention of teachers to use

Blackboard.

The results show that, among students, the appreciation of the use of Blackboard significantly

decreased in 2015 compared to 2011. Is this only due to technical problems when showing the

timetable? Have students been sufficiently involved in the design and formulation of the

results?

What did the Police Academy of the Netherlands learned from this research?

• There has been an insufficient focus on implementation at the level of the students.

Involve students in setting up and using Blackboard

• It is recommended to spread to large changes in the time. Too many changes,

(introduction Blackboard 9.1(learn), Single Sign On, new student administration

system, new curriculum, new schedule program, many management changes) at the

same time addressing are asking for trouble.

• Make a programme for the implementation of ICT.

• Teachers who form a vanguard may be used as ambassadors within the school by

sharing good practices

• Members of the project team have to remain present in the implementation, in order to

support key users, answer questions, motivate and so on.

• Looking at "four in balance model", it can be stated that the balance is disturbed. It is

recommended to invest in the professional skills of managers and teachers. It is also

recommended to use instructions in the teacher’s manuals for Blackboard use. Above

all, technology has to be perfect

Although there are many more questions than answers regarding teaching with Blackboard as

learning management system, it is hoped that these results and experiences may encourage

further pedagogical dialogue, and empirical results about how to deliver and organize courses

in this technological environment effectively and successfully.

21

References

Agarwal R and Karahanna E. (2000). Time Flies When You're Having Fun: Cognitive

Absorption and Beliefs about Information Technology Usage. MIS Quarterly, Vol. 24, No. 4

(Dec., 2000), pp. 665-694

Albirini, A. (2006) Teachers’ attitudes toward information and communication technologies:

the case of Syrian EFL teachers. Computer & Education. Volume 47, Issue 4, December

2006, Pages 373-398

Al-Drees, A., Khalil, M.S., Meo, A. and Abdulghani, H.M. (2015). Utilization of blackboard

among undergraduate medical students: Where we are from the reality? Journal of Taibah

University Medical Sciences. Volume 10, Issue 1, March 2015, Pages 16–20.

doi:10.1016/j.jtumed.2014.07.002

Assen, D. en van der Pol, R. (2008). Duurzaam implementeren van e-learning.

Leerinfrastructuur realiseren voor lerende organisaties. Thema Nieuwe Technologie

DEVELOP NR 4-2008 blz. 44 – 55.

Attwell, G. (2005), “E-learning and sustainability”, available at: www.ossite.org/Members/

GrahamAttwell/sustainibility/attach/sustainibility4.doc (accessed 1 September 2006).

Atwell, G. (2011) Pedagogic Approaches to using Technology for Learning – Literature

Review Last accessed at June 12 2011 at URL:

http://www.pontydysgu.org/2011/05/pedagogic-approaches-to-using-technology-for-learning-

literature-review

Aversano, N. (2005). Technology rejection of mobile telephones. Case Western Reserve

University.

Benbasat, I. and Barki, H. (2007). Quo vadis, TAM? Journal of the Association of

Information Systems 8 (4)): 211–218.

Banus S. and Baptist T. (2005). De Acceptatie van Online Shoppen. Bachelor Thesis

Marketing. Radboud Universiteit Nijmegen. Netherlands

22

Barclay, D., Higgins, C., Thompson, R. (1995). The partial least squares approach to causal

modelling: personal computer adoption and use as an illustration. Technology Studies 2, 285-

309.

Chuttur M.Y. (2009). Overview of the Technology Acceptance Model: Origins,

Developments and Future Directions. Indiana University, USA . Sprouts: Working Papers on

Information Systems, 9(37). http://sprouts.aisnet.org/9-37

Clark, D. (2011) 7 objections to social media in learning (and answers) Last accessed at June

12 2011 at URL: http://donaldclarkplanb.blogspot.com/2011/06/7-objections-to-social-media-

on.html

Cross, J. & Dublin, L. (2002). Implementing e-learning. Alexandria: ASTD

Davis. F.D. (1993). User acceptance of information technology: system characteristics, user

perceptions and behavioural impacts. International Journal of Man–Machine Studies 38, pp.

475–487.

Donghua T. (2009) Intention to Use and Actual Use of Electronic Information Resources:

Further Exploring Technology Acceptance Model (TAM). AMIA Annu Symp Proc. 2009;

2009: 629–633. Published online 2009 Nov 14.

Ertmer , P.A., Ottenbreit-Leftwich, A.T., Sadik, O., Sendurur, E., and Sendurur, P. (2012).

Teacher beliefs and technology integration practices: A critical relationship. Computers &

Education Volume 59, Issue 2, September 2012, Pages 423–435

Gallwey, W. T. (1999). The Inner Game of Work. Random House, New York.

Goetz, T., Hall, N. C., Frenzel, A. C. and Pekrun, R. (2006) A Hierarchical

Conceptualization of Enjoyment in Students. Learning and Instruction, v16 n4 p323-338 Aug

2006

23

Hudson, B. (2008). Didaktik Design for Technology Supported Learning. Zeitschrift fur

Erziehungswissenschaft, Vol. 9. pp.139-158

Jacobs, F. (2013). Slagvaardig met ICT. Proefschrift TU Delft

Kennisnet (2008/2013). Vier in Balans Monitor 2008/2013. Kennisnet

Kirschner, P. (2013). Knopvaardig is wat anders dan digitaal geletterd. In: 4W. Weten wat

werkt en waarom (2-2013).

Kotter J.P. (1996) Leading Change. Havard Business School. Press, Boston, Massachusetts,

USA.

Legris, P., Inghamb, J. and Collerette P. (2003). Why do people use information technology?

A critical review of the technology acceptance model. Information & Management 40 (2003)

191–204.

Lund, A and Lund, M. (2010). Laerd Statistics. Lund Research Ltd 2010 Last accessed on

13 December 2011 on URL: http://statistics.laerd.com/statistical-guides/pearson-correlation-

coefficient-statistical-guide.php

Maleko M., Hamilton M., D'Souza D. and Scholer F. (2014) Facebook versus Blackboard for

Supporting the Learning of Programming in a Fully Online Course: The Changing Face of

Computing Education. Learning and Teaching in Computing and Engineering (LaTiCE)

International Conference

Miedema, W.G. and Stam, M. (2008). Leren van innoveren: wat en hoe leren docenten van

het innoveren van het eigen onderwijs? Thesis. Universiteit van Amsterdam

Netteland G., Wasson B., and Anders I. (2007) E-learning in a large organization A study of

the critical role of information sharing. Journal of Workplace Learning Vol. 19 No. 6, 2007

pp. 392-411

24

Okantey M and Addo H. (2016) Effect of theoretical and institutional factors on the

adoption of e-learning. European scientific Journal Vol 12, No 16. Last assessed on 5 July

2016 at URL: http://www.eujournal.org/index.php/esj/issue/view/235

Pei-Chen Sun , Ray J. Tsai, Glenn Finger, Yueh-Yang Chen and Dowming Yeh (2008). What

drives a successful e-Learning? An empirical investigation of the critical factors influencing

learner satisfaction. Computers & Education Volume 50, Issue 4, May 2008, Pages 1183–

1202

Pikkarainen, T.; Pikkarainen, K.; Karjaluoto, H. (2004). Consumer acceptance of online

banking: An extension of the Technology Acceptance Model. Internet Research-Electronic

Networking Applications and Policy 14 (3): 224–235,doi:10.1108/10662240410542652

Rosenberg, M. (2001), E-Learning: Strategies for Delivering Knowledge in the Digital Age.

McGraw-Hill, New York, NY

Rosenberg, M. (2006) Beyond E-learning: Approaches and Technologies to Enhance

Organizational Knowledge, Learning and Performance. Last accessed on 16 July 2004 at

URL: http://www.learningcircuits.org/2006/March/rosenberg.htm

Rubens, W. (2013). E-learning. Trends en ontwikkelingen Uitgeverij InnoDoks

Rubens, W. (2015). Geleerde lessen op het gebied van onderwijs innovaties. Blog:

Technology Enhanced Learning. Last accessed on 23June 2015 on URL:http://www.te-

learning.nl/blog/geleerde-lessen-op-het-gebied-van-

onderwijsinnovatie/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3

A+typepad%2FfIYD+%28Wilfred+Rubens+TE-

learning+centrum%3A+technology+enhanced+learning%29

Salmon, G. (2005) Flying not flapping: a strategic framework for e-learning and pedagogical

innovation in higher education institutions. ALT-J, Research in Learning Technology Vol. 13,

No. 3, October 2005, pp. 201–218

25

Schaper, L and Pervan, G. (2004). A Model of Information and Communication Technology

Acceptance and Utilisation by Occupational Therapists. The IFIP TC8/WG8.3 International

Conference 2004.

Selim, H. M. (2003). An empirical investigation of student acceptance of course websites.

Computers & Education 40, 343-360.

Selim, H. (2007). Critical success factors for e-learning acceptance: Confirmatory factor

models. Computers & Education, 49, 396–413. doi:10.1016/j.compedu.2005.09.004

Shu-Sheng Liaw (2008). Investigating students’ perceived satisfaction, behavioural intention,

and effectiveness of e-learning: A case study of the Blackboard system. Computers &

Education Volume 51, Issue 2, September 2008, Pages 864–873

Timothy T., and Noyes J. (2011). An assessment of the influence of perceived enjoyment and

attitude on the intention to use technology among pre-service teachers: A structural equation

modeling approach. Computers & Education, Volume 57, Issue 2, September 2011, Pages

1645-1653

Vrielink, R. (2006). Predicting the use of Blackboard and predicting the use of a Personal

Digital Analyser with the Technology Acceptance Model. Current Developments in

Technology-Assisted Education. Formatex 2006. Vol. 1. pp. 591 - 595.

Vrielink, R. (2015). Appreciation of the use of Blackboard at the Police Academy of the

Netherlands using the Technology Acceptance Model. The Journal of Technology Enhanced

Learning, Innovation & Change, 1 (1)

Wu J.H. and Wang S.C (2005) What drives mobile commerce? An empirical evaluation of

the revised technology acceptance model Information & Management 42 (2005) 719–729

Yi, Mun Y. and Yujong Hwang. (2003). Predicting the use of web-bases information systems:

self-efficacy, enjoyment, learning goal orientation, and technology acceptance model.

International Journal of Human-Computer Studies. 59, 431-449.