on the COLLECTION of QUESTIONNAIRES

SEVAQ - European report Version 2

Appendix 2. Details of the assessments08/03/2006 Page 1 of 1

Self-Evaluation and

Quality in

E-learning

EUROPEAN REPORT on the COLLECTION of QUESTIONNAIRES

COUNTRY EFODL

SEVAQ PARTNERS BE-ODL, BLA, CESI, ISQ, NADE, Le PREAU, VDU, VIDEOSCOP

Rik Dalle

E-MAIL [email protected] This is the report of the first phase of the SEVAQ project in which the participating partners have enquired the status of existing evaluation instruments for the assessment of e-learning courses. In the first phase of the SEVAQ project we have concentrated on the use of questionnaires that are presented to the users of e-learning courses who are the learners. These questionnaires could be presented to the learners in an on-line form or in paper form. In this report we describe the results for the whole of the European contributions. We draw conclusions and remarks on the different topics of the enquiry and on the whole of the investigation. We also point to differences in the results between the different country samples. For the conclusions on each topic, we have based our results also on the country reports provided by each SEVAQ partner. For more details on the results of the enquiry, we refer to these country reports. We have to take into account that some sample were rather small, so that we have to consider the conclusions with care. The first chapter of the report provides an overview of the results from the SEVAQ mailing sent to a large number of organisations in the participating countries, while the second chapter describes the results of the assessments by the SEVAQ partners of the questionnaires sent by the respondents. CHAPTER 1. THE MAILING RESULTS

For each topic we present tables with the total of all European data and a histogram displaying the distribution per partner. The details of the scores for each country are presented in appendix 1.

SEVAQ - European report Draft

08/03/2006 Page 2 of 2

THE MAILING

Number Percentage

Number of mails sent 9170 --- Number of mail replies received 181 1,97 % Number of organisations that sent their questionnaire(s) 78 0,85 % Number of questionnaires received 152 Mean number of questionnaires sent per organisation that sent mails 1,95

The participating partners in the SERVAQ project have sent out a large number of mails to a variety of organizations that are possibly dealing with e-learning. From the 9170 mails that were sent out, not even 2 percent of answers were received and les than 1 % of the organizations sent one or more questionnaires. The number of questionnaires that were sent was a bit disappointing. The Belgian partner also organized a telephone enquiry to organizations that hadn’t replied to the mailing. For a report on the telephone enquiry we refer to …

GENERAL INFORMATION

What was the profile of the organisations that replied on the mailing?

1. Type of organisation

Type Number PercentageNo answer 24 13

Company 48 25

Non profit organisation 19 10

Educational institute 99 52

Total 190 100


08/03/2006 Page 3 of 3

041116

10

1

12

10

611

17

01

53101540

5334

25

17

30

12

0

10

20

30

40

50

60

70

80

90

100

Number of answers

No answer Company Non profit organisation Educational instituteType of organisation

SEVAQ - TYPE OF ORGANISATION

BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP In the European sample more than half of the respondents were educational institutes like universities and a quarter were companies. There are however clear differences between the countries. Some countries like Norway, France (Vidéoscop), Lithuania and Portugal have a large majority of educational respondents. In Belgium, the United Kingdom and Spain, at least 50 % of the responders are companies. At Le Préau there is an equal amount of companies and educational organisations. The educational institutions in the Préau sample don’t include universities but private educational organisations that form an important market in France. The conclusions drawn on the type of organisations are partly due to the different compositions of the networks of the partners involved, and so of the composition of their mailing lists. Some networks are merely oriented to the educational world (universities or not academic), while others are more oriented towards commercial enterprises.

2. Provider or user of e-learning

Number Percentage No answer 26 14 Provider 102 54 User 17 9 Provider and user 45 24

Total 190 100


08/03/2006 Page 4 of 4

241116

101

8

11

21

25

23

22

10

60100730

657429

9

3

0

20

40

60

80

100

120

Number

No answer Provider User Provider and userProvider or user

SEVAQ - PROVIDER OR USER

BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP 78 % of the respondents are either provider or provider and user. 33 % is also a user of e-learning courses. In the Norvegian sample we found the largest share of pure providers (89 %) in a group that are mainly educational institutes. From these results we can conclude that the providers that are most represented in the enquiry are the effective target of the SEVAQ research and are also interested in the results of this project. Probably the providers are “waiting” for general quality standards for e-learning and an appropriate tool to measure quality.

3. Size of organisation

Number of employees Number Percentage No answer 29 15 < 50 64 34 50 – 200 22 12 200 – 500 19 10 > 500 56 29

Total 190 100


08/03/2006 Page 5 of 5

3

4112

6

11

1

2

9

21

11

16

23

0

13106

10

10

5002

7

230

11

4

7

22

11

6

13

0

10

20

30

40

50

60

70

Number

No answer < 50 50 – 200 200 – 500 > 500Size of organisation

SEVAQ - SIZE OF ORGANISATION

BE-ODL N BLA N CESI N ISQ N NADE N LE PREAU N VDU N VIDEOSCOP N Conclusions and comments on the type and size of the organisations Two groups are mostly present in the results: the small organisations of less than 50 employees (34 %) and the large organisations with more than 500 employees (29 %). We can observe some differences between the countries. Small organisations of less than 200 employees are less represented in the Belgian sample (14 %). We suppose that most of the e-learning players in Belgium are large organisations, especially large companies. We see the same profiles in Spain. Videoscop (93 %) and also ISQ have a majority of large (educational) institutes (universities). At the other side, the BLA (45 %), Le Préau (36 %), NADE (39 %) and VDU (52 %) have significant proportions of small organisations (< 50) in their samples.

INFORMATION REGARDING THE QUESTIONING AND EVALUATION

1. Can the e-Learning student and trainer give feedback concerning the e-learning course?

Answers Number Percentage No answer 25 13 Yes 155 82 No 10 5


08/03/2006 Page 6 of 6

Total 190 100

041106

121

22

12

105

28

38

28

12

04000141

0

20

40

60

80

100

120

140

160

Number

No answer Yes NoAnswers

SEVAQ - CAN LEARNER GIVE FEEDBACK?

BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP It is not surprising that almost 82 % want to receive feedback from the learners. A reason for a negative answer can be that the organisation is not yet dealing with e-learning. The reasons why organisations ask for feedback can be very different: - questionnaires are made for the customer and to measure the quality of the serviced provided; - questionnaires are used to evaluate the quality of the provider and to measure the ROI; - in a process of change, feedback is used to measure the receptiveness of learners to this new way of learning; - feedback can also be part of a formal quality management programme; - to improve their system; - to feed their reflection on distance learning.

2. The form of the feedback

Form Number PercentageNo answer 38 20 A written questionnaire 76 40 An online form 93 49 Informal conversation (e.g. during coffee break) 25 13


08/03/2006 Page 7 of 7

Telephone conversation 27 14 Other 26 14

Total 285 ---

181117

16

3

9

644

10

19

17

7

11

11

72

18

16

23

5

42323461

433208

61

4223010

41

0

10

20

30

40

50

60

70

80

90

100

Number

No answer A writtenquestionnaire

An online form Informalconversation (e.g.

during coffeebreak)

Telephoneconversation

Other

Form of feedback

SEVAQ - FORM OF FEEDBACK

BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP The majority of the feedback is gathered in a structured way through a written or an online questioning. In 49 % of the organisations online feedback is being used. At Le Préau, Vidéoscop and ISQ written questionnaires are used more than online ones. This can be due to blended learning where paper versions can be used more frequently. We can conclude that online questionnaires are getting common in e-learning. Organisations can also use other channels as e-mail, videoconferencing or direct assessment by the controller of the worker. It is obvious that the respondents often use more than 1 way of evaluation. The gathering of this information is not finished yet. In the samples of Le Préau (40 %), Vidéoscop (42 %) and BE-ODL (37 %) a significant group used 2 to 4 ways of evaluation. Number of evaluation types Number PercentageNo answer 1 way of evaluation


08/03/2006 Page 8 of 8

2 ways of evaluation 3 ways of evaluation 4 ways of evaluation

Total To be finished.

3. Use of the feedback obtained

Feedback Number PercentageNo answer 38 20 Feedback is being archived 53 28 Feedback is made available to the trainer 95 50 Feedback is used to improve the quality of the evaluation system

92 48

Feedback is used to improve the quality of the e-learning course

124 65

Other 16 8 Total 418 ---

471126

16

1

4321

20

12

65

9

552

25

24

16

9

9

9

74

18

18

17

10

15

13

82

24

32

19

11

21121810

0

20

40

60

80

100

120

140

Number

No answer Archive Available to trainer Quality of evaluation Quality of e-learning OtherUse of feedback

SEVAQ - USE OF FEEDBACK

BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP 65 % of the organisations are using the feedback to improve the quality of the e-learning course. The other most important objectives of feedback are giving feedback to the trainer (50 %) and to improve the quality of the evaluation system


08/03/2006 Page 9 of 9

(48 %). These scores are very homogenuous spread over the countries. Only in Norway there is a significant high score for making the feedback available for the teacher. There are also other objectives for feedback as focussing on the outcomes of the course and the achievement of the objectives. Feedback can also be delivered to e-learning professionals like designers, tutors, authors and others. Organisations have more than one objective to assess the course: in France more than 80 % have more than one reason to let the learner assess the courses, while in Belgium this is only 46 %. These data are further collected. Number of objectives in doing evaluation Number PercentageNo answer Just one use of the feedback 2 objectives of feedback 3 objectives of feedback 4 objectives of feedback 5 objectives of feedback

Total To be finished.

4. Preceding and interim evaluation

Do the organisations work with preceding and/or interim evaluations in addition to the standard evaluation after the course? Answers Number Percentage No answer 30 16 Yes 80 42 No 80 42

Total 190 100


08/03/2006 Page 10 of 10

341106

12

3

8

6

5

4

16

19

18

4

11

10

51

12

20

14

7

0

10

20

30

40

50

60

70

80

Number

No answer Yes NoAnswers

SEVAQ - PRECEDING / INTERIM EVALUATION

BE-ODL N BLA N CESI N ISQ N NADE N LE PREAU N VDU N VIDEOSCOP N 42 % of the organisations work with preceding and/or interim evaluations. In some samples it seems more common (BLA 50 % and NADE 57 %), while Videoscop (29 %) and BE-ODL (36 %) have lower scores on this item.

KEEPING THE ORGANISATIONS INFORMED ABOUT THE SEVAQ PROJECT

Can we keep the organisation informed about the SEVAQ project? Answers Number Percentage No answer 190 100 Yes 0 0 No 0 0

Total 190 100 100 % didn’t answer this question. After observing all European scores, we can conclude that this score is due to a technical bug.


08/03/2006 Page 11 of 11

CHAPTER 2. ANALYSIS of THE ASSESSMENT OF THE QUESTIONNAIRES

In this chapter we describe the results of the assessments of the questionnaires of the respondents. These assessments were executed by the SEVAQ partners on the sample that each partner had collected. We used 4 main criteria with a few subcriteria: content, usability and delivery, technical specifications and organisation. The subcriteria were assessed in a range from 1 to 5: 1 = not at all 2 = a little bit 3 = average 4 = quite a lot 5 = very much In the next pages we discuss the main results per criteria. The detailed scores and graphs of the subcriteria can be found in appendix 2. We refer to “SEVAQ-e-learning-EFQM1.doc” by Jeanne Schreurs for a theoretical foundation of the set of quality criteria.

1. CONTENT

We measured 6 topics in Content criterion: 1.1. The questionnaire inquires about the matching of the learning objectives.

1.2. The questionnaire inquires about the learning level.

1.3. The questionnaire inquires about the foreknowledge of the learner.

1.4. The questionnaire inquires about the difficulty level of the e-learning course.

1.5. The questionnaire inquires about the pace of the e-learning course.

1.6. The questionnaire inquires about the availability and quality of the assignments, tests and exercises.

The results for all the SEVAQ partners together are displayed in the next table and graph.

1 2 3 4 5

1.1 33 26 27 15 6

1.2 36 33 22 13 2

1.3 56 22 16 8 5

1.4 43 31 18 14 2

1.5 45 40 11 7 4

1.6 33 18 33 14 8


08/03/2006 Page 12 of 12

CONTENT

0

10

20

30

40

50

60

1 2 3 4 5

levels

scor

es

1.1 1.2 1.3 1.4 1.5 1.6

Conclusions and comments about the content criterion Generally, the scores are towards the low end. Only a small part of the questionnaires are concerned about the learners competencies development during the course. The best results are for 1.1 matching of the learning objectives and 1.6 availability and quality of the assignments, tests and exercises. The subcriterion with the lowest general scores is 1.5 on the pace of the course. Some mention that questionnaires are dealing with other questions like the resources of the course, its relevance and its professional breadth and depth.


08/03/2006 Page 13 of 13

2. USABILITY AND DELIVERY

We measured 5 topics in this criterion: 2.1. The questionnaire inquires about the quality of the navigation in the course.

2.2. The questionnaire inquires about the quality of integrated communication.

2.3. The questionnaire inquires about the quality of embedded tutoring.

2.4. The questionnaire investigates whether the learning environment or e-course is motivating, stimulating or encouraging to learn (fun).

2.5. The questionnaire asks about the feedback to the end user concerning his progress, scores, etc.

The results

1 2 3 4 5

2.1 60 13 18 10 4

2.2 53 24 15 9 3

2.3 49 21 15 11 9

2.4 55 22 15 9 3

2.5 43 23 15 18 6

USABILITY

0

10

20

30

40

50

60

70

1 2 3 4 5

levels

scor

es

2.1 2.2 2.3 2.4 2.5


08/03/2006 Page 14 of 14

Comments and conclusions about the usability and delivery The scores of the usability and delivery criteria are also towards the low end. The attention for usability is rather limited. Some remark that usability is not really a problem anymore, although I doubt about that. There are no questions found on embedded tutoring in Belgium, although especially Norway and France found a number of questionnaires that showed some interests in this matter. Maybe this has to do with the usage of embedded tutoring that is yet made in the different countries. This was also a remark that tutoring mostly concerns about human interaction and not the embedded way.


08/03/2006 Page 15 of 15

3. TECHNICAL SPECIFICATIONS

We measured 5 subcriteria in this category: 3.1. The questionnaire inquires about the set up (installation) of the e-learning environment.

3.2. The questionnaire inquires about the session start-up, the performance and uploading of the pages.

3.3. The questionnaire inquires about the browser independency of the learning environment.

3.4. The questionnaire inquires about the availability and system stability.

3.5. The questionnaire inquires about the quality of the helpdesk facilities.

The results

1 2 3 4 5

3.1 72 10 9 10 3

3.2 58 24 11 9 2

3.3 90 11 2 1 0

3.4 70 14 8 9 2

3.5 62 18 13 6 3

TECHNICAL SPECIFICATIONS

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5

levels

scor

es

3.1 3.2 3.3 3.4 3.5


08/03/2006 Page 16 of 16

Comments and conclusions about the technical specifications There is also a minor attention to the technical aspects of the learning management systems and the e-courses. The “best” score, although still very weak, is found in 3.2 start-up, performance and uploading of the pages. If there are some questions about technical issues, they seem to be very general.


08/03/2006 Page 17 of 17

4. ORGANISATION

We measured 2 topics in this category: 4.1. The questionnaire inquires about the welcoming at the beginning of the course.

4.2. The questionnaire assesses the following administration topics: course registration, scheduling of the training programme, payment, and other information offered by the learning organisation.

The results

1 2 3 4 5

4.1 60 26 11 4 5

4.2 42 27 13 15 7

ORGANISATION

0

10

20

30

40

50

60

70

1 2 3 4 5

levels

scor

es

4.1 4.2

Comments and conclusions about the organisation of e-learning There is only a minor interest in organisation matters, although 4.2 about the administration topics show generally a relatively good score. Partners report that questions sometimes ask about the locations where learners go online and about the possibility to use an internet connection.


08/03/2006 Page 18 of 18

GENERAL CONCLUSIONS CONCERNING THE ASSESSMENT OF THE PARTICIPATING ORGANISATIONS AND THEIR QUESTIONNAIRES

General conclusions concerning the Belgian examination: - The questionnaires in the European samples scored rather low in the 4 criteria

we had chosen. - The criteria in our inquiry did possibly lack some important themes or some

criteria were to general (like coaching) and need to be more detailed.. - Typical and important aspects of online learning are seldom evaluated: usability

and technical issues. Although, these matters can be significant in the success of e-learning.

- Some remarks noticed that a number of questionnaires that organizations sent where questionnaires that are also used in face to face education. These are often very general questionnaires or they ask for topics that are typical for class room teaching as questions about the quality of the location, the teacher, the catering and the hand outs. That influenced also the assessments.

Some partners have added imported recommendations for further research so that other interesting criteria can be added in the tool that has to be developed in the next stages of the SEVAQ project. I refer to the different country reports. We can conclude from this enquiry that it’s very obvious that there is a need for an evaluation tool that is created to assess the quality of e-learning. This conclusion has to be finished further.



APPENDIX 1. DETAILS OF THE MAILING RESULTS General information concerning the mailing

BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE

Number of mails sent 1500 2500 583 20 350 3300 359 558 9170 Number of mail replies received 26 20 11 6 15 45 44 14 181 Number of organisations that sent their questionnaire(s) 16 6 6 3 11 19 9 8 78

Number of questionnaires received 20 20 11 6 28 45 10 12 152 1. Type of organization

BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 0 4 1 1 1 6 10 1 24 Company 12 10 6 1 1 17 0 1 48 Non profit organisation 5 3 1 0 1 5 4 0 19 Educational institute 5 3 3 4 25 17 30 12 99

Total 22 20 11 6 28 45 44 14 190 2. Provider or user of e-learning

BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 2 4 1 1 1 6 10 1 26 Provider 8 11 2 1 25 23 22 10 102 User 6 0 1 0 0 7 3 0 17 Provider and user 6 5 7 4 2 9 9 3 45

Total 22 20 11 6 28 45 44 14 190


Appendix 1. Details of the mailing results 08/03/2006 Page 20 of 20

3. Size of organisation

Number of employees BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 3 4 1 1 2 6 11 1 29 < 50 2 9 2 1 11 16 23 0 64 50 – 200 1 3 1 0 6 10 1 0 22 200 – 500 5 0 0 2 7 2 3 0 19 > 500 11 4 7 2 2 11 6 13 56

Total 22 20 11 6 28 45 44 14 190 INFORMATION REGARDING THE QUESTIONING AND EVALUATION 1. Can the e-Learning student and trainer give feedback concerning the e-learning course?”

BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 0 4 1 1 0 6 12 1 25 Yes 22 12 10 5 28 38 28 12 155 No 0 4 0 0 0 1 4 1 10

Total 22 20 11 6 28 45 44 14 190



2. In which form?

BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 1 8 1 1 1 7 16 3 38 A written questionnaire 9 6 4 4 10 19 17 7 76 An online form 11 11 7 2 18 16 23 5 93 Informal conversation (e.g. during coffee break)

4 2 3 2 3 4 6 1 25

Telephone conversation 4 3 3 2 0 8 6 1 27 Other 4 2 2 3 0 10 4 1 26

Total 33 32 20 14 32 64 72 18 285 BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 1 7 3 11 Just one way of applying evaluation 13 22 7 42 2 ways of applying evaluation 5 12 4 21 3 ways of applying evaluation 3 2 0 5 4 ways of applying evaluation 0 1 1 2

Total 22 0 0 0 0 44 0 15 81



3. Use of the feedback obtained

BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 4 7 1 1 2 6 16 1 38 Archive 4 3 2 1 20 12 6 5 53 Available to trainer 9 5 5 2 25 24 16 9 95 Quality of evaluation 9 9 7 4 18 18 17 10 92 Quality of e-learning 15 13 8 2 24 32 19 11 124 Other 2 1 1 2 1 8 1 0 16

Total 43 38 24 12 90 100 75 36 418 BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 4 6 1 11 Just one way of applying evaluation 8 6 2 16 2 ways of applying evaluation 3 15 2 20 3 ways of applying evaluation 3 9 4 16 4 ways of applying evaluation 4 5 4 13 5 ways of applying evaluation 0 2 0 2

Total 22 0 0 0 0 43 0 13 76



4. Preceding and interim evaluation Do the organisations work with preceding and/or interim evaluations in addition to the standard evaluation after the course? Preceding / interim evaluations BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 3 4 1 1 0 6 12 3 30 Yes 8 6 5 4 16 19 18 4 80 No 11 10 5 1 12 20 14 7 80

Total 22 20 11 6 28 45 44 14 190 KEEPING THE ORGANISATIONS INFORMED ABOUT the SEVAQ PROJECT Can we keep your organisation informed about the SEVAQ project?

BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 22 20 11 6 28 45 44 14 190 Yes 0 0 0 0 0 0 0 0 0 No 0 0 0 0 0 0 0 0 0

Total 22 20 11 6 28 45 44 14 190



1. CONTENT

1.1. The questionnaire inquires about the matching of the learning objectives.

1 2 3 4 5 BE-ODL 8 6 3 1 2 BLA 1 0 3 2 0 CESI 0 1 2 2 0 ISQ 4 0 0 0 0 NADE 6 7 8 4 0 LE PREAU 7 2 4 3 1 VDU 5 0 3 1 3 VIDEOSCOP 2 10 4 2 0 EUROPE 33 26 27 15 6

8

10

4

6

7

5

2

6

010

7

20

10

3

3

20

8

4

3

4

12

20

4

3

12

200001

30

0

5

10

15

20

25

30

35

Number

1 2 3 4 5Assessment

SEVAQ - 1.1 CONTENT - OBJECTIVES

BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP


Appendix 1. Details of the mailing results 08/03/2006

1.2. The questionnaire inquires about the learning level.


14

104

1

4

8

4

3110

12

3

4

9

13

30

9

402

11102

5

03

00001100

0

5

10

15

20

25

30

35

40

Number

1 2 3 4 5Assessment

SEVAS - 1.2 CONTENT - LEARNING LEVEL




1.3. The questionnaire inquires about the foreknowledge of the learner.


11

4

3

4

7

7

8

12

6

1006

603

1010

9

203

11102210

10001030

0

10

20

30

40

50

60

Number

1 2 3 4 5Assessment

SEVAQ - 1.3 FOREKNOWLEDGE




1.4. The questionnaire inquires about the difficulty level of the e-learning course.


11

22

4

3

6

7

8

4

2

30

10

3

2

7

2000

8

7

10

3

200312

3

00001100

0

5

10

15

20

25

30

35

40

45

Number

1 2 3 4 5Assessment

SEVAQ - 1.4 DIFFICULTY LEVEL




1.5. The questionnaire inquires about the pace of the e-learning course.


13

4

13

10

3

7

4

5

221

11

4

4

11

00003

4

13

20100400

00101200

0

5

10

15

20

25

30

35

40

45

Number

1 2 3 4 5Assessment

SEVAQ - 1.5 PACE




1.6. The questionnaire inquires about the availability and quality of the assignments, tests and exercises.


9

4

00

7

5

5

3

2120

8

212

7

100

6

4

3

12

002

4

3

4

01

201002

30

0

5

10

15

20

25

30

35

Number

1 2 3 4 5Assessments

SEVAQ - 1.6 ASSIGNMENTS & TESTS




2. USABILITY AND DELIVERY

2.1. The questionnaire inquires about the quality of the navigation in the course.


17

324

12

5

8

9

00006

214

21102

6

15

02204200

10000120

0

10

20

30

40

50

60

Number


SEVAQ - 2.1 NAVIGATION




2.2. The questionnaire inquires about the quality of integrated communication.


14

324

5

5

9

11

4

320

8

403

10104

324

10003

410

00003000

0

10

20

30

40

50

60

Number


SEVAQ - 2.2 INTEGRATED COMMUNICATION




2.3. The questionnaire inquires about the quality of embedded tutoring.


20

4

23

6

5

6

3

0101

6

3

5

5

0110230

8

0000

7

202

00203

310

0

5

10

15

20

25

30

35

40

45

50

Number


SEVAQ - 2.3 EMBEDDED TUTORING




2.4. The questionnaire investigates whether the learning environment or e-course is motivating, stimulating or encouraging to learn (fun).

1 2 3 4 5

BE-ODL 15 1 3 0 1 BLA 0 3 1 2 0 CESI 3 1 0 1 0 ISQ 4 0 0 0 0 NADE 9 9 2 2 1 LE PREAU 6 4 4 1 1 VDU 9 0 2 1 0 VIDEOSCOP 9 4 3 2 0 EUROPE 55 22 15 9 3

15

03

4

9

6

9

9

1310

9

404

31002423

02102112

10001100

0

10

20

30

40

50

60

Number


SEVAQ 2.4 MOTIVATION

BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP 2.5. The questionnaire asks about the feedback to the end user concerning his progress, scores, etc.




15

200

8

5

8

5

2220

6

31

7

22103

5

02

001

4

6

21

4

10101120

0

5

10

15

20

25

30

35

40

45

Number


SEVAQ - 2.5 FEEDBACK ON PROGRESS




3. TECHNICAL SPECIFICATIONS

We measured the following technical specifications: 3.1. The questionnaire inquires about the set up (installation) of the e-learning environment. 1 2 3 4 5 BE-ODL 20 0 0 0 0 BLA 2 1 1 1 0 CESI 5 0 0 0 0 ISQ 4 0 0 0 0 NADE 7 7 6 3 1 LE PREAU 5 2 1 6 2 VDU 11 0 1 0 0 VIDEOSCOP 18 0 0 0 0 EUROPE 72 10 9 10 3

20

25

4

7

5

11

18

01007

200

01006110

01003

600

00001200

0

10

20

30

40

50

60

70

80

Number


SEVAQ - 3.1 THE SET-UP




3.2. The questionnaire inquires about the session start-up, the performance and uploading of the pages.


12

233

7

7

10

14

2110

10

424

32105000

30011400

00001100

0

10

20

30

40

50

60

Number


SEVAQ - 3.2 PERFORMANCE




3.3. The questionnaire inquires about the browser independency of the learning environment.


20

544

19

13

9

16

00105212

00000020

00000100 000000000

10

20

30

40

50

60

70

80

90

Number


SEVAQ - 3.3 BROWSER INDEPENDENCY




3.4. The questionnaire inquires about the availability and system stability.


16

250

15

8

7

17

32004

401

10001330

01042110

00001010

0

10

20

30

40

50

60

70

Number


SEVAQ - 3.4 SYSTEM STABILITY




3.5. The questionnaire inquires about the quality of the helpdesk facilities.


20

324

6

7

11

9

01105

407

01105

312

00004200

00102000

0

10

20

30

40

50

60

70

Number


SEVAQ - 3.5 HELPDESK FACILITIES




4.1. The questionnaire inquires about the welcoming at the beginning of the course.


15

324

7

9

10

10

1230

12

12

5

20004

203

11001100

10001300

0

10

20

30

40

50

60

Number


SEVAQ - 4.1 WELCOMING




4.2. The questionnaire assesses the following administration topics: course registration, scheduling of the training programme, payment, and other information offered by the learning organisation.


11

22

3

4

5

7

8

4

220

6

5

4

4

31003

213

2100

6

303

0010

5

100

0

5

10

15

20

25

30

35

40

45

Number


SEVAQ - 4.2 ORGANISATIONAL TOPICS


Documents

on the COLLECTION of QUESTIONNAIRES