1

e-Bug Pack Evaluation

2

Evaluation Process

Ireland England

France

Spain

Finland

Greece

Poland

Denmark

Belgium

Czech Rep.

Slovenia

Latvia

Lithuania

Hungary

Slovakia

Croatia

England

France

Czech Rep.

• Control and Intervention

Schools

• Junior and Senior

Schools

• Minimum of 151 students

in each group

• Evaluation carried out in

3 countries

3

Regions Evaluated

England

Czech Republic

France

Gloucester London

Bordeaux

Nice

Prague Ostrava

4

Primary Outcome

• Knowledge Change

– Baseline knowledge

– Knowledge change

– Knowledge retention

• Variables

– Teacher knowledge of microbiology

– Class size

– Region

– Learning disability register

5

Secondary Outcome

• Satisfaction

– Teacher Questionnaires

– Student Questionnaires

– Focus groups

6

Knowledge Change Results

• Variable did not have any effect on

knowledge change at any of the time points

• Exception Region

7

Introduction to Microbes

• An Introduction

– Make microbes out of paper cut

outs and Arts & Crafts

materials

• Good Microbes

– Yogurt Making Activity

• Bad Microbes

– Interactive Discussion

8

Introduction to Microbes

Control Intervention

Baseline Change Retention Baseline Change Retention

Gloucester 6.8

(-4.4, 18.0)

33.5

(22.3, 44.7)

33.7

(22.6, 44.9) 0

32.5

(30.3, 34.7)

29.8

(27.0, 32.6)

London - - - - - -

Nice 8.1

(-0.7, 16.9)

24.3

(14.5, 34.1)

26.5

(15.5, 37.4)

7.0

(-0.8, 14.7)

30.0

(22.3, 37.8)

32.5

(24.3, 40.6)

Bordeaux 9.3

(-0.6, 19.1)

29.1

(18.8, 39.4)

31.5

(21.3, 41.8)

13.2

(2.2, 24.2)

36.4

(24.2, 48.5)

36.2

(23.8, 48.6)

Prague 3.5

(-4.7, 11.7)

14.1

(5.3, 22.9)

13.4

(4.7, 22.0)

6.0

(-2.4, 14.4)

26.4

(18.0, 34.8)

28.4

(19.9, 36.9)

Ostrava -0.7

(-8.7, 7.3)

2.1

(-6.2, 10.4)

11.6

(3.6, 19.6)

4.7

(-3.5, 12.8)

30.8

(22.7, 39.0)

29.1

(21.0, 37.3)

9

Introduction to Microbes

Control

Change

Control

Retention

e-Bug

Change

e-Bug

Retention

Gloucester

London - - - -

Nice

Bordeaux

Prague

Ostrava

Significant improvement in knowledge across all regions

No loss in student knowledge over time except Gloucester*

10

Spread of Infection

• Hand Hygiene

– Hand washing activity using UV

gel/powder to represent microbes

• Respiratory Hygiene

– Sneezing Runway

• Food Hygiene

– Chicken salad activity using UV

gel/powder to represent microbes

on raw chicken

11

Spread of Infection

Control Intervention

Baseline Change Retention Baseline Change Retention

Gloucester 5.7

(-3.5, 14.9)

10.5

(1.3, 19.8)

12.2

(3.0, 21.4) 0

10.6

(8.6, 12.6)

12.2

(9.6, 14.8)

London - - - - - -

Nice 2.8

(-4.7, 10.3)

-7.7

(-16.2, 0.8)

-2.1

(-11.7, 7.5)

-3.0

(-9.5, 3.4)

6.2

(-0.3, 12.6)

8.1

(1.2, 14.9)

Bordeaux 1.8

(-6.3, 9.9)

11.5

(3.0, 20.1)

12.4

(3.9, 21.0)

5.0

(-4.0, 14.0)

15.5

(5.3, 25.7)

11.9

(1.5, 22.3)

Prague 3.3

(-3.5, 10.1)

3.4

(-4.1, 10.9)

2.1

(-5.2, 9.4)

-3.8

(-10.9, 3.3)

9.4

(2.4, 16.5)

6.9

(-0.2, 14.1)

Ostrava -4.4

(-11.1, 2.2)

-5.3

(-12.3, 1.7)

-4.3

(-11.0, 2.4)

-2.7

(-9.5, 4.1)

8.6

(1.8, 15.4)

10.7

(3.9, 17.5)

12

Spread of Infection

Control

Change

Control

Retention

e-Bug

Change

e-Bug

Retention

Gloucester

London - - - -

Nice

Bordeaux

Prague

Ostrava

Significant improvement in knowledge across all regions

No loss in student knowledge over time

13

Treatment / Prevention of Infection

• Vaccinations

– Reading comprehension

‘The story of Edward Jenner’

• Antibiotic Use and

Medicines

– Comic Strip and classroom

discussion

14

Control Intervention

Baseline Change Retention Baseline Change Retention

Gloucester 2.8

(-8.4, 14.1)

16.8

(5.5, 28.1)

14.7 (3.5,

25.9) 0

15.4

(12.6, 18.1)

14.6

(11.1, 18.1)

London - - - - - -

Nice 15.9

(6.7, 25.0)

24.1

(13.6, 34.6)

12.1

(0.0, 24.3)

13.3

(5.5, 21.1)

22.5

(14.6, 30.3)

27.4

(19.0, 35.9)

Bordeaux 19.0

(9.1, 28.9)

26.9

(16.4, 37.5)

32.2

(21.6, 42.7)

18.7

(7.8, 29.7)

30.8

(18.1, 43.5)

22.3

(9.3, 35.3)

Prague 14.0

(5.7, 22.3)

19.8

(10.5, 29.2)

24.4

(15.4, 33.4)

16.9

(8.2, 25.5)

29.8

(21.2, 38.4)

26.0

(17.3, 34.7)

Ostrava 12.4

(4.3, 20.5)

10.1

(1.4, 18.7)

10.3

(2.1, 18.5)

9.5

(1.2, 17.8)

28.3

(19.9, 36.6)

20.0

(11.7, 28.3)

Treatment / Prevention of Infection

15

Control

Change

Control

Retention

e-Bug

Change

e-Bug

Retention

Gloucester

London - - - -

Nice

Bordeaux

Prague

Ostrava

Significant improvement in knowledge across all regions

No loss in student knowledge over time except Gloucester

Treatment / Prevention of Infection

16

Knowledge Change Summary

• Significant Improvement in all pack

teaching areas over time

• No reduction in student knowledge after a

6 week period

• Control schools performed equally as well

17

Student Satisfaction

0%

20%

40%

60%

80%

100%

Intro

duct

ion

Goo

dBad

Han

d

Res

pira

tory

Food

Vac

cines

Ant

ibio

tics

Disliked OK Liked

18

Student Satisfaction

• Likes – Interesting Facts

– Interactive, fun activities

– Team work

– Seeing and naming the microbes

– Characters

• Dislikes – Waiting too long to take your turn

– Some activities too difficult

– When the teacher did the work

– Finishing the activity!

19

Teacher Satisfaction

• Junior Schools

– 100% satisfaction in

England and Czech

Republic

– 3/11 teachers in France

said they would NOT

use e-Bug in the future

20

Teacher Satisfaction

• Teacher Likes

– Interactive

– Engaged students

• Teacher dislikes

– Too much photocopying

– Some activities too advanced

– Yogurt activity not suitable for practical reasons

– Pages didn’t photocopy well

– Not enough multimedia support

21

General Changes

• Photocopying issues

– Merged student handouts

– Created handouts in MS PowerPoint

– Changed the colour of the resource

22

General Changes

• Multimedia Support

– Films of activities

– Added alternative activities to website

– Included more posters etc on the website

– Pack available in MS Word and PDF

23

General Changes

• Other

– Provide more alternative materials

– Include more prompts and suggestions for

teachers

– Provide more animations

– Introduce an Immunity Section

24

Specific Sectional Changes

• Introduction

– Changed arts and craft materials to playdough

• Good and bad

– Changed titles to USEFUL and HARMFUL

microbes

– Created a new ‘Yeast Races’ activity

• Food Hygiene

– Food Hygiene: Condensed main activity

• Vaccinations

– Vaccinations: Added a role play element