Developing online activities for postgraduate students in computing

Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).

Judith JeffcoateAssociate Teaching FellowCOLMSCT

2

Background

• This project focuses on support for postgraduate students taking OU computing courses

• M883 Software Requirements for Business Systems– Presented twice a year (May and November)– Printed and online materials via a VLE (Moodle)– Limited tutorial support via three assignments

• Outcomes include: “understand the state-of-the-art in requirements engineering, leading to a deep and systematic understanding of the subject area”

• Achieved “through the use of books, reference papers (e.g. conference and journal publications) and other audio/visual media”

3

Project structure

Step 1Survey of tutorial

group from Nov 06presentation

Step 2Pilot study with

volunteers on May 07 presentation

Step 3Full study

open to all on May 08 presentation

Analysisof findings

Analysisof findings

Evaluation

4

Survey: online toolsStudents were asked to consider how useful five online tools were for ten different tasks

Very useful (1)

(2)

(3)

(4)

Not at all useful (5)

Chat

Forum

Wiki

Blog

Journal

0

5

10

15

20

25

30

35

40

Total score (all tasks)

They preferred certain tools to carry out specific tasks

5

Survey: examples (1)

How useful do you think each of the online tools might be in helping you to complete this task?

Task 5: Specify a set of requirements in a given context

0

1

2

3

4

5

6

Very useful (1) (2) (3) (4) Not at all useful (5)N

o o

f re

sp

on

se

sChat Forum Wiki Blog Journal

Task 9: Use the OU Library and other online resources to identify papers

0

1

2

3

4

5

6

Very useful (1) (2) (3) (4) Not at all useful (5)

No

of

res

po

ns

es

Chat Forum Wiki Blog Journal

6

Survey: requirements• Students were

asked to identify up to three requirements for online activities

• Responses analysed using NVivo

7

Survey: examples (2)• Task (description)

– “A clear ‘framework for use’ is circulated prior to the exercise”– “The criteria for completion of the tasks should be clearly

stated”

• Group (identification of others)– “Ability to easily see who wrote what and when (depends on

nature of activity)”

• Individual (learning)– “Online activities should back up what I’ve learned from reading

the course materials”

• System (usability)– “Easy and reliable means of creating and editing contributions”

8

Pilot study

• Group of volunteers from May 07 presentation– Six students (3 male and 3 female)– Undertook to take part in one or more activities and provide

feedback

• Separate VLE space provided, access limited to volunteers and researcher

• Five activities1. Introductions (General Forum)

2. Evaluating a paper (Wiki)

3. Writing a critical summary (Q & A Forum)

4. Comparing styles – two extracts (Wiki)

5. How to criticise – three papers (Q & A Forum)

9

Summary of activitiesNumber of student activities logged by VLE between 12 June and 9 October 07

Student Active Passive TotalActive

%

A 27 145 172 16

C 7 186 193 4

H 1 18 19 5

K 8 119 127 6

M 21 385 406 5

R 1 15 16 6Active = add, delete, update, edit

Passive = view

10

VLE log: example

Original includes links to relevant forum and wiki pages

Course Time IP AddressFull name Action Information

JMJ38-WORKSPACE2007 July 3 19:43 A wiki info Activity 2: Evaluating a paper: Evaluating a paperJMJ38-WORKSPACE2007 July 3 19:35 A wiki view Activity 2: Evaluating a paper: Evaluating a paperJMJ38-WORKSPACE2007 July 3 19:33 A forum view discussion 3035JMJ38-WORKSPACE2007 July 3 19:33 A forum add post 10428JMJ38-WORKSPACE2007 July 3 19:04 A forum view discussion 3035JMJ38-WORKSPACE2007 July 3 18:52 A wiki view Activity 2: Evaluating a paper: Evaluating a paperJMJ38-WORKSPACE2007 July 3 18:51 A forum view discussion 3035JMJ38-WORKSPACE2007 July 3 18:51 A forum add post 10426JMJ38-WORKSPACE2007 July 3 18:49 A forum view discussion 3035JMJ38-WORKSPACE2007 July 3 18:49 A forum add post 10425JMJ38-WORKSPACE2007 July 3 18:47 A forum view discussion 3035JMJ38-WORKSPACE2007 July 3 18:47 A forum view forum 1720JMJ38-WORKSPACE2007 July 3 18:46 A course view 3083JMJ38-WORKSPACE2007 July 3 17:36 M wiki view Activity 2: Evaluating a paper: Evaluating a paperJMJ38-WORKSPACE2007 July 3 17:36 M wiki info Activity 2: Evaluating a paper: Evaluating a paperJMJ38-WORKSPACE2007 July 3 17:36 M wiki view Activity 2: Evaluating a paper: Evaluating a paperJMJ38-WORKSPACE2007 July 3 17:36 M course view 3083JMJ38-WORKSPACE2007 July 3 17:36 M forum view discussion 3035JMJ38-WORKSPACE2007 July 3 17:35 M forum view forum 1720JMJ38-WORKSPACE2007 July 3 17:35 M course view 3083

11

Example: activity

Activity 3: Writing a critical summary1. What is the main question addressed in this paper? Is this

question important? If so, why?

2. What is the significance of the findings in this paper for requirements engineering?

3. Do the findings help explain the phenomenon under study and make predictions about it?

4. Are the findings only true for the particular example(s) studied? Or is there reason to believe they will apply in other cases?

5. Does this paper provide a significant contribution to the subject area?

12

Example: interactions

Figures in circles are the number of active contributions to the discussion on a given day

Visualisation of Moodle logs

0

5

10

15

20

25

30

35

40

45

50

30/0

7/20

07

31/0

7/20

07

01/0

8/20

07

02/0

8/20

07

03/0

8/20

07

04/0

8/20

07

05/0

8/20

07

06/0

8/20

07

07/0

8/20

07

08/0

8/20

07

09/0

8/20

07

10/0

8/20

07

11/0

8/20

07

12/0

8/20

07

13/0

8/20

07

14/0

8/20

07

15/0

8/20

07

16/0

8/20

07

17/0

8/20

07

18/0

8/20

07

No

of

acti

on

s

A C H K M

Activity 3 started

7

2

1

2

1

1

1

1

2

3

6

13

Examples: extracts

Comments on Activity 3• “I swore I would not just paraphrase the paper but, in

Q3, I think I just have.” (A)• “Well I've had a go, and yes, it would probably have

been better to do questions 1 and 2 first!” (M)• “Wish i'd answered them in order and read them through

first to make sure they flowed into a complete critique. Also found it hard to evaluate with respect to the subject area when my knowledge of it is so limited.” (K)

14

Conclusions and future plans

• Volunteers found activities helpful in preparing for assignments and exams

• Requirements– Ease of use– Ability to see who wrote what and when

• Preferred three or four activities over course lifetime, using discussion forums

• Stage 3: three optional activities in May 08 presentation• Evaluation

– Examination of VLE logs

– Telephone interviews