Online Environment Evaluation

8/14/2019 Online Environment Evaluation

1/25

Evaluating students' performance in discussion

boards in Thinking and Creativity blended course

at Sultan Qaboos University

Hiba AL-Julandani Tamadher AL-Za'abi

usueduom usueduom

Instructional and Learning Technologies Department, College of Education,

Sultan Qaboos University

May ,


2/25

Abstract

This paper examines students' performance in asynchronous discussion in Thinkingand Creativity blended course. Specifically, students' participation in terms of number ofmessages posted in different discussion boards. Moreover, the quality of messagesposted is also examined. Students' participation is observed throughout the course to

address the number of messages posted by each individual student. The quality ofmessages is examined according to a checklist that contains items about message start,opinion responses, clarity and topic context.

Introduction

As we are moving fast in the digital age, here at Sultan Qaboos University manyapplications have been implemented by academics. The entrance of online learningthrough moodle and WebCT has influenced students learning and performance. Usingmoodle allows asynchronous communication between students and instructors. In the

discussion boards, an instructor and students in the course share their thoughts and ideaswith each other and essentially engage in "conversation. This is an important aspect offully online courses. But, what about blended courses. Usually, the more interactionbetween and among the students, instructor, and course content, the more successful thedistance course. In the case of blended courses, there should be a balance between classactivities and online activities. Repeating same activities makes learning process boring forboth instructor and students.

Participants and context of the study

Participants are 240 students. The participants are enrolled in three sections of anelective two-credit course entitled Thinking and Creativity, spring 2009 at Sultan QaboosUniversity. This course addresses the development of human intelligences and discussesthe recent researches in this filed. It aims to improve students' creativity through helpingthem to identify their learning styles, types and through helping them to draw conceptmaps that leads to improve their cognition and meta-cognition abilities. Moreover, thiscourse addresses some success techniques. This course is a blended course. The e-partof the course is delivered using moodle. Students have to have an enrolment key from thecourse tutor to access the course.


3/25

Course Home Page

Forums Page


4/25

Topic (4) page

Purposes of the study

This short paper aims to:

1. Specify the degree of student participation in discussion forums in terms of number ofmessages posted by each student.

2. Analyze the quality of messages in two topics of Thinking and Creativity blendedcourse posted by students in terms of Self-introduction, Repeating information in anothermessage, responding to accept or reject others views and opinions without explanation,

responding to accept or reject others views and opinions with more explanation, contextrelated, clarity, grammar and provocative.

Questions of the study:

1. How many messages are posted in the whole discussion boards and by eachindividual student?

2. For which extent is the quality of messages posted by students in terms of Self-introduction, Repeating information in another message, responding to accept orreject others views and opinions without explanation, responding to accept or reject

others views and opinions with more explanation, context related, clarity, grammarand provocative?


5/25

Evaluation Methodology

These researchers will utilize a quantitative method in this study to find out the totalnumber of messages posted by student. Moreover, a qualitative methodology will be

applied to evaluate the quality of messages posted by students. A checklist will be used toanalyze messages' contents. Checklist consists of seven items that evaluate the messagefrom different aspects. First, the checklist tends to examine if the student introduceshim/herself at the beginning of the post. The second item examines repeating informationin another message. The third and fourth items examine the student's response to acceptor reject other's views with or without explanation. The rest examine the message clarity,context related and correct grammar. Moreover, the checklist examines massageproactive. Proactive means that the message prompts other student to respond.

Results and Discussion

Question 1 finding:

Question 1 was: How many messages are posted in the whole discussion boardsand by each individual student? For answering this question the discussion boards havebeen observed throughout the course. The average number of messages sent by studentsduring studying the whole course includes:

1. The total number of messages posted to the discussion boards

In this course, seven discussion topics were addressed by the course tutor. Resultsfrom observing the discussion board showed that the total responses of students'messages are 878 through out the course. Assuming that each student from those whoenrolled in the course (240 students) should participate at least by posting one message toeach topic, this number (878) represents only 52.3% of the expected total number ofmessages (1680) that should be sent to the discussion boards (Table 1).

The total number of messages posted to the discussion boards (Table 1)Number ofstudents

Total number of messagesassumed to be sent

Actual total number ofmessages sent by

students

Percent

240 7*240= 1680 878 52.3%


6/25

2. The total number of messages posted to each topic, number of posts and

number of replies to discussion board:The number of messages posted to each topic differs from topic to other topics. This

difference may be as a result of how much students like the topic. In addition, the degree

of students' motivation to reply and post differ from topic to another according to theirinterests. Table 2 shows the total number of messages posted to each topic. It is clear thatSuccess Secrets topic has the highest number of posts (132) and the highest number ofreplies (317). However, topic No.6 that asks students to critique a video clip for Dr. AL-Faqi has the lowest number of replies.

The total number of messages posted to each topic, number of posts and number of

replies to discussion board (Table 2)

TopicNumber

Topic title Number of posts Number of replies

1. - 1 45

.2 95 140

.3 132 317

4. 1 38

5. 13 19

6. . 13 12

7. : 1 51

Sub Total 256 622

Total 878

3. The minimum and maximum number of messages per topic

As discussion boards have been observed, we found that the minimum number ofmessages per topic posted is 25 messages, whereas the maximum number of messagesper topic posted per topic is 449 messages as shown in table 3.


7/25

The minimum and maximum number of messages per topic (Table 3)

Number ofmessages per topic

N Minimum Maximum Mean

7 25 449 125.42

4. The number of messages posted per student

Students' responses varied from ignoring all topics or some topics in the discussion

boards to positive involvement. The minimum number of messages per student sent to

the discussion boards was zero (Table 4). Table 5 shows that the majority of students

posted between one and thirty messages. Around 112 students choose not to respond

to any topic in discussion boards.

The minimum and maximum number of messages per student (Table 4)

Number ofmessages per

student

N Minimum Maximum Mean

240 0 74 3.56

The number of messages per student(Table 5)


student

Frequency Percent ofstudents

0 112 47%

1 20 8%

2 27 11%

3 15 6%

4 9 4%

5 6 3%

6 9 4%

7 2 1%

8 8 3%

9 6 3%


8/25


student

Frequency Percent ofstudents

10 2 1%

11 4 2%

12 2 1%

13 3 1%

14 2 1%

15 2 1%

16 1 0%

17 2 1%

20 2 1%

22 1 0%

25 1 0%

29 2 1%

30 1 0%

74 1 0%

Question 2 finding:

Question 2 was: For which extent is the quality of messages posted by students intwo topics in terms of Self-introduction, Repeating information in another message,responding to accept or reject others views and opinions without explanation, respondingto accept or reject others views and opinions with more explanation, context related,clarity, grammar and provocative?

Topic1:

Topic one asks students to post references of human resources development. Thistopic has 45 replies. Table 6 shows frequencies of messages after evaluating posts

quality.


9/25

Table 6

INT_SEL

INT_SEL

noyesFrequency

40

30

20

10

0

Chart (1) Number of messages that contain self introduction in topic No (1)

Around nine messages contain self introduction before the real post. Some examples forthe replies that have self-introduction are:

Frequency

(Grammer)

Frequency

(Prov)

Frequency

(Clear)

Frequency

(Related)

Frequency

(Int-Acex)

Frequency

(Int-Acc)

Frequency

(Int-Rep)

Frequency

(Int-sel)

yes 36 36 45 22 1 44 1 9no 9 9 23 44 1 44 36

Total

45 45 45 45 45 45 45


10/25

INT_REP

INT_REP

noyesF

requency

50

40

30

20

10

0

Chart (2) Number of messages that contain repeated statements from anothermessages in topic No (1)

The chart shows that one message only contains repeated statement. That isbecause students tend to post their responses to the original topic directly without reading

or commenting on others posts.


11/25

INT_ACC

INT_ACC

noyesF

requency

40

30

20

10

0

Chart (3) number of messages that shows students replies to accept or rejectothers views and opinions without explanation in topic No (1)

The chart shows that most students responded to accept or reject others'views andopinions without explanation. Example of message that accept others' view without

explanation:


12/25

INT_ACEX

INT_ACEX

noyesF

requency

50

40

30

20

10

0

Chart (4) number of messages that shows students replies to accept or rejectothers views and opinions with explanation in topic No (1)

The chart shows that most students responded to accept or reject others'views andopinions without explanation. Example of message that accept others' view with

explanation:


13/25

RELATED

RELATED

noyesF

requency

30

20

10

0

Chart (5) Number of messages that are related to topic context in topic No (1)

Around 22 messages are related to the context of the topic posted by the tutor. However,around 23 messages are not related to the topic context.

CLEAR

CLEAR

yes

Frequency

50

40

30

20

10

0

Chart (6) Number of clear messages in topic No (1)

The chart shows that all messages posted by students are clear.


14/25

PROV

PROV

noyes

Frequency

40

30

20

10

0

Chart (7) Number of messages that prompt students to respond in topic No (1)

Around 36 messages are provocative, whereas the rest are not.


15/25

GRAMMER

GRAMMER

noyesF

requency

40

30

20

10

0

Chart (8) number of grammatically correct messages in topic No (1)

The chart shows that around 36 messages are grammatically correct. That is may bebecause most messages are copied from different sources without any changes.

Topic2:

This topic asks students to post some stories about successful people. This topic has 38replies. Table 7 shows frequencies of messages after evaluating posts quality.

Table (7)

Frequency

(Grammer)

Frequency

(Prov)

Frequency

(Clear)

Frequency

(Related)

Frequency

(Int-Acex)

Frequency

(Int-Acc)

Frequency

(Int-Rep)

Frequency

(Int-sel)

yes 35 30 32 28 4 2 9no 3 8 6 10 38 34 36 29

Total

38 38 38 38

38 38 38


16/25

Chart (9) Number of messages that contain self introduction in topic No (4)

Most of the replies does not have self introduction. Some examples for the repliesthat have self-introduction are:

INT_SEL

INT_SEL

noyesFrequen

cy

40

30

20

10

0

*

*

- - *

*

*


17/25

Chart (10) Number of messages that contain repeated statements from

another messages in topic No (4)

There are about 2 repeated replies in this topic and this may be by mistakes from thestudents because they were done by the same students and at the same time with thepost which is repeated from.

INT_REP

INT_REP

noyesFrequen

cy

40

30

20

10

0


18/25

Chart (11) number of messages that shows students replies to accept or reject

others views and opinions without explanation in topic No (4)

INT_ACC

INT_ACC

noyesFrequen

cy

40

30

20

10

0


19/25

Chart (12) number of messages that shows students replies to accept or

reject others views and opinions with explanation in topic No (4)

There are about 4 students who comments for other replies by agree or disagreewithout explanation and no replies by agree or disagree with explanation. This maybecause of the nature of the topic which is written real stories of people success in theirlife. Because the stories are real there is no agree or disagree. Almost of the replies aredifferent stories rather than comments for others. Some example for those replies is below:

INT_ACEX

INT_ACEX

noFrequen

cy

40

30

20

10

0

...

..

...


20/25

Chart (13) Number of messages that are related to topic context in topic No(4)

Ten replies from (38) replies are unrelated to the main topic. One example ofthe unrelated replies is below:

RELATED

RELATED

noyesFrequen

cy

30

20

10

0

.


21/25

This reply is not related to the topic and it is not related to others replies. Most ofthe unrelated replies are taking about tips in how to success in life which is interfered withthe main topic. So, it maybe the misconception of the topic is the reason.

Chart (14) Number of clear messages in topic No (4)

Most of the replies in this topic are clear accept about 5 of them. They are not clearbecause they are not related to the topic and not related to other replies which means thatthey are not meaningful.

CLEAR

CLEAR

noyesFrequency

40

30

20

10

0


22/25

Chart (15) Number of messages that prompt students to respond in topic No (4)

Most of the replies are provocative which means that they encourage othersto reply and read them. This may be because of the nature of the topic. The mostprovocative replies are those which talk about stories about successful Omani people. Thismay be because they are more related to the participants' context and culture.

PROV

PROV

noyesFrequency

40

30

20

10

0


23/25

Chart (8) number of grammatically correct messages in topic No (1)

Most of the replies in this topic are written in good grammar and using correct Arabiclanguage.

Pros and Cons of using discussion boards in blendes courses

Instructors in face-to-face universities should think carefully before integrating

online learning for their students. One advantage of using such technology is that it allowsasynchronous communication between classmates and course tutor. However, studentswho prefer face-to-to-face communication can not respond effectively in discussionboards. Moreover, instructors should specify the evaluation criteria for students at thebeginning of the course to lead the discussion messages posted by students. In addition,course instructor should participate on the discussion boards and comments on students'posts which will influence and motivate students to respond.

GRAMMER

GRAMMER

noyesFrequen

cy

40

30

20

10

0


24/25

References:

1. http://www.mirandanet.ac.uk/ejournal/uploads/48/Critique.pdf

2. www.nnu.edu/fileadmin/Elearning/bb_tips/BbAssess_EvaluateDiscussion.pdf3. www.stjohns.edu/media/3/c84d5c3eb0f54c19baba0ba5e59af116.pdf4. http://www.surveyconsole.com/console/TakeSurvey?id=570745


25/25

Appendices

Appendix I : checklist

N Item Yes No1. The student introduces him/herself at the beginning of the post

2. The student repeats information in another message.

3. The student Responds to accept or reject others views and opinionswithout explanation.

4. The student Responds to accept or reject others views and opinions withmore explanation.

5. The student replies to the post in the same context.

6. the student reply is clear

7. The student reply stimulate others responses.

8. The student reply is grammatically correct.

Documents

Online Environment Evaluation