Reviewed Papers

inroads – The SIGCSE Bulletin 39 Volume 35, Number 4, 2003 December

An Experience on Students’ Participation in Blended vs. Online Styles of Learning

Juan Manuel Dodero, Camino Fernández, and Daniel Sanz DEI Laboratory, Computer Science Department

Universidad Carlos III de Madrid Av. Universidad 30

Leganés 28911, Madrid, Spain {dodero,camino,dsanz}@dei.inf.uc3m.es

Abstract

This work compares two learning experiences developed with different styles during an academic semester in two universities. The objective of the study has been to test the advantages of the blended style of learning, in terms of students' participation and initiative in the learning process, compared with those of pure virtual e-learning. The study shows how information technologies act as an incentive to improve students' participation during traditional classroom-located teaching, but do not help to increase their participation when the learning process is completely virtual and not complemented by regular classes.

1. Introduction During the experience, two groups of students were studied: the first group was a course on Object Oriented Programming (OOP) in a traditional university Computer Science (CS) faculty, whereas the second was a course on OOP Basics in a virtual university CS studies. The former was developed according to a mixed style of both traditional and online learning also known as blended learning [1,3] and the latter was guided by a completely virtual methodology and process development. The blended experience took place during a semester of the last year of CS studies. Besides traditional teaching, along with practical classes in computer rooms, the course was enhanced by using a web forum where specialized groups on small parts of the course contents could help the rest of the class. The evaluation of the course took into account both the traditional written exam including theory and practice, and the role of students in the course as part of the specialized groups and as users of the forum, both stating questions and answering them. The pure virtual experience was held during a semester of CS studies in a virtual university. The course was supported by a virtual campus web application, which included discussion for a besides regular communication tools like e-mail, bulletin boards, and electronic delivery applications. For the evaluation, the completion of a set of assigned activities in a continuous evaluation basis and the fulfillment of a written, non-virtual exam were taken into account. Students’ performance and initiative in the forum were also considered as evaluation parameters. In order to analyze the participation of students, varied measurements were compiled in the discussion fora, concerning the types of messages posted by students, and the class of events that encouraged them to post such messages (including messages from other class/forum

mates and task assignments from the instructor). Such measurements have taken into account the accomplishment of assigned tasks such as the posting of complementary digital resources that might help other colleagues in realizing their tasks, the answering to questions, and posing new ones. Results have shown that students' participation in the fora is promoted for the blended style of learning. In the same way, students' participation and own initiative in the learning process were improved when combining physically-commanded task assignments with a positive assessment for the development of assigned tasks, which were asynchronously fulfilled by students. The experience showed also that the same electronic tools used in a pure e-learning course do not have the same effect on students participation because they mainly understand the learning process as a task to be developed on their own in communication with the teacher and almost no contact with other students. 2. Blended Framework The subject Advanced Programming of CS studies at a traditional, “presential” University was chosen as the blended-styled framework. Besides traditional teaching on the blackboard using slides, and practical classes in the computer rooms, the course was enhanced by organizing students in groups specialized in different parts of the course contents. Taking theoretical classes as an starting point, students were assigned optional tasks that they could complete to receive positive assessments. In these Tutor-Marked Assignments (TMA) [4], students are first in charge of complementing the teacher explanations with programmed practical examples, and secondly requested to solve the related questions that are sent by their colleagues and by the tutor to the course forum.

Reviewed Papers

inroads – The SIGCSE Bulletin 40 Volume 35, Number 4, 2003 December

The teacher supervised the behavior of groups during the whole semester, preventing them from making severe deviations from assigned tasks and also complementing the discussions produced. Both in regular classes and in the forum, the teacher stated standard problems to be solved by each group. This way, there is a chance for the teacher to observe the class evolution and point out concrete, unclear aspects. An open-source, readily available web forum was used as the tool where the course evolution was logged in detail [2]. Every course material was also available in the web forum, including the TMAs developed by the specialized groups together with discussions that were posed in regular classes. That software tool was adapted to authenticate students for the teachers to be able to evaluate their contributions. The course evaluation had taken into account four elements: An optional TMA that students attending to regular classes received.

A compulsory TMA that students must complete, which consists in the object-oriented development of a reusable piece of software in Java language.

The participation of every specialized group in the forum, making example proposals and answering to questions, as well as the active roles of students during the TMA that were assigned to them.

A writing exam including both theoretical and practical concepts that is held at the end of the semester.

3. Pure Virtual Framework The subject Programming Fundamentals II of CS studies in a Virtual Open University was chosen as the pure virtual scenario of the study. The educational process is based upon learning materials, a virtual campus, and continuous assessment. Learning materials include didactic units in both paper and multimedia form, as well as software material. The virtual campus is a web-based telematic application, intended to overcome time and space barriers facilitating communication among members of the virtual classrooms and work in a cooperative way. The virtual classroom provides students with electronic documents that plan the requested advance in the subject and give them study guides for each didactic unit, respectively. Besides that, several interaction fora are available for tutor announcements, general-purpose but subject-oriented online discussions, and cooperative assignments realization. Continuous Assessment (CA) allows students to face with knowledge, solve problems, and with the help of their tutors, they can verify their learning progress. CA is not compulsory, so students that cannot keep up with it or do not pass it must do a final evaluation test. The main task of tutors includes guiding and encouraging students, as well as evaluating their progress in the learning process for the subject. The major evaluation instruments to achieve that

aim were also Tutor-Marked Assignments (TMA). These are practical tasks and they are made up, published, and marked by tutors to track and evaluate the progress of students. We divided the course evaluation into several parts: A set of optional TMAs A compulsory TMA that students are required to complete in order to pass the subject. It consists in the development of a mid-sized software artifact in Java language by two-person groups in a cooperative fashion.

A writing, “presential” exam that includes both theoretical and practical concepts, held at the end of the semester. Students can substitute this exam by a more brief and specific validation test, whenever they have passed the CA successfully.

The level and quality of students’ participation in specialized fora is also considered as an evaluation factor.

4. Compared Study We began the methodology of the work by selecting two subjects and two groups of students for the study. Both groups were the same size (50 students approximately) and featured similar issues, related in Table 1. Table 1 Features of both populations for the study

Blended framework Pure virtual framework The subject takes place in the last year of studies, when the vast majority of students are working and cannot attend classes regularly, so they are offered a kind of distance support that is supposed to be positively received

Most students have their job and cannot attend presential classes so they were motivated to enrol it. Therein distance help is the only available form of support.

In the last year of CS studies, students can be considered perfectly acquainted with computers and electronic resources available through the Internet.

The subject is compulsorily studied as a second-year subject, after studying first-year subject Programming Fundamentals I and an introductory course to the virtual campus. Since all these are Internet-based courses, students are perfectly acquainted with computers and available electronic resources.

There are nearly 50 students attending classes on the subject.

Students' interaction in electronic fora is limited to 50-person virtual classrooms.

Students evaluation considers their active participation in teaching sessions and the forum, and the fulfillment of optional TMAs. Due to the practical nature of the subject, the evaluation includes a compulsory TMA.

Students evaluation considers their active participation in the fora, and the fulfillment of optional TMAs. Due to the practical nature of the subject, the evaluation includes a compulsory TMA.

An electronic assignment and delivery application was installed to facilitate the automatic delivery of TMAs to the tutor, in order to equalize the helper facilities with those available in the pure virtual experience.

The electronic assignment and delivery system of the virtual campus facilitates the convenient submission/pick up of TMAs to/from the tutor's private mailbox.

Reviewed Papers

inroads – The SIGCSE Bulletin 41 Volume 35, Number 4, 2003 December

Above all, evaluation criteria were selected as similar as possible in both frameworks, in order to ease the differences that several assessment demands could impose. Each group behavior was conducted by the tutor according to blended and pure virtual styles of learning, respectively. Tutors act in both cases as catalysts of participation in the fora. To compare students' participation and success in both frameworks, the following measurements were considered: Participation level: percentage of students that presented and completed their assignments optional TMA, compulsory TMA, and final exam

Pass level: percentage of students that completed and passed their assignments optional TMA, compulsory TMA, and final exam

Results showing the attainment of students with

respect to both measurements are depicted in figure 1 (participation level) and figure 2 (pass level), with respect to compulsory TMA, optional TMA, and final evaluation. It has to be pointed out that results depicted in last bar (named Final) include results from the two previous (named CTMA and TMA). We did not measured the final exam alone since there are a lot of interrelated parameters to be considered when eventually evaluating a student. As showed in figure 1, the participation level is higher when the blended style of learning is deployed, for optional and compulsory TMAs as well as for the final evaluation. Under the hypothesis of having arranged the same assessment scheme and provided equivalent tools for the development of the experience, we consider the influence of “presentiality” factor of the tutor as decisive in the increase of the participation level. With respect to the success in passing the subject we provide two different percentages, one measured upon the total number of registered students (see figure 2a), and another measured upon the number of students that completed the assignments (see figure 2b). In the first case, we can observe that the final evaluation results (which included results from continuous assessment and the final exam) are alike in both frameworks. Therefore, we stand up for the existence of two alternative ways to attain the same achievements, but with different levels of participation. However, in the second case, although the pass level keeps equal for the two kinds of TMA, the results for the blended style fall under those from the pure virtual one. This can be given the following interpretation: If you consider only people that actively participate in the continuous assessment, the influence of an exam is lesser in the final pass level of students.

Figure 1. Percentage of students that completed Tutor-Marked Assignments (TMA), Compulsory (CTMA), and final evaluation

Figure 2. Percentage of students that passed Tutor-Marked Assignments (TMA), Compulsory (CTMA), and final evaluation, measured (a) upon all enrolled students and (b) upon presented ones 5. Conclusions In this paper, we presented an study to compare the level of participation and the final achievements of two groups of students attending the same course in two different settings, i.e. according to a blended style of learning vs. a pure virtual style. Both courses were about similar subjects and developed with equivalent facilities and evaluation criteria. Results have shown that the holding of a limited set of regular classes in the blended framework improved the level of participation, but did not influence the overall qualification results. Nevertheless, when considering only people that actively participated in the continuous assessment, the style of learning had not much influence in the final qualification results.

Reviewed Papers

inroads – The SIGCSE Bulletin 42 Volume 35, Number 4, 2003 December

References [1] Douglis, F. “Blended learning: Choosing the right blend”, In Encyclopedia of Educational Technology,

http://coe.sdsu.edu/eet/Articles/blendlearning/index.htm, 1993-2003. [2] Landingin, B., and Withers, C. “Squishdot: The Open Source Discussion Forum Software for Zope”, http://squishdot.org, 1999-2001. [3] Masie, E. “Blended Learning: The Magic is in the Mix”, In A. Rossett (ed.), The ASTD E-Learning Handbook, McGraw-Hill, 2002.

[4] Thomas, P., and Taylor, D. “Reducing the Distance in Distance Education”, In H. Cooper and S. Clowes (eds.), Proceedings of the 4th International Computer Assisted Assessment Conference, Loughborough University, June 2000.

Computing Curriculum 2001 (CC 2001)

Computer Engineering

<http://www.eng.auburn.edu/ece/CCCE>

Computer Science

<computer.org/education/cc2001/>

Information Systems

< http://www.acm.org/education/curricula.html>

Software Engineering

< http://www.acm.org/education/curricula.html>