10

Click here to load reader

Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

Embed Size (px)

Citation preview

Page 1: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

1 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

Developing Undergraduate Students’ Multiliteracies with

an Outcomes-based Education Approach

Samuel Chu, Kai Wah Faculty of Education

The University of Hong Kong [email protected]

Randolph Chan, Chun Ho Faculty of Education

The University of Hong Kong [email protected]

Celina Lee, Wing Yi Faculty of Education

The University of Hong Kong [email protected]

ABSTRACT

This study examined the effectiveness of applying

outcomes-based education (OBE) in developing

information literacy, computer software literacy, and Web

2.0 literacy (collectively termed as multiliteracies) among a

group of undergraduate students at the University of Hong

Kong. These BSc Information Management (BScIM)

students completed three paper-based perceptual surveys,

which assessed their perceived importance and familiarity

of the three kinds of literacy, upon entry into the program

and again after each academic year. Individual interviews

were conducted to further document their learning

experience. Preliminary findings indicated that students

improved in the three forms of literacy after each academic

year. The findings suggest a positive influence of OBE on

students‟ attainment of multiliteracies and give insights on

improving the implementation of OBE for students‟

learning.

KEYWORDS

Outcomes-based education, tertiary education,

multiliteracies, 21st century skills

INTRODUCTION

In the 21st century, the literacy landscape has changed from

a print-saturated system to a multimodal semiotic system

(Iyer & Luke, 2010) owing to the rapid technology

advancement and globalization (Cazden, 1996). As a

response to such transformation, multiliteracies have been

regarded as increasingly important skills to be acquired by

students in the 21st century society. The term

„multiliteracies‟ was coined by the New London Group in

1996, and despite the ongoing discussion surrounding this

concept, scholars and researchers have yet to reach a

consensus over its definition. Westby (2010) has broadly

defined multiliteracies to be consisted of any literacy that

extends beyond the conventional areas of reading and

writing skills.

With the emerging emphasis on multiliteracies, educators

have been struggling to find an effective pedagogy to help

students develop these knowledge and skills. As the more

traditional didactic approach encourages a unidirectional

knowledge transmission (Chang, Jones & Kunnemeyer,

2002), it appears to be incongruent with multiliteracies,

which could not be developed through rote memorization.

Teachers tend to lack experience and confidence in

teaching multiliteracies (Rowsell et al., 2008) as it is a

relatively new education focus. Moreover, as the term

„multiliteracies‟ remains broadly defined, teachers may

have vague ideas of what knowledge and skills exactly are

needed.

With the challenges faced by educators in facilitating the

development of multiliteracies among students, this study

tested the feasibility of the outcomes-based education

(OBE) as a pedagogical approach. In this study, we focused

on three specific multiliteracies: information literacy,

computer software literacy, and Web 2.0 literacy. The

familiarity and importance of the literacy components as

perceived by a group of undergraduate students in the BSc

Information Management (BScIM) program in the

University of Hong Kong was measured. It is expected that

there is a linear relationship between the perceived

importance and familiarity. If students become more

familiar with a kind of literacy, they will tend to perceive

that as more important. Changes in students‟ perceptions on

multiliteracies were examined. Based on the findings of

this study, suggestions on how to maximize the potential of

OBE are given.

LITERATURE REVIEW Multiliteracies – an overview

The New London Group (1996) pointed out that with the

rapid development caused by globalization, technology and

social diversity, the traditional language-based education

approaches required a breakthrough change. Since then, a

new education focus has emerged, with multiliteracies

being regarded as important skills to be acquired. Despite

the rapid growth of interest and emphasis put on

„multiliteracies‟, definitions have varied. In Tynes‟ (1998)

interpretation, multiliteracies include amongst others,

computer literacy, networking literacy, technology literacy,

information literacy, media literacy and visual literacy.

Anstey and Bull (2006) defined a multiliterate person as

one who “is flexible and strategic and can understand and

use literacy and literate practices with a range of texts and

This is the space reserved for copyright notices.

ASIST 2012, October 26-31, 2012, Baltimore, MD, USA.

Copyright notice continues right here.

Page 2: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

2 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

technologies; in socially responsible ways; in a socially,

culturally, and linguistically diverse world; and to fully

participate in life as an active and informed citizen” (p. 55).

More recently, Westby (2010) emphasized that literacy has

to be extended beyond the conventional areas of reading

and writing skills.

Information Literacy

Amongst a wide range of components comprising

multiliteracies, information literacy has been widely

discussed (Bawden, 2001; Bruce, 1997; Catts & Lau, 2008;

Hepworth, 1999; Huvila, 2011; Spitzer et al., 1998; Webber

& Johnston, 2000). Sawetrattanasatian (2008) summarized

that the central features of information literacy include (1)

searching for information effectively and efficiently, (2)

evaluating information sources critically, (3) organizing

and using information properly and ethically, and (4)

contributing new ideas and knowledge. In this study, these

common features of information literacy were considered

in assessing students‟ perceptions.

Computer software literacy

Haigh (1985) broadly defined computer literacy as the

knowledge and skills a person needs to acquire, to work

and live in the society. It is also defined by Martin and

Grudziecki as knowing how to use applications and

computers for practical purposes (as cited in Covello, 2010).

Despite the significant increase in the use of computers in

classrooms over the past few decades, the term „computer

literacy‟ has remained poorly defined (Goodson & Mangan,

1996). In this study, computer software literacy refers to

the ability to use computer software purposefully.

Web 2.0 literacy

Web 2.0 is defined by Chiang et al. (2009) as an umbrella

term to explain the various Web developments and its key

concepts, including collaboration, user participation, file

sharing, social networking and rich user experience. This

study examines Web 2.0 literacy as it refers to the ability to

make use of technology such as blogs and social

networking tools to interact with other users.

Multiliteracies and students’ learning ability

One of the reasons behind the importance of multiliteracies

comes from their potential to enhance students‟ learning

experiences. Many researchers have suggested that students

with higher level of multiliteracies are able to utilize

learning opportunities better. Alexander (2008) suggested

that with the use of Web 2.0 tools in learning, students

develop information literacy that allows them to explore,

critique and learn from a rich reservoir of links and

resources. He also suggested that a higher level of critical

thinking skill is required from students to sift through

materials using search engines. He further claimed that the

open Web has a positive effect on learning as students have

more opportunities to write. Similarly, Dang and Robertson

(2010) pointed out the benefits of Web 2.0 applications

associated with increased opportunities for students to

express their ideas, hence enhancing their confidence and

engagement in learning activities. Chan and Cmor (2009)

also noted that students perceived blogs as avenues to learn

research and information skills, share knowledge with peers,

and consequently improve the quality of their assignments. Developing students’ multiliteracies

Tierney (2006) noted that educators struggle with teaching

multiliteracies. Webber (2000) suggested that in order to

help students acquire multiliteracies, schools should allow

more time and space for the librarians, who are information

experts. Chu et al. (2011), in their study on the

development of multiliteracies of a group of primary school

students, suggested that the combination of collaborative

teaching that involved librarians, and inquiry project-based

learning contributes to the development of information

literacy and IT skills. They found that the characteristics of

these pedagogical methods, which require deep thinking,

the ability to apply knowledge, reasoning skills and in

depth exploration of issues, contribute towards the

development of multiliteracies.

Outcomes-based Education (OBE)

OBE has its roots from competency-based education, a

pedagogical approach introduced in the North America

during the 60s to cope with the concerns about students‟

lack of necessary competencies upon school completion

(Butler, 2004). Spady and Marshall (1994) defined OBE as

a teaching strategy that organizes and develops an

education system around a set of objectives to be achieved

by students upon completion of the learning experiences.

Hence, OBE puts more emphasis on the learning outcomes,

which are perceived to be more measurable than the more

personal learning process (Kovalik & Dalton, 1997).

Compared to the traditional didactic teaching approach,

OBE is student-centered (McDaniel, et al., 2000). Instead

of focusing on the knowledge transmission process from

teachers to students, OBE emphasizes the knowledge and

skills that students can learn from the lessons (Botha, 2002).

Being a student-oriented pedagogy, OBE requires teachers

to assume the role of facilitators who create learner-friendly

environments through teacher-student interactions, peer

interactions and the use of new technologies in the

classrooms (McDaniel, et al., 2000).

OBE had been adopted worldwide (Aldridge, et al., 2006;

Berman, 1995; Brindley, 2001; Rees, 2004; Shipley, 1995;

Wien & Dudley-Marling, 1998) and in all levels of

educational settings ranging from primary to tertiary

institutions. OBE had been widely employed in tertiary

education, especially in the subject areas of medical

education (Harden, 2002; Rees, 2004; Ross, 1999), food

science (Hartel & Gardner, 2003) and life sciences (Ryder,

2004).

The major component of OBE is the „outcomes‟ and

studies have defined and measured learning outcomes

(Butler, 2004; Faris, 1998; Hartel & Gardner, 2003; Jenkins

& Unwin, 1996; Kovalik & Dalton, 1997; Lorenzen, 1999;

Shipley, 1994; Spady, 1994). Faris (1998) defined learning

Page 3: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

3 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

outcomes as “clear statements specifying what a learner

should know, understand, and able to do” (p. 11). More

recently, Hartel and Gardner (2003) presented a more

detailed meaning of learning outcomes as “precise

statements of what faculty expects students to know and be

able to do as a result of completing a program, course, unit

or lesson” (p. 35).

Together with well-defined learning outcomes, the

effectiveness of OBE can be gauged by conducting

evaluations and assessments. The faculty members need to

critically review the instructional approaches in order to

recapitulate whether each learning outcomes statement is

sufficiently addressed (Hartel & Gardner, 2003). Minimum

learning outcomes achievement should be explicitly

identified for the students (Faris, 1998). Authentic and

tailor-made assessments should be developed (Lorenzen,

1999) in order to examine if the students are able to attain

the desired specific learning outcomes. Students‟ project-

based assignments in the forms of portfolios or course-

specific tests or examinations are some of the common

methods used in assessment (Brindley, 2001). A few

studies had also recommended the use of self-assessment

surveys in determining the students‟ attainment of course

level and program level learning outcomes (Brindley, 2001;

Hartel & Gardner, 2003; McCullough, 2008; Shipley,

1994). The suggestions by Lorenzen (1999) and Spady

(1994) for the successful implementation of OBE could be

summarized in three main stages:

1. Delineate learning outcomes to be achieved by

students;

2. Develop appropriate curriculum and incorporate

suitable instructional methods to help students achieve

the desired learning outcomes;

3. Assess students‟ attainment of learning outcomes

through various evaluation methods.

While OBE has been widely implemented, there have been

relatively few studies that utilized the approach specifically

for the development of multiliteracies. This study focuses

on the application of OBE and changes in students‟

multiliteracies.

RESEARCH QUESTIONS

This study implemented OBE over a 2-year period, and

changes in multiliteracies were examined among

undergraduate students. The implementation of OBE in

facilitating these changes was also evaluated. The desired

learning outcomes for information literacy, Web 2.0

literacy, and computer software literacy were identified,

and the following research questions were set to guide the

research process:

1. Did the students‟ perceptions of their learning

outcomes change over the 2-year period?

1. Information literacy

2. Web 2.0 literacy

3. Computer software literacy

2. Did students‟ learning expectation change over time

and did it meet with the learning outcomes?

3. How did the students and lecturers perceive OBE as a

pedagogy for facilitating the development of

multiliteracies?

PROCEDURE

This study employed a mixed methods research design,

combining quantitative and qualitative data to generate a

more comprehensive understanding of the research focus.

Instructional design

The BScIM (2009–2011) was a two-year full-time

undergraduate program. Students were required to

complete 16 courses, covering information management,

information retrieval, knowledge management, digital

libraries, database systems and other topics related to the

scope. Among the 16 compulsory courses, „Professional

experience‟ and „Project‟ are two courses that allow

students to apply the theories and knowledge they learnt in

other courses into real-life situations. Program outcomes

were devised by the curriculum design team and every

course was designed, in terms of teaching strategies and

subject knowledge. Information literacy, computer software

literacy and Web 2.0 literacy were identified as the generic

academic learning outcomes of the BScIM program. The

students were expected to have a progressive development

in all three kinds of literacy. To facilitate monitoring of

students‟ development, self-reports of learning outcomes

were administered.

The learning outcomes self-report of information literacy

consisted of a list of components that were categorized into

three parts: source types, electronic databases/internet

resources/ search engines, and information search

knowledge and skills. Similarly, the curriculum design

team also came up with a list of Web 2.0 applications

consisting of 8 items that refer to blogs, wikis and social

networking tools. Lastly, another list consisted of essential

software in information management. It was expected that

students should attain at least a familiarity of 3 (somewhat

familiar) in every item to be considered as having

successfully achieved the learning outcome.

Participants

This study involved two groups of participants: students

and the teaching staff. The student participants included 21

undergraduate students of the Faculty of Education of the

University of Hong Kong. They were the students who

joined the BScIM program in September 2009. They

received education in the field of information management

and technology studies. In addition to the student

participants, 7 teaching staff were interviewed. Participants

from the teaching staff were lecturers from the Division of

Information & Technoolgy Studies in the Faculty of

Education. They were involved in teaching the BScIM

students in the period of 2009 – 2011.

Page 4: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

4 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

Data collection and analysis

Three data collection methods were employed in this

research. Self-report assessments of learning outcomes

were administered to all participating students (n=21).

Telephone interviews with students (n=17) were conducted,

as well as individual face-to-face interviews with the

teaching staff (n=7).

Two self-report questionnaires were administered to the

undergraduate students in three phases of the study: before

(Sept 2009), during (Apr 2010) and after (Apr 2011)

undertaking the program. The first instrument aimed to

assess students‟ familiarity and perceptions of importance

of information literacy components. Students were asked to

evaluate their level of familiarity and perceptions of

importance of three dimensions: (a) source types, (b)

electronic databases/internet resources/search engines, and

(c) information search knowledge and skills. The second

instrument examined information management and

technology skills. It required the students to report their

familiarity and perceptions of importance of computer

software and Web 2.0 applications. Both questionnaires

utilized a five-point Likert-type scale of familiarity (1 being

“not familiar”, 5 being “very familiar”, and 0 as “don‟t

know”) and perceptions of importance (1 being “not

important”, 5 being “very important”, and 0 as “don‟t

know”). It was expected that students‟ familiarity should be

at least 3 (somewhat familiar) in order to be classified as

successful attainment of the learning outcome. It was also

expected that there would be a linear relationship between

students‟ familiarity and perceived importance.

Telephone interviews were conducted 6 months after the

completion of the program with 18 graduates from

November to December 2011. The interviews aimed to

probe deeper on the students‟ familiarity and perceptions of

importance of their multiliteracies after undertaking the

program. Open-ended questions were developed from the

results obtained from the self-reports. Students were asked

to further describe any perceived individual changes in the

level of multiliteracies. They were also asked to comment

on their learning experience under OBE.

Face-to-face interviews were conducted in January 2012

with teachers, who were asked to comment on their

students‟ change in multiliteracies throughout the 2-year

program. Open-ended questions were developed based on

the findings obtained from the student interviews. All

interviews were recorded and transcribed.

SPSS version 16.0 was used for all the statistical analyses.

Paired-sample t-tests was used to analyze and compare the

survey data obtained before, during and after the program

in order to determine if there is any change concerning

students‟ familiarity and perceived importance of

multiliteracies within the two years. Items to which the

respondents answered “don‟t know” were not included in

the analysis. The statistical significance level in this study

is set at 0.05, associating with a 95% confidence interval.

FINDINGS The purpose of the current study was to determine whether

students attained learning outcomes for multiliteracies

during the implementation of OBE in the BScIM program.

To investigate how students gained learning outcomes, the

researchers considered the changes in students‟ self-reports

of (a) information literacy, (b) computer software literacy,

and (c) Web 2.0 literacy.

Changes in Familiarity and Perceived Importance of Information Literacy Students' perceived importance of various source types

Respondents were surveyed on their perceived importance

of different potential sources of information. Several source

types, including journals (t=-4.183, p<.001) and statistics

sources (t=-3.23, p=.004), exhibited an overall

improvement in their perceived importance. Besides,

perceived importance of conference papers (t=-3.179,

p=.005) and encyclopedias (t=-2.225, p=.038) also

increased when comparing the second and third survey,

even the change between the first and second surveys was

not significant. This indicates that students came to

recognize their importance during the later stage of the

program. Overall, journals, conference papers,

encyclopedias and statistics sources had a significant

improvement in their perceived importance whereas the

perceived importance of consulting lecturers significantly

decreased (t=2.459, p=.024) comparing the first and the

third surveys.

Students’ familiarity and perceptions of importance of various electronic databases/internet resources/search engines

In this section, participants were asked to rate their

familiarity and perceived importance with various

resources and databases. The mean of the total ratings in

the third survey is 3.19 which means that students are

“somewhat familiar” with the listed electronic databases

and internet resources. Results of dependent t-tests

indicated that the perceived familiarity of CSA (t=-3.0,

p=.012), ERIC (t=-3.145, p=.006) and Google Scholar (t=-

2.447, p=.024) increased between the second and third

surveys. The study showed that seven items (EBSCOhost,

Eric, Google Scholar, Lexis-Nexis, Scopus, Web of

Science and Wise News) had an overall increase in their

familiarity (all with p<.05) comparing the first and the third

surveys.

On the other hand, multiple-disciplinary databases, such

EBSCOhost, Eric, Google Scholar and HKALL, increased

significantly in the perceived importance from the first and

third surveys (all with p<.05). Other items such as CSA,

LISA, ProQuest and Research Pro, also increased in their

corresponding ratings on perceived importance between the

second and third surveys (all with p<.05).

The paired-sample t-tests was computed to compare the

perceived familiarity and perceived importance of various

electronic databases during the third survey. As

demonstrated in Figure 1, six electronic databases,

Page 5: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

5 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

including Google Scholar, Lexis-Nexis, ProQuest, PsycInfo,

Web of Science and WorldCat, showed significant

differences in terms of their perceived familiarity and

perceived importance (all with p<.05). Students tended to

perceive these databases as important but had a relatively

low perceived familiarity with them.

Figure 1. Comparison between familiarity and perceived importance of various electronic databases/internet resources/search engines (posttest).

Students’ perceived familiarity and importance of various information search knowledge and skills

Participants were also required to rate a number of

information search knowledge and skills in terms of their

familiarity and perceived importance. The overall change

of perceived familiarity with various information search

knowledge and skills is significant when comparing the

first and third surveys (t=-3.002, p=.007). Students tended

to show greater improvement in the familiarity of five

information search knowledge and skills (AND, OR,

Parentheses, Wildcard, Proximity and Step 4: conduct a

search statement) between the first and third surveys (all

with p<.05).

In addition, the results indicated that there was an overall

increase of perceived importance from “somewhat

important” (M=3.72) to “important” (M=4.21) from the

first to third surveys. The perceived importance of primary

search knowledge and skills, including search by

Date/Year/Time period and Boolean Operators (AND, OR,

NOT) have increased when comparing the first and third

surveys (all with p<.05). On the other hand, there was also

a significant increase in the perceived importance of

advanced research knowledge and skills, including

Parentheses, Truncation, Proximity and Step 4: conducting

a search statement throughout the program (p<.05).

Similarly, in an interview graduate LYW expressed that “I

became more familiar with searching skills, with which I

can find more accurate information with less time. It helps

me work more efficiently in online information searching

and research”.

The paired-sample t-tests showed that in the third survey,

several information search knowledge and skills have

significant differences between perceived familiarity and

perceived importance (t=-3.824, p=.001). Seven items were

rated with high importance but low familiarity (field search

by author, field search by title, thesaurus, Step 1: identify

key concepts, Step 2: choose search terms, Step 3: Decide

on appropriate databases search, and Step 4: conduct a

search statement). As Figure 2 reveals, Students tended to

perceive field search by author, field search by title,

thesaurus and the four step keyword search as important

but they were not familiar with the ways of performing

those search skills (all with p<.05).

One lecturer (FS) mentioned that “all of the listed

databases are necessary in this field, especially for

information literacy. Students should know how to make

use of different sources and perspectives to find the

information”, echoing students‟ high levels of perceived

importance. However, many graduates, such as HCH

perceived that their low level of familiarity to a range of

databases was a result of the “lack of practice

opportunities”.

Page 6: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

6 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

Figure 2. Comparison between perceived familiarity and importance of various information search knowledge and

skills (posttest).

Changes in Computer Software Literacy Students' perceived familiarity and importance of computer software

In most of the computer software, including bibliographic

software, project management software, data mining

software, record management tools, online survey tools,

digital library software and animations on web, students

had significant increases in their familiarity from the first to

the third surveys (all with p<.05). On the other hand, some

of the items, for example database management software,

web page authoring tools, digital storytelling software,

video editing software and advanced spreadsheet software,

had significant changes between the first and second

surveys (all with p<.05). In general, the overall mean rating

is below 3, which indicates that students did not reach the

minimum level of desired familiarity with the various kinds

of computer software.

Results of paired-sample t-tests showed that there was a

significant increase in the perceived importance of web

page authoring tools when comparing the first and the

second surveys (t=-3.0, p=.007). On the other hand, digital

library software has a significant decrease in the perceived

importance particularly in the later stage of the program

when comparing the second and third surveys (t=4.135,

p=.001).

The paired-sample t-tests comparing the perceived

familiarity and importance of computer software during the

third survey demonstrated that eight items were found

important but students reported low familiarity. As

presented in Figure 3, computer software including

statistical software, project management software, database

management software, data mining software, web page

authoring tools, record management tools, online survey

tools and advanced spreadsheet software, are perceived to

be important by the students, but their level of familiarity

was significantly lower than the perceived importance (all with p <.05).

Regarding such discrepancies, graduate LKF commented

that he “did not have the chance to practice SPSS

(Statistical Software) but only know about its functions in

the lecture”. Graduate HCH also reflected that he “lacked

the opportunity to apply the software in real-life situations”.

Graduate CSK commented that “the program progress

needs to be slowed down, particularly for those computer

courses. For some students without much prior computer

knowledge they had to struggle”. Graduate LHN also noted

that the program “needs to be more in-depth and detailed

in terms of knowledge delivered by lecturers in class. When

I learnt MySQL, I did not have much help from the others. I

do think I need more guidance in the process”.

Page 7: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

7 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

Figure 3. Comparison between perceived familiarity and importance of computer software (posttest).

Changes in Web 2.0 Literacy

Students' perceived familiarity and importance of Web 2.0 applications

Students were also required to rate their familiarity and

perceived importance of Web 2.0 applications. The result

of the t-test indicated that students‟ familiarity with Google

docs (t=-2.5, p=.021) and RSS (Really Simple Syndication)

(t=-4.036, p=.001) significantly increased between the

second and third surveys.

The result of paired-sample t-tests that compares the mean

ratings between the first and third survey illustrated that the

overall rating of perceived importance of Web 2.0

applications increased significantly after the program (t=-

2.757, p=0.013). Also, the results showed that specific

items, such as Wikis (t=-4.025, p=.001) and RSS (t=-2.511,

p=.026) significantly increased in perceived importance

between the first and second survey. Whereas, the

perceived importance of Google docs (t=-3.758, p=.001)

and social networking (t=-2.792, p=.012) significantly

increased comparing the first and the third surveys.

Paired-sample t-tests compared the perceived familiarity

and perceived importance of Web 2.0 applications during

the third survey. As indicated in Figure 4, there is a

significant difference between the perceived familiarity and

perceived importance of blogs (t=2.752, p=0.012). Students

tended to have a high familiarity with blogs but did not

consider them as important.

With regard to this discrepancy, graduate HCH commented

that “The program should consider the actual needs of the

students. There are too many compulsory courses. There is

a mismatch between what lecturers taught and what

students wanted to learn”. In order to minimize the

mismatch between learning outcomes and students‟

expectation, lecturer JL suggested that “there should be

more communication between the students and lecturers”.

Furthermore, lecturer SC suggested that, “more flexibility

on course selection should be given to students so that they

can develop in different streams”.

Figure 4. Comparison between perceived familiarity and importance of Web 2.0 applications (posttest).

Page 8: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

8 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

DISCUSSION General improvement in multiliteracies

Students in the BScIM program appear to have improved

multiliteracies after undergoing outcome-based teaching

and learning. General improvement was demonstrated in

the learning outcomes of information literacy, computer

software literacy and Web 2.0 literacy. In terms of

information literacy, there was a substantial increase in

students‟ ability to identify source types, recognize

electronic databases and use information search skills.

After the 2-year program, students were found to be more

familiar with the skills of locating, evaluating and

retrieving information from a wide range of database

resources. This positive outcome suggests that OBE could

contribute to teaching and learning information literacy.

In terms of computer software literacy, students

demonstrated a significant development in the perceived

familiarity with computer software. However, the mean

score of familiarity after the 2-year programme was not

adequate to meet the desired outcome of familiarity (i.e.

score of 3.0).

In terms of Web 2.0 literacy, students reported a high level

of perceived familiarity with Web 2.0 applications after the

2-year period. Such rating suggests that students have

become more aware of how to use Web 2.0 tools to

collaborate, co-publish, and interact effectively with

multiple users. In general, the findings of the study indicate

that with the use of OBE as a teaching strategy,

improvements in multiliteracies could occur.

Mismatch between program outcomes and students’ learning expectation

The results of this study show that students possessed high

familiarity but relatively low perceived importance for

some items, such as blogs, indicating that there might be a

mismatch between the program outcomes and students‟

learning expectation.

Although program outcomes can guide the direction of

students‟ development in multiliteracies, students‟

expectations also played a significant role in shaping their

overall learning process. It was recommended that lecturers

let the students have a thorough comprehension about the

potential and importance of the multiliteracies before the

teaching and learning process. By introducing the learning

outcomes to the students, they understand better and

become more aware of the needs of acquiring

multiliteracies. Consequently, they could be more

committed to their learning throughout the program.

The learning outcomes could be revised and modified

according to the needs of the students. By communicating

the learning outcomes to the students, lecturers can receive

feedback and comments, and make necessary changes on

course content and lesson plans. The process of interaction

and collaboration can provide flexibility in developing the

learning outcomes. Balancing the expectations of the

program review panel and students could potentially

increase the effectiveness of OBE.

Furthermore, dividing the program into several streams of

specialization could make learning outcomes be more

specific and customized to the expectation of the students.

Therefore, it is anticipated that the students can have a

deeper and more specialized development in term of

multiliteracies.

Discrepancy between expected achievement and students’ acquired level of multiliteracies

Students reported a high degree of perceived importance

but a relatively low degree of familiarity on a range of

computer software and search skills. For instance, the

comparison between perceived familiarity and perceived

importance of computer software (in the third survey)

demonstrated that the student participants regarded most of

the listed computer software as somewhat important while

the reported familiarity was rather low. Students‟

comments might indicate that they need more scaffolding

support and practice opportunities during the learning

process. As suggested by Davis (2003), OBE does not

specify certain educational strategies or teaching methods.

It is more important to emphasize on whether the students

can achieve the learning outcome rather than the way they

get there. Therefore, flexible teaching strategies, such as

small tutorials, practical session and online scaffolding

should be considered to create a learner friendly

environment in order to accommodate the needs of

different students. OBE should be supported by flexible

teaching strategies, which aim at directing all students

towards the same learning outcomes.

Students‟ lack of prior related knowledge might be another

cause of their insufficient familiarity with a number of

components of multiliteracies upon program completion.

Adequate time and assistance have to be provided so that

each student can reach the maximum potential (Towers,

1996). It is recommended that the program should direct

more efforts and resources in the process of teaching the

students with multiliteracies, in order to enhance the

students‟ accomplishment with generic academic learning

outcome. More importantly, students‟ ability and faculty

resources have to be taken into account in the stage of

program development, so as to set up challenging but yet

achievable learning outcomes for the students.

LIMITATION AND FUTURE STUDIES

The study relied on the subjective rating and perception of

the student participants. It lacked objective assessment

methods, such as tests, to evaluate the development of

multiliteracies (e.g., Web 2.0 literacy) among the students.

Thus, future research could be conducted with the

consideration of both subjective and objective assessment

methods. The generalizability of the study findings are

restricted by the sampling strategy such that the participants

consist of a single class of undergraduate students. It is

suggested that future researchers could expand the scale of

Page 9: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

9 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

this study in order to obtain a more representative sample

of undergraduate students.

CONCLUSION

This study aimed to examine changes in multiliteracies‟

learning outcomes with the use of OBE among

undergraduate students. It was found that in a programme

that used OBE pedagogy, students‟ multiliteracies gained

significant improvements after the programme duration.

Students showed improvements in information literacy,

computer software literacy and Web 2.0 literacy. Students‟

and teachers‟ comments showed that the OBE

implementation could be enhanced by carefully setting

learning outcomes, such that students‟ ability, learning

expectation and faculty‟s resources are well balanced and

considered.

REFERENCE

Aldridge, J.M., Laugksch, R.C., & Fraser, B.J. (2006).

School-level environment and outcomes-based

education in South Africa. Learning Environments

Research, 9(2), 123-147.

Alexander, B. (2008). Web 2.0 and emergent

multiliteracies. Theory Into Practice, 47, 150-160.

Andrich, D. (2002). A Framework Relating Outcomes

Based Education and the Taxonomy of Educational

Objectives. Studies in Educational Evaluation, 28(1),

35-59.

Anstey, M., & Bull G. (2006). Teaching and learning

multiliteracies: Changing times, changing literacies.

Newark, DE: International Reading Association.

Baguley, M., Pullen, D., & Short, M. (2010).

Multiliteracies and the new world order. In D.L. Pullen

& D.R.Cole (Eds.), Multiliteracies and technology

enhanced education: social practice and the global

classroom (pp. 1-17). Hershey, PA: IGI Global.

Bawden, D. (2001). Information and digital literacies: A

review of concepts. Journal of Documentation, 57(2),

218-259.

Botha, R. J. (2002). The Introduction of a System of OBE

in South Africa: Transforming and Empowering a

Marginalized and Disenfranchised Society. Paper

presented at the 46th Annual Meeting of the

Comparative and International Education Society,

Orlando, FL

Brindley, G. (2001). Outcomes-based assessment in

practice: Some examples and emerging insights.

Language Testing, 18(4), 393-407.

Bruce, C. (1997). The Seven Faces of Information Litearcy.

Adelaide: Auslib Press.

Butler, M. (2004). Outcomes based/ Outcomes focused

education overview. Retrieved from

http://scholar.google.com.hk/scholar?hl=en&q=Outcomes+based%2F+Outcomes+focused+education+overview&as_sdt=2000&as_ylo=&as_vis=0

Catts R. & Lau J. (2008). Towards Information Literacy

Indicators. Paris: UNESCO. Retrieved from

http://unesdoc.unesco.org/images/0015/001587/158723

e.pdf

Cazden, C., Cope, B. & Fairclough, N., Gee, J., et al.

(1996). Pedagogy of multiliteracies: designing social

futures. Harvard Educational Review, 66 (1), 60-92.

Chang, W., Jones, A. & Kunnemeyer, R. (2002).

Interactive Teaching Appraoch in Year One University

Physics in Taiwan: Implementation and Evaluation.

Asia-Pacific Forum on Science Learning and Teaching,

3(1), Article 3, p.3

Chu, S., Tse, S.K., & Chow, K. (2011). Using collaborative

teaching and inquiry project-based learning to help

primary school students develop information literacy

and information skills. Library and Information Science

Research, 33, 132 – 143.

Davis, M.H. (2003), Outcome-based education, Journal of

Veterinary Medical Education, 30(3), 227-232.

Doyle C. (1992). Outcome Measures for Information

Literacy Within the National Education Goals of 1990:

Final Report of the National Forum on Information

Literacy. US Department of Education, Washington,

DC.

Faris, R. (1998). From Elitism to Inclusive Education:

Development of Outcomes-Based Learning and Post-

Secondary Credit Accumulation and Transfer Systems

in England and Wales. 1-18. Retrieved from

http://eric.ed.gov/PDFS/ED449830.pdf

Harden, R.M. (2002). Developments in outcome-based

education. Medical Teacher, 24(2), 117-120.

Hartel, R.W. & Gardner, D. (2003). Making the transition

to a food science curriculum based on assessment of

learning outcomes. Journal of Food Science Education,

2(2), 32-39.

Hepworth, M. (1999). A study of undergraduate

information literacy and skills: the inclusion of

information literacy and skills in the undergraduate

curriculum. IFLA Council and General Conference,

Bangkok, Thailand, Aug 20-28, 1999. Retrieved from

http://www.ifla.org/IV/ifla65/papers/107-124e.htm

Huvila, I. (2011). The complete information literacy?

Unforgetting creation and organization of information.

Journal of Librarianship and Information Science,

43(4), 237-245.

Iyer, R., & Luke, C. (2010). Multimodal, multiliteracies:

texts and literacies for the 21st century. In D.L. Pullen

& D.R.Cole (Eds.), Multiliteracies and technology

enhanced education: social practice and the global

classroom. (pp. 18-34). Hershey, PA: IGI Global.

Jenkins, A., & Unwin, D. (1996). Writing learning

outcomes for the Core Curriculum. NCGIA GISCC

Learning Outcomes. Retrieved from

http://www.tcc.edu/welcome/collegeadmin/OIE/SOA/re

view/toolkit/documents/Article_Writing_Learning_Out

comes_for_the_Core_Curriculum_NCGIA_GISCC.pdf

Kovalik, C.L., & Dalton, D.W. (1997). A Conceptual

Framework for Assessment: The Process/Outcome

Evaluation Model. Retrieved from

http://eric.ed.gov/PDFS/ED409848.pdf

Lorenzen, M. (1999). Using outcome-based education in

the planning and teaching of new information

Page 10: Developing Undergraduate Students’ Multiliteracies withtl.hku.hk/wp-content/resources/tdg-deliverables/87-3.pdf · 2013-04-19 · Developing Undergraduate Students’ Multiliteracies

10 J:\Sam-research\TDG-OBL-Eng\publications\Conference - ASIST 2012\ASIST-v12.docx 4/30/2012 6:43:56 PM

technologies. Journal of Library Administration, 26(3),

141-152.

Mailula, E.M., Laugksch, R.C., Aldridge, J.M., & Fraser,

B.J. (2003). School-Level Environment and the

Implementation of Outcomes-Based Education in South

Africa. Retrieved from

http://eric.ed.gov/PDFS/ED477541.pdf

McCullough, C.A. (2008). Transforming Course

Evaluations into a Meaningful Measure of Student

Outcomes Achievement. Assessment Update, 20(5), 3-5.

McDaniel, E.A., Felder, B.D., Gordon, L., Hrutka, M.E., &

Quinn, S. (2000). New faculty roles in learning

outcomes education: The experiences of four models

and institutions. Innovative Higher Education, 25(2),

143-157.

Pullen, D.L. & Cole, D.R. (Eds.), Multiliteracies and

technology enhanced education: social practice and the

global classroom. (pp. 186-188). Hershey, PA: IGI

Global.

Rees, C.E. (2004). The problem with outcomes-based

curricula in medical education: insights from

educational theory. Medical education, 38, 593-598.

Ross, N. (1999). AMEE Guide No. 14: Outcome-based

education: Part 4-Outcome-based learning and the

electronic curriculum at Birmingham Medical School.

Medical Teacher, 21(1), 26-31.

Rowsell, J. Kosnik, C. & Beck, C. (2008). Fostering

multiliteracies pedagogy through preservice teacher

education. Teaching Education, 19(2), 109-122.

Ryder, J. (2004). What can students learn from final year

research projects? Bioscience Education E-journal, 4(2),

4-2. Retrieved from

http://www.bioscience.heacademy.ac.uk/journal/vol4/be

ej-4-2.pdf

Sawetrattanasatian, O. (2008). University library web site

design: a case study of the relationship between

usability and information literacy development

(Doctoral dissertation). Retrieved from

http://erl.canberra.edu.au/public/adt-

AUC20090311.091629/

Shipley, D. (1994). Outcome Based Education: Its Impact

on Program Review and the Evaluation of Learners.

Paper presented at the Annual Conference of the

Association of Canadian Community Colleges, Ottawa,

Ontario, Canada.

Spady, W., & Marshall, K. (1994, November). Light, not

heat, on OBE. The American School Board Journal,

181, 29-33.

Spitzer, K. Eisenberg, M. & Lowe, C. (1998). Information

Literacy: Essentials Skills for the Information Age.

Information Resources Publications. Syracuse, NY:

Syracuse University.

Tierney (2006). Examining literate lives as students engage

with multiple literacies. Theory into Practice, 45(4),

359-361.

Towers, J.M. (1996). An elementary school principal's

experience with implementing an outcome- based

curriculum. Catalyst for Change, 25(2), 19-23.

Tynes, K.R. (1998). Literacy in a digital world: teaching

and learning in the age of information. Mahwah, NJ:

Lawrence Erlbaum Associates.

Webber (2000). Conceptions of information literacy: new

perspectives and implications. Journal of information

Science, 26, 381-397.

Westby, C. (2010). Multiliteracies the changing world of

communication. Topics in Language Disorders, 30(1),

64-71.

Wien, C. A., & Dudley-Marling, C. (1998). Limited vision:

The Ontario curriculum and outcomes-based learning.

Canadian Journal of Education/Revue canadienne de

l'education, 23(4), 405-420.