Turku University Library on Information Literacy

Mission

Evaluating the Impact of Information Literacy Skills Teaching

Jaana TaylersonMathematics & Natural Sciences Library

University of Turku

Second largest university in Finland 18 000 students Six faculties: Humanities (HUM), Mathematics

and Natural Sciences (MNS), Medicine, Law, Social Sciences (SOC), Education (EDU)

Turku University Library Approximately 100 employees Main Library and 20 other library units Team of 10 Librarians/Information specialists

providing various amounts of IL teaching

Mission

To raise the awareness and importance of IL at institutional level

To integrate IL into the academic curriculum

To justify the needs of more resources towards IL teaching

To find out what our students really thought about IL,their own skills, library resources and IL teaching

Mission Strategy

Visits to all faculties and departments Marketing 8 hours of free IL teaching at various stages of

academic studies

Survey of the scope of IL teaching First Year Students (FYS) Bachelors’ (B) Masters’ (M)

Self-assessment survey of students’ use of information resources and their own IL skills

Students’ feedback evaluation of the IL teaching

IL Teaching at University of Turku IL not compulsory (except for

1st Year Medicine Students)

The library in charge of IL teaching

Considerable differences between faculties, departments and subjects

Considerable differences in group sizes (1-144 in 2008)

Due to visits to faculties IL teaching requests are on the increase

IL Teaching

2005 2007

Teaching hours

375 517

Taught students

3448 5643

Scope of IL Teaching at MNS Faculty

FYS 62% (National average 91-100%) BSc 79% (National average 75-80%) MSc 15% (National average 15-20%)

Based on subject coverage, national average based on student numbers

71 hours of teaching during last academic year 150 hours needed to meet the target (8h per

student altogether)

The scope of IL teaching at MNS

Students’ Self-Assessment Survey on IL Skills

A pilot questionnaire on the Internet (Webropol), pre and post IL teaching

A total of 350 pre and 279 post replies Faculties of Humanities (HUM) Education (EDU) Social Sciences (SOC) Mathematics and Natural Sciences (MNS)

Results analyzed according to a year of studies, subject and faculty

Results of 1st Year Students (FYS) Overconfidence, ignorance and Googling roles

Use of Volter Library Cataloque increased noticeably In some areas teaching had no or negative effect

Nelli Portal (The National Electronic Library of Finland) was considered just one of the Internet’s search engines (have seen better…) or part of free Internet – muddy, fussy, confusing and difficult

However, students regarded having very good skills using Nelli Portal…

Questions

Excellent or Good 

pre post

Skills in Author Search in Volter  73 95Skills using Volter  57 84Keyword Knowledge   45 77Skills in Keyword Searches  36 72

Skills using e-books  15 41

Use of Boolean operators  24 41

Skills using Nelli  14 41

Results of 2nd-3rd Year Students (B)

Insecurity skills only average or poor in most areas

Very low use of Boolean operators, E-books, abstract and fulltext databases and the Main Library (call slip requests) before teaching 56 % 2nd years and 58 % 3rd years had not used e-

books 37 % 2nd years and 35 % 3rd years had not used Nelli

Portal

44 % of 3rd year students had not used Boolean operators at all, still 28 % hadn’t used them after teaching…

Results of 2nd-3rd Year Students (B) cont. Teaching had a noticable impact on the use of e-resources

37 % had not used Nelli Portal at all before teaching, only 7 % after teaching

Huge effect on students’ awareness of subject-specific e-resources

Students have started using the e-resources but are not confident – they rate their own skills only average or poor in most areas (journal database searches, use of e-books, kowledge of subject-related resources) IL teaching mainly covers the introduction of e-resources but only scratches the surface of search strategies

E-books still not well used after teaching (36% not used)

Results of 4th-6th Year Students (M) Very good skills using Volter Library Catalogue

Big improvement in understanding and performing keyword searches and Boolean operators

Big impact on awareness and use of subject-specific resources

67 % are still not using either Search Alert Services or RefWorks/EndNote

Still 35%-40% had not used e-books or any databases (full text/abstract) before teaching and situation did hardly improve after teaching

Only 16 % new very well or well their subject specific resources, after teaching the number rose to 36%, ca 30% had not used the subject specific resources at all…

Skills using Nelli Portal still only good or average

Everybody had been using Volter Library Catalogue

Faculty Differences

Students' IL Skills by Faculties

0

10

20

30

40

50

60

70

pre T post T pre T post T pre T post T

Excellent/Good Skills Average Skills Fair/Poor Skills

HUM

MNS

EDU

SOC

Faculty Differences cont. IL Teaching had rather variable effect on students’ IL skills –

impact was greatest on HUM and MNS and less effective on EDU and SOC – is there a reason for it or is it just statistical bias?

Overestimation of own skills Varying research/study requirements Timing Insufficient teaching

HUM and SOC know Volter best MNS skills are best in Nelli environment EDU uses abstract databases (ERIC) more than the others SOC is well equipped in exploiting the paper resources in

the libraries MNS is very cognisant of its own subject specific resources

Subject differences

Subject differences among Mathematics and Science Faculty students

%

Skills in Using Nelli Portal

The impact of IL teaching on students’ skills in using Nelli, The National Electronic Library of Finland

0

5

10

15

20

25

30

35

40

45

%

Excellent Good Average Poor Not used

All students' skills in using Nelli Portal

Pre Teaching

Post Teaching

Skills in Using Nelli Portal

One of the interesting findings is the astonishing amount of 3rd (52 %) and 4th – 6th (38 %) year students not been using Nelli’s electronic resources at all before teaching -have to make an impact earlier

IL Skills of University of Turku Students

0

10

20

30

40

50

60

Excellent Good Average Fair Poor

All students' IL-skills

Pre Teaching

Post Teaching

Over 50 % of the students consider having average IL skills

However,teaching did have an impact especially on good skills

Feedback Questionnaire Results 07-08 90 % students regarded teaching important or very important Importance grew the further the cycle the students were in

their studies

The Importance of IL Teaching (MNS)

0

10

20

30

40

50

60

70

80

Very Important Important Less Important Waste of Time

%

MNS 1stY

MNS B

MNS M

Feedback Questionnaire 2007-2008 524 feedback forms were collected and analyzed

between 2007-2008, here are some of the questions and results

Were there enough practical excercises? The further cycle the students are in the more hands-on

exercises are required ”Excercises, no powerpoint presentations”, ”Computers on”

Was the tuition of suitable duration? The higer stage the students are the the more IL-

teaching is required – 1st year reasonably satisfied with their 2-hour session but BA and MA clearly required much more tuition -”good stuff but too much packed into too little…so more of it! Otherwise OK”

Feedback Questionnaire 2007-2008 cont. How well did the IL-teaching fit into your

studies? Over 80 % were very satisfied or satisfied with the

timing

FYS felt that teaching given after January came too late, it should have taken place soon after Freshers’ week

the session might be the first and last tuition for quite a few students, even at Masters’ stage

Many frustrated students receive their only IL teaching at BA stage – “WHY HAVEN’T I BEEN TOLD OF THESE

BEFORE???” Better late than not at all - Motto of the question

Feedback Questionnaire 2007-2008 cont. Would you like to have more tuition?

The further cycle the students are in the more IL-teaching is appreciated and required

55% wanted more, only 12 % considered having no further tuition. 1st year students were not able to make up their minds but BA/MA were certain of the need of more tuition.

”More of it! Good stuff but too much packed into too little, so more time, more sessions, more one to one tuition, just more please! And make it compulsory for the 1st year…!

All the NO answers of MNS BA were from Physics – good reminder of the differences of each student group/department/discipline.

Conclusions

IL teaching is important to students

IL teaching has an impact on students’ Awareness of resources Use of resources IL-skills Knowledge of subject-specific resources

The further cycle the students are in their studies the more IL-teaching is appreciated and required and more hands-on exercises needed

Conclusions cont. Library has vital role in raising the awareness

of subject-specific resources Whose responsibility? Librarians have boarder view, academics more specific view Do academics know the resources well enough? Do we? Does it still come down to “unless the faculty are

information literate themselves, students will not be” (Young & Harmony 1999)???

1st year students’ confidence is not competence 1st year students very confident about their own skills,

don’t realize yet what research and academic study truely requires ” have used search engines before – piece of cake”

Compulsory tuition for FYS during the first semester – students’ own wish

IL Questionnaire improvements

More extensive attendance (pre teaching) Second questionnaire only to the 1st

questionnaire repliers Results sorted according to the cycle of

studies, not the year of studies Computerized analysis, now too time

consuming Some questions confusing, need changes More chocolate…

Contact Details

Jaana TaylersonMathematics and Natural Sciences LibraryUniversity of TurkuFI-20014 TURKU

FINLAND

Email:jaana.taylerson@utu.fiPhone:+358 2 333 5464

By Lydia Taylerson

References

ASSESSING STUDENT LEARNING OUTCOMES FOR INFORMATION INSTRUCTION IN ACADEMIC INSTITUTIONS (2003). Association of College and Research Libraries, Chigago.

CRAIG, A. and CORRALL, S. (2007) Making a difference? Measuring the impact of an information literacy programme for pre-registration nursing students in the UK. Health Information and Libraries Journal, 24, 118-127.

DILLER, K.R. and PHELPS, S.F. (2008) Learning outcomes, portfolios, and rubrics, oh my!Authentic Assessment of an Information Literary Program. Libraries and the Academy, 8 (1), 75-89.

IANUZZI, P. (1999) We are teaching, but are they learning: accountability, productivity, and assessment. The journal of Academic Librarianship, 25 (4), 304-305.

INFORMAATIOLUKUTAITO YLIOPISTO-OPETUKSESSA (2007). Palmenia, Helsinki.

References cont. INFORMATION BEHAVIOUR OF THE RESEARCHER OF THE FUTURE. A Ciber

briefing paper (2008). [available at

http://www.ucl.ac.uk/slais/research/ciber/downloads/ggexecutive.pdf] KNIGHT, L.A. (2002) The role of assessment in the library user education.

Reference Services Review, 30 (1), 15-24. MACKLIN, A.S. (2001) Integrating information literacy using problem-based

learning. Reference Services Review 29 (4), 306-314. TAYLERSON, J. (2004) Information Literacy Teaching at University College

Chester. M.A. Dissertation, Liverpool John Moores University. VICKERY, S. and COOPER, H. Confidence or Competence? – auditing

information literacy skills of biology undergraduate students. [available at http://www.lib.mq.edu.au/about/conferences/educause/competence.pdf ]

WALLACE, M., SHORTEN, A. and CROOKES; P.A. (2000) Teaching information literacy skills: an evaluation. Nurse Education Today, 20, 485-489.

YOUNG, R.M. and HARMONY, S. (1999) Working with faculty to design undergraduate information literacy programs. Neal-Schuman Publishers, New York