Is MALL really useful?

Is mobile-assisted language learning really useful?

An examination of recall automatizationand learner autonomy

Takeshi SATO1

Fumiko MURASE1

Tyler BURDEN2

1Tokyo University of Agriculture & Technology, Japan 2Meisei University, Japan

EuroCALL 2015 @ University of Padova, Italyon 28th of August, 2015

Critical CALL

Does CALL really help L2 learning?

Does MALLreally help L2 learning?

Overview

1. Introduction

2. Research Questions

3. Procedures

4. Findings

– L2 knowledge building

– Learner autonomy

5. Conclusion & Implication

INTRODUCTION

Advantages of MALL

• ownership

• mobility

• convergence of technologies

(Kukulska-Hulme, 2009)

Ownership

• 88.1% of Japanese high school students have their mobile phones

• 55.2% of them have their PCs

(Official Announcement of Internet Literacy, 2014)

Retrieved 25th of August 2015 from http://www.asahicom.jp/articles/images/AS20140306004951_commL.jpg

http://www.asahicom.jp/articles/images/AS20140306004951_commL.jpg

Ownership

All students in a Japanese university have mobile devices

• 94% smartphone

• 21% mobile phone

• 7% Tablet

(Kindai University 2013)Retrieved 2nd of August from http://goo.gl/UWzH47

http://goo.gl/UWzH47

Mobility

“The mobility of digital technologies creates intriguing opportunities for new forms of learning”

(Laurillard 2007 p. 153)Retrieved 2nd of August from http://goo.gl/h0qw69

Convergence of technologies

Technologically advanced presentation of learning resources

(e.g. Chun & Plass 1996; Sato & Suzuki 2010; Sato et al. 2013)

Retrieved from Eigo Kumitate Towndeveloped by COCONE

Successful MALL

• Agency (Pachler at al. 2010) or Learner Autonomy (Holec1981)

• Learners are expected to be autonomous agents who take control over learning content (Benson 2001)

Retrieved 28th of August 2015 fromhttp://www.obirin.ed.jp/

Is MALL really useful?

• in recalling linguistic items

• in enhancing learner autonomy

RESEARCH QUESTIONS

1. If L2 learners study phrases for academic essays with a mobile-based application for a certain period, could they recall more phrases in the written test than those studying with a paper-based list?

2. If L2 learners study the phrases with the application, could they recall more quickly in the test than those with a paper list?

3. If L2 learners study the phrases with the application, could they use more of those phrases in writing an essay than those with a paper list?

4. If L2 learners study the phrases with the application, could they enhance the learner autonomy?

Procedures

1. Randomly divided into 2 groups – mobile application (n=40) vs. paper list (n=54)

2. Introduced the treatment 3. Answered the questionnaire about learner

autonomy by accessing the website outside the classroom via their mobile devices

4. Studied 100 phrases for 3 weeks by themselves5. Fill-in-the-blank test (10 minutes)6. Essay writing (75 minutes)7. Answered the questionnaire about learner

autonomy (49 items plus additional section (6 items) per group)

Teaching Context• Sophomore from

faculty of engineering

• No student specialized in English

• Approximately B1 level

• One of the tasks in their writing classes

• The classes were allocated according to their scores of G-TELP test

Procedures

1. Randomly divided into 2 groups – mobile application (n=40) vs. paper list (n=54)

2. Introduced the learning tools

3. Answered the questionnaire about learner autonomy by accessing the website

4. Studied 100 phrases for 3 weeks

5. Fill-in-the-blank test (10 minutes)

6. Essay writing (75 minutes)

7. Answered the questionnaire about learner autonomy (49 items plus additional section)

Phrase list

Mobile application

FINDINGS

L2 Knowledge Section

The average scores of the test

10.08

6.48

0.00

2.00

4.00

6.00

8.00

10.00

12.00

experimental group (n=40) control group (n=54)

n M SD df t Sig. r

Control 54 6.48 5.79 92 2.82 .006** .28

Experimental40 10.01 6.50

T-test results comparing two groups on the average scores of the test (N=94)

A significant difference was found between the two groups

*p<.05 **p<.01

Recall time (second)

520.10

548.76

505

510

515

520

525

530

535

540

545

550

555


n M SD df t Sig. r

Control 54 548.8 108.1 92 -1.22 .23 -.13


T-test results comparing two groups on the average time of the test (N=94)

No significant difference was found between the two groups

Essay writing

• Asked to write an essay about a certain topic with at least three paragraphs

• Also asked to put as many phrases they studied as possible on their essays.

• No reference (ex. dictionary) was permitted

• Their essays were graded (1-9) according to IELTS criteria

• The number of the phrases was counted

G-TELP Bands

6.05

5.96

6.05

5.7

5.855.82

5.85

5.56

5.3

5.4

5.5

5.6

5.7

5.8

5.9

6

6.1

task achievement cohesion andcoherence

lexical resources grammaraccuracy

Experimental (n=40)

Control (n=54)

Average band of G-TELP writing section

M (SD) df t Sig. r

Control(n=54)

Experimental(n=40)

task achievement

5.85 6.05 92 1.21 .23 .12

cohesion and coherence

5.82 5.96 92 1.32 .19 .13

lexical resources

5.85 6.05 92 1.59 .11 .16

grammar accuracy

5.56 5.70 92 1.26 .21 .13

T-test results comparing two groups on two dimensions (N=94)

*p<.05 **p<.01

No significant difference was found

Number of phrases on an essay

2.60

1.48

0.00

0.50

1.00

1.50

2.00

2.50

3.00


n M SD df t Sig. r

Control 54 1.48 1.71 92 3.15 .002** .31


T-test results comparing two groups on the number of the phrases (N=94)

A significant difference was found between the two groups

*p<.05 **p<.01

Summary

RQ1 : Could they recall more phrases in the written test than those studying with a paper-based list?

Answer : YES

RQ2: could they recall more quickly in the test than those with a paper list?

Answer: NO

RQ3: could they use more of those phrases in writing an essay than those with a paper list?

Answer: YES

Learner Autonomy Section

Questionnaire

• An Internet-based questionnaire about their attitudes and views toward learning English,which was designed to measure the technical and psychological dimensions of learner autonomy, was used for this study.

Measuring Instrument for Language Learner Autonomy (Murase 2015)

Questionnaire

Measuring Instrument for Language Learner Autonomy (MILLA) （言語学習における学習者オートノミー尺

度）

• 89 items– covering the 4 different dimensions of learner autonomy

• 5-point Likert scale – (SD <D < NAND <A < SA)

• Paper-based• Written in Japanese

– + English translation ver.

(Murase 2015)

An online version (using Google form) was created for this study.

Dimensions of Learner Autonomy (Murase 2015)

Technical

Socio-cultural

Political-philosophical

Psychological

“capacity”(Little, 1991)

“ability” (Holec, 1981)

“individual autonomy””group autonomy”(Pennycook, 1997)

“political version” (Benson, 1997)

“sociocultural ” (Oxford, 2003)

Why did we look at only 2 dimensions?

We planned to administer the post-test only 3weeks after the pre-test.

1. Answering a survey with a total of 87 items twice within such a short interval would be a burden to the participants (which may lead to low response rates).

2. Items measuring the other two dimensions (i.e., political-philosophical and socio-cultural dimensions) may not change dramatically in such a short period of time.

Technical Autonomy

Q19: I write down what kinds of materials I used for my English study. （どんな教材を使って英語を勉強したか、書き留める。）

Q21: I take notes of my feelings while I am studying English. （英語を勉強している時の自分の気持ちをメモに取る。）

(21 items)

Psychological Autonomy

Q30: Every student ought to reflect upon how he/she studied after he/she finishes studying English for the day. （どの学生も、その日の英語の学習を終えた後、自分がどうやって勉強したかを振り返るべきだ。）

Q33: A good learner of English keeps records of what he/she learned from his/her English study. （良い英語学習者は、自分が英語の勉強で何を学んだか、記録をつける。）

(28 items)

Political-Philosophical Autonomy

Q57: Students should always follow their teachers’ instructions. （学生はいつでも先生の指示に従うべきだ。）

Q58: What a teacher says is always correct. （先生の言うことはいつも正しい。）

(20 items)

Socio-Cultural Autonomy

Q78: I sometimes adopt what other students are doing into my own learning. （他の学生がやっていることを自分の学習に取り入れることがある。）

Q85: Japanese students tend to take the initiative in their learning. （日本人学生は主体的に学習に取り組む傾向がある。）

(18 items)

Technical Autonomy

Q19: I write down what kinds of materials I used for my English study. （どんな教材を使って英語を勉強したか、書き留める。）

Q21: I take notes of my feelings while I am studying English. （英語を勉強している時の自分の気持ちをメモに取る。）

(21 items)

Psychological Autonomy

Q30: Every student ought to reflect upon how he/she studied after he/she finishes studying English for the day. （どの学生も、その日の英語の学習を終えた後、自分がどうやって勉強したかを振り返るべきだ。）

Q33: A good learner of English keeps records of what he/she learned from his/her English study. （良い英語学習者は、自分が英語の勉強で何を学んだか、記録をつける。）

(28 items)

the learners’ act of learning a language on their own outside the classroom, without the aid of a teacher and, also as the situation in which learners, for some reason, are obliged to take control of their own learning

(see Benson 1997)

the ‘capacity’ of individual learners which ‘allows learners to take more responsibility for their own learning’

(see Benson 1997)

Data Analysis

1. Pre-test (comparing Control and

Experimental)

2. Post-test (comparing Control and

Experimental)

3. Comparing Pre-test and Post-test

まったくしないめったにしない時々する頻繁にするいつもする

(Never) (Rarely) (Sometimes) (Often) (Always)

1 2 3 4 5

まったくそう思わないそう思わないどちらともいえない

そう思う強くそう思う

(Strongly Disagree) (Disagree) (NeitherAgree nor Disagree)

(Agree) (Strongly Agree)

1 2 3 4 5

ScoringQ 1- 21 (Technical)

Q 22- 49 (Psychological)

Higher score = higher autonomy

1. Pre-test

Findings

• Compared the mean scores of –individual items and –each of the two dimensions between two groups.

• no significant difference was observed

2. Post-test

a) Individual items (Q1-49)

M (SD) df t Sig. r

Control(n=46)

Experimental(n=30)

Q17 2.24(1.777)

1.77(.817)

74 2.06 .043* 0.32

Q21 1.76(1.037)

1.33(.661)

74 2.12 .031* 0.47

T-test results comparing two groups on individual items (N=76)

*p<.05

Q17英語の勉強にどれだけ時間を費やしたか、メモを取る。

(I take notes about how much I spent on my English study.)

Q21

英語を学習している時の自分の気持ちをメモに取る。

(I take notes of my feelings while I am studying English.)

As both items are concerned with taking notes while learning, it can be assumed that it might have been easier to physically take notes if they were working with the paper-based list.

b) Technical/Psychological dimensions

M (SD) df t Sig. r

Control(n=46)

Experimental(n=30)

Technical 2.60(.568)

2.38(.586)

74 1.665 .100 .19

Psychological 3.34 (.509)

3.40 (.400)

74 -.491 .625 .06

T-test results comparing two groups on two dimensions (N=76)

No significant difference between the two groups.

3. Post-test (Additional Section)

1. Frequency

During the three weeks, how often (on average) did you study the expressions?

• Almost everyday• 3 to 4 times a week • 1 to 2 times a week • I hardly studied

n M SD df t Sig. r

Control 46 1.78 .513 74 -.702 .485 .08

Experimental30 1.87 .507

T-test results comparing two groups on the frequency of their learning (N=76)

no significant difference was found between the two groups in the frequency of their learning during the three weeks.

2. Place

Where did you mainly study? • At home • At university • On the train or bus • Other

At homeAt

universityOn the

train or busOther

Control(n=46)

27 (59%)

12 (26%)

7 (15%) 0

Experimental(n=30)

13 (43.3 %) 0 14

(46.7%)3*

(10%)

3. MotivationControl Group

Experimental Group

3. By using a paper list, did you feel motivated towards learning essay phrases?

• Very motivated• A little motivated• Not very motivated• Not at all motivated

6. By using mobile devices, did you feel motivated towards learning essay phrases?

• Very motivated• A little motivated• Not very motivated• Not at all motivated

n M SD df t Sig. r

Control 46 2.63 .645 74 -2.01 .048* .23

Experimental 30 2.93 .640

T-test results comparing two groups on their motivation towards learning (N=76)

*p<.05

Students who learned essay phrases on mobile devices felt higher motivation than those who used the traditional paper-based list.

4. Pre-test - Post-test

• When comparing the results of the pre-test and the post-test , both groups marked higher scores (i.e., higher autonomy) in the post-test.

• As for the control group, there was nosignificant difference between the two tests.

• However, as for the experimental group, there was a significant difference between the two tests in terms of the psychological dimension.

M SD df t Sig. r

TechnicalPre-test 2.41 .682 18 -.578 .570 .14

Post-test 2.44 .567

PsychologicalPre-test

3.25 .373 18 -2.356 .030* .49

Post-test 3.42 .368

Experimental Group (Pre – Post)

T-test results comparing two tests on two dimensions (n=19)

*p<.05

Summary (Learner Autonomy section)

RQ4: If L2 learners study the phrases with the application, could they enhance the learner autonomy?

Answer: YES (partly)

• Both groups were homogeneous in terms of learner autonomy

• Those who learned on mobile devices felt greater motivation than those who used a paper list.

• Different patterns of learning were shown in each group.

• Those who learned on mobile devices showed significant difference between the pre- and post-test in the psychological dimension.

CONCLUSION & IMPLICATION

• MALL has advantages not only in L2 vocabulary building but also the enhancement of learner autonomy– Recall in the test and the writing task

– Psychological dimension• contributed to developing positive attitudes towards

autonomous learning

– Motivation• meaningful to provide the students with the

opportunity of MALL regardless of their initial hesitancy or resistance.

• Longer-term studies would be necessary to see more meaningful changes in learner autonomy.

Reference (CALL & MALL)

Chun,D.L., & Plass, J.L. (1996). Effects of multimedia annotations on vocabulary acquisition.

Modern Language Journal, 80(2), 183-198.

Kindai University (2013). Survey of college students about the use of information technology

devices. Retrieved 10th of August 2015 from http://www.kindai.ac.jp/130208.pdf

Kukulska-Hulme, A (2009). Will mobile learning change language learning? ReCALL, 21(2), 157–

165.

Laurillard, D. (2007) Pedagogical forms for mobile learning: framing research questions, In:

Pachler, N. (ed) (2007). Mobile learning: towards a research agenda. London: WLE Centre,

IoE.

Pachler, N., Bachmair, B., & Cook, J. (2010). Mobile learning: Structures, agency, practices. New

York: Springer.

Sharples, M., Taylor, J., & Vavoula, G. (2007) A Theory of Learning for the Mobile Age. In R.

Andrews and C. Haythornthwaite (eds.) The Sage Handbook of Elearning Research. London:

Sage, pp. 221-47.

Sato, T & Suzuki, A. (2010). Do multimedia-oriented visual glosses really facilitate EFL vocabulary

learning? : A comparison of planar images with three-dimensional images. Asian EFL

Journal, 12(4), 160-172.

Sato, T., Matsunuma, M., and Suzuki, A. (2013). Enhancement of automatization through vocabulary l

earning using CALL: Can prompt language processing lead to better comprehension in L2 reading?

ReCALL, 25(1), 143-158.

(References)

Benson, P. (1997). The philosophy and politics of learner autonomy. In P. Benson & P. Voller (Eds.), Autonomy and independence in language learning (pp. 18-34). London: Longman.

Holec, H. (1981). Autonomy in foreign language learning. Oxford: Pergamon. Little, D. (1991). Learner autonomy 1: Definitions, issues and problems. Dublin:

Authentik.Murase, F. (2015). Measuring language learner autonomy: Problems and

possibilities. In C. J. Everhard, & L. Murphy (Eds.), Assessment and autonomy in language learning (pp. 35-63). Basingstoke, UK: Palgrave Macmillan.

Oxford, R. L. (2003). Toward a more systematic model of L2 learner autonomy. In D. Palfreyman & R. C. Smith (Eds.), Learner

autonomy across cultures: Language education perspectives (pp. 75-91). Basingstoke: Palgrave Macmillan.

Pennycook, A. (1997). Cultural alternatives and autonomy. In P. Benson & P. Voller (Eds.), Autonomy and independence in

language learning (pp. 35-53). London: Longman.

Thank you for listening

Takeshi SATO

[email protected]

Fumiko MURASE

[email protected]

Tyler BURDEN

[email protected]

mailto:[email protected]



Education

Is MALL really useful?