LUR (Language Utilization Report) in Action. Strengthening On-Program Learner Self-Assessment.

FLAGSHIP ANNUAL MEETING MANHATTAN BEACH, CA

Dan E. Davidson President, American Councils Professor of Russian, Bryn Mawr College

MAY 19, 2014

Reinventing Study Abroad: L2 Outcomes and Beyond Overseas L2 capstone can support learning outcomes that include: intercultural competency (ICC), “global mindedness,” critical perspective and adaptation (IDI), as well as ILR Level-3 outcomes. Recent cognitive research has also established linkages between serious SA and the development of -critical thinking skills, thinking outside the box -access to multiple academic traditions and professional networks -advanced problem-solving, “multiple lenses,” heteroglossia, dialog -decrease in bias, -increased tolerance, -rapid task-switching (cognitive multi-tasking) -improved executive function and self-management skills, -ability to work and communicate effectively across cultural divides. Flagship grads have a distinctive advantage in today’s job markets: understanding of the internal dynamics that shape markets and social movements before they actually happen;

L-2/C-2 competence as central, not incidental these goals. Language can be challenging:

“Dialogism applies to individual words and utterances, but also to the language system as a whole, which is embedded with the products of a continuing generalized collective dialog with “other” users of the language. To know a language, you must also “know” the general collective dialog. A word is always half someone else’s. It has to be populated, adjusted, before it is yours. Words carry the scent of other voices.” Jakob Shpet, Inner Form of the Word, 1927

The Study Abroad Challenge: Making the Appropriate Culturally-Situated Choice

“In order to develop their own voice, the FL learner must first develop an “ear” for other voices.” Kramsch, 2014. “Given the increased complexity in the way we conceive of language, communication, culture, and learning in an era of globalization calls for greater reflexivity and interpretive capacities on the part of both teachers and learners. While modernist perspectives on FL education have favored doing over reflecting, and performance over analysis, late modernist approaches add a reflective element of self-awareness that enable language users to better interpret the communicative situations in which they are to make situated choices.” Kramsch, 2014.

Low-stakes and high-stakes social interaction (or Living with Cognitive Overload) Overseas L2 learner is challenged to: • make choices based on contexts that have real consequences • produce meaning with words (requesting, apologizing, thanking, etc.); • interpret situations requiring these speech acts; • monitor the appropriateness of their utterances (e.g., for register) • repair their utterances, as needed;

•The fixed model of the L2 NS is changing under the forces of globalization in the way languages are used, mixed, and “meshed” NNS is taught a normative language, but must process much more.

Low-stakes and high-stakes social interaction (or Living with Cognitive Overload) Overseas L2 learner is challenged. Learners need guidance in all of the above: Time management, engagement versus avoidance strategies Setting short and longer-term goals Interpreting why and how to engage in local communicative situations Becoming skillful interpreters of what they observe, of what is spoken, and what is not spoken. The higher the level, the more complex the contexts, and, very often, the higher the stakes.

MAINTAINING A REFLECTIVE STANCE ON LEARNING THROUGH REGULAR SELF-ASSESSMENT

The task requires expert balancing of challenge and support to the learner who will invariably experience phases of euphoria, cultural shock and occasional cognitive overload. Effective programs encourage a reflective stance through regular self-assessments, goal setting, and feedback from facilitators/advisors, occasional help in analysis of events from both the local and the student’s own perspective,

THE ONLINE LANGUAGE UTILIZATION REPORT (LUR)

consists of two parts: 1. Concise target-language time/activity map

2. Three open-ended questions about the student’s individual language and cultural use over the reporting period.

I. Language Use How many clock hours during the past 7 days did you spend using

L-2 in the following activities (remember that your individual language classes last for 90 minutes each).

Please use decimal-point notation, i.e. 1 hr. — "1"; 45 min. — "0.75";

30 min. — "0.5"; 20 min. — 0.30; 15 min. = "0.25"; 10 min. — "0.15". Please only record the activities for ONE week. Do not add the

time for the two week period of the report.

Language Utilization Report (Specimen) Student Name Started: Not yet started — Completed: Not yet submitted. Due: Sep 16

II. Communication Challenges A. What challenges did you encounter during the past week in operating in (your L2) ? (Consider interpersonal, interpretive, and presentational modes in responding, e.g., scanning detailed text for relevant information, understanding detailed written instructions or spoken advice, note taking, letter writing, holding the floor in a discussion, reacting appropriately to a culturally "sensitive" situation, etc.) Describe the situation briefly. B. In view of the challenges described above, what language/cultural resources will you need to master or have greater control over to deal more effectively, when a similar situation arises in the future? III. Communication Successes Describe a linguistic interaction in the L2 during the past week which might be seen to characterize your command of the language at its best. (This may be an interaction of any kind which you consider you handled well.)

IV. Goals for the week(s) Ahead Identify 2-3 thematic areas, language functions, or speech genres which you plan in the coming weeks to make stronger in your own Russian. (E.g., getting better at making requests or offering apologies, understanding certain recurrent colloquialisms in the speech of friends, catching references to contemporary target culture and/or politics, mastering abstract expressions, learning how to keep up a conversation, improving phonetics/intonation, becoming more proficient at offering toasts, responding to compliments, invitations, leading a discussion, chairing a meeting, etc. )

V. Additional Comments Please note any other observations that come to mind about your use of Russian during the past week?

Table 4. Descriptive statistics for ALL ROF students (Part 2)

Activity N Minimum Maximum Mean SD

Average weekly number of formal language learning classes 104 4.82 23.61 11.57 3.33

Average weekly number of hours spent on homework 104 0.00 25.66 7.17 5.20

Average weekly number of internship hours 104 0.60 19.33 5.02 3.65

Average weekly number of hours of tutoring sessions 104 0.78 11.97 2.94 1.37

Average weekly number of hours of acad. reading 104 0.00 9.64 2.26 2.45

Average weekly number of hours attending cultural events 104 0.00 9.66 2.47 1.80

Average weekly number of hours with host family 104 0.00 24.66 7.10 4.88

Average weekly number of hours in public transportation 104 0.00 16.56 3.08 3.33

Average weekly number of hours participating in OTHER activities 104 0.00 10.49 1.45 1.91

Average weekly number of hours spending on reading for pleasure 104 0.00 10.31 3.24 2.65

Average weekly number of hours following the press 104 0.00 9.50 2.46 1.87

Average weekly number of hours listening/watching local radio or TV 104 0.19 15.71 4.53 2.83

Average weekly number of hours with friends 104 0.25 39.28 10.22 6.71

Average weekly number of total hours spent on all activities 104 22.25 133.79 63.50 21.22

Average weekly number of total hours spent on acad activities (langclass, homework, tutorial, and acad read) 104 13.32 45.03 23.94 7.35

Average weekly number of total hours spent on three academic activities (langclass, homework, and tutorial sessions) 104 12.31 43.50 21.68 6.43

Average weekly number of total hours spent on reading activities (prof/acad read, read for pleasure, and read the press) 104 0.19 22.06 7.96 5.09

Average Weekly Number of Hours Spent on Activities: ROF By Post-test OPIs (All Years)

Activity Post-test OPI 2+ (N = 5)

Post-test OPI 3 (N = 60)

Post-test OPI 3+/4/4+ (N = 39)

Homework 1.8 6.9 8.2

Internship 6.8 5.0 4.9

Tutoring 3.5 2.9 2.9

Academic Reading 4.7 1.8 2.7

Cultural Events 1.4 2.4 2.7

Host Family 6.0 6.5 8.2

Reading for Pleasure 0.8 3.1 3.7

Following the Press 2.2 2.1 3.0

Local TV/Radio 3.7 4.5 4.6

Time Spent with Friends 8.1 10.0 10.9

Average Weekly Number of Hours Spent on Activities: By Heritage Status (All Years)

Activity All Heritage (N = 15)

All Non-Heritage (N = 89)

Homework 7.8 7.1

Internship 5.2 5.0

Tutoring 2.4 3.0

Academic Reading 4.3 1.9

Cultural Events 3.0 2.4

Host Family 6.8 7.2

Reading for Pleasure 3.5 3.2

Following the Press 2.2 2.5

Local TV/Radio 3.9 4.6

Time Spent with Friends 10.9 10.1

Percentage of Total Time Spent in L-2 Activities

• Friends 15% • Host Family 13% (3+), 9.5(3), 8.6(2+) • Academic 40% • Reading 14% (3+), 11%(2+/3) • Cultural events 4%(3+), 3.1% (3), 1.7% (2+) • Internships 10% • Other 4%

Reflection supports learner to better understand, internalize, and apply new learning; increases self-efficacy, improves motivation Field Experiment: Wipro, a tech support call center in Bangalore Francesca Gino & Gary Pisano, HBS 5-5-2014 learning by doing will be more effective if deliberately coupled with learning by thinking:

16-day Training conducted in preparation for service of a particular overseas client. Groups randomly assigned to control, reflective, and sharing groups. At the end of each training day, reflective group took final 15 minutes writing and reflecting on the lessons they had learned that day. Participants in the sharing group did the same, but spent an additional 5 minutes explaining their notes to a fellow trainee. Those in the control group just kept working at the end of the day, but did not receive any additional training. In comparison to the control group: Reflection group performed on average 22.8% better on the final test. Sharing group performed on average 25% better on the final test

Effects of formative self-assessment on self-regulation, performance, self-efficacy: LUR basis 1. Under the proper conditions, and if applied over time,

students who self-assess have been shown to learn better, develop deeper understanding, improved work product, and take more responsibility for their own learning.” (Landrum, Dochy et al, 1999; Panandero, 2013, 2014).

2. In the overseas immersion environment, the criteria for self-assessment are provided by communicative situations in which the students find themselves: do they cope with them effectively, or not, and by their teachers and tutors.

3. “I meant X, but that is not how it was received. Will find out whether it was my choice of words or intonation, that caused the mis-understanding.”

*Criteria alone do not guarantee that students will use them, but it does increase the probability that they will.

Self-Evaluation/Goal setting

“I have trouble expressing appreciation in Russian, whether that’s appreciation of a beautiful view or a well prepared meal. So far, I’ve been relying on a few standard expressions, and I’d like to be able to branch out more and clarify myself.” “I’ll have to pay special attention to the way my Russian friends express themselves when they are pleased about something. Unless I set a goal of focusing on these expressions of pleasure, I’ll keep on getting caught up in the intonation and miss the words that are being spoken.”

Self-Efficacy

“I spent Saturday afternoon with my new Russian friends from the church youth group. While discussing Dostoevsky, one of the boys in this group and I discovered we both have an interest in philosophy. From then on, we spent the evening trying to stump each other with difficult questions such as, “Which is more important: Freedom or Security?” I feel like I handled myself fairly well in this game…in any case, it was an excellent way to gain abstract vocabulary!” “This past week I’ve been thrown into a bunch of situations where I’m using Russian in ways I never have before. The role-playing exercises we use during class are, I’m certain, a huge help to me in this respect, but I still dread them. As a not so creative person, what I do best is memorize. I have trouble coming up with a character to play on the spot. It is not so much a lack of language as a lack of ideas. I supposed that I should look at this year as an opportunity to not only practice my language skills , but also my quick thinking skills. “ “Remembering to use иK-1 intonation in declarative sentences, lowering my sentence-final pitch during questions.”

Monitoring /Evaluation/Self-Efficacy

“It’s a small thing, but I finally managed to properly place the stress in вacилeocтpoвckaя and although my host mother didn’t comment since I had pronounced it correctly, I still felt proud that I pronounced it properly and spontaneously without having to remember where the stress was.”

Self-Evaluation/Goal setting

“Additional long-term goal, be able to fully, yet clearly discuss topics related to science fields, specifically biochemistry/virology, with native Russian speakers who don’t have a strong science background as I find myself constantly being asked to explain why I chose my major or am posed questions that I could answer in English, but not in Russian due to general lack of vocabulary, or even when I do know the Russian equivalent, often it seems a moot point since the Russian interlocutor is unfamiliar with the terminology and I am unable to adequately give a definition.” 2 weeks later “Having taken the time to learn basic vocabulary over the course of the week related to my major, biochemistry, I was able to explain to a Russian citizen, who had very little background knowledge of science, the basic main points of the scientific article I had read earlier, without them cocking their head and staring at me like I had grown two heads or being absolutely confused about what I was trying to explain.”

Self-Assessment

“This past week I went to see Mozart’s “Die Zauberflöte” performed in Russian. Although I already knew the plot and meaning of each of the arias and other songs, I was surprised how much of the general libretto or even the songs I understood, as I was expecting to understand less than maybe 50% of the interlude text and barely none of the songs, but I would say it was about 75-80% of the conversations and roughly 60-65% of what the performers were singing.”

Self-Efficacy

“A stranger came up to me in the subway asking for directions. Since I actually knew where the place in question was located (Mariinsky Teatr), I was able to explain the best way to get there, that is, which metro line, where to make a transfer, which station to get off of, which streets to follow and what bridges to cross. And given the fact that I didn’t have to repeat myself too many times, with quite few понятноs, I don’t feel completely hopeless in giving directions.”

Self-Efficacy

“One of the girls in my elective course approached me and decided to introduce herself and start up a conversation. She of course recognized that I was most definitely not another Russian student, but all the same expressed her surprise at how much I understood/was able to converse and that I, as a non-native speaker, was taking a course that in her opinion was difficult even for her as a native speaker.”

Forced Code-Switching

“This deals not only operating in Russian, but forced code-switching between English and Russian due to my elective course/science courses in general. Between the copious numbers of words directly borrowed from English and ‘russified’ (e.g. «тредмиллинг»)along with learning organisms by their Latin name along with other names of proteins and other scientific acronyms still in English that goes on, it starts getting difficult to concentrate in either one or the other language, and after several hours it genuinely starts leading to a minor headache.”

Evaluating Failure

“Hopefully this will be a one-off event, but on Thursday I simply could not operate in Russian. I felt as though I was back in my first semester of study. I barely could understand anything, had to ask teachers and other students to repeat themselves, could not function. This lead to much frustration, especially as it leads to people, especially Russians, trying to finish what I am trying to say for me. And although I understand the gesture to be friendly and showing that they’re trying to help, it ends up only exacerbating the situation and made everything worse.”

Monitoring Speech/Self-Management/Goal Setting

Register “Something that will probably always remain relevant: enriching my vocabulary for colloquial/slang. While it is nice to speak on a formal, professional level in a sterile, isolated environment, I have a desire to actually be able to converse additionally with the average Russian on the street and although I could get by with the formally taught standard set of clichés, I know it would distract the listener and that is far from the ideal situation. I have in mind especially all the Russian students with whom I attend my elective course. The girl who often chats with me passed comment on that while I for the most part can, at least in her opinion, sufficiently converse in Russian, she thinks I speak too formally, as though from a newspaper or a textbook.”

Self Efficacy/Identity Competence

“Talking to my host mom earlier this week and making her laugh was one interaction that I thought went well. Usually we don’t have personal conversations but instead talk about broad topics like the weather, current events and so on. On Monday we talked about her school and her class and I made a few jokes that she enjoyed so that was a nice thing for me. I am a person who likes to make jokes and play around but that seems to be the hardest part of someone’s personality to translate into another language/culture.”

Self-Assessment

“Going to the zoological museum on Thursday was interesting. I’ve gotten to the point were I don’t expect to be caught off-guard by the language. Unless it is a subject like nuclear physics with which I am in no way familiar, I generally have the capacity to figure out what is being talked about and contribute. So when I got to the zoological museum, I know a fair a bit about how animals are put together thanks to bioanthro, I was a bit shocked at how many of the signs I could not decipher and how much adlibbing I had to do to work around the glaring gaps in my vocabulary.”

Self-Evaluation/Self-Management/Auditory Learner Intervention

“I am unsure why I make so many mistakes when I write. When I read it out loud I immediately hear how incorrect it sounds, but somehow on paper I do not see it. I think I need to more carefully reread my writings and read more in the official style on order to fix this problem.”

Contact Maria D. Lekic lekic@americancouncils.org Dan E. Davidson ceo@americancouncils.org