Tracking context with usage and attention metadata in multilingual Technology Enhanced Learning

Riina VuorikariRiina VuorikariEuropean Schoolnet / European Schoolnet / Open Univ. of the NetherlandsOpen Univ. of the Netherlands

Bettina Berendt Bettina Berendt KU LeuvenKU Leuven

www.berendt.dewww.berendt.de

„„Learning Object Repositories Learning Object Repositories foster re-use and improve foster re-use and improve learning“ learning“

Really? Really?

Ochoa (2008), Vuorikari & Koper (2009): Ochoa (2008), Vuorikari & Koper (2009): re-use ~ 20%re-use ~ 20%

A basic challenge: contextA basic challenge: context

Political ScienceBiology

Researchingfor a

term paperin ...

MotivationMotivation(application goals)(application goals)

Use case Use case Technology Enhanced Learning / Learning Object Technology Enhanced Learning / Learning Object

Repositories Repositories teachers from different linguistic and country backgroundsteachers from different linguistic and country backgrounds

Goal: foster re-use of resourcesGoal: foster re-use of resources Assumption: understanding user context can improve Assumption: understanding user context can improve

system behavioursystem behaviour Approach:Approach:

consider UAM as part of context measurement, specifically:consider UAM as part of context measurement, specifically: Investigate different roles of social taggingInvestigate different roles of social tagging Profit from parallel research in different areas Profit from parallel research in different areas

Motivation Motivation (general - workshop)(general - workshop)

To exploit usage and attention metadata: need to define what properties of usage to use and

why how to gather data on that property what to do with that information

MotivationMotivation(general – our work)(general – our work)

Automatically gathered indicators of Automatically gathered indicators of contextcontext, , attention, ...attention, ... What‘s common across fields?What‘s common across fields? How can we communicate better?How can we communicate better? What‘s specific in T.E. What‘s specific in T.E. learninglearning??

How can How can tagstags help learning & teaching? help learning & teaching? How to bridge (e.g. How to bridge (e.g. linguallingual) barriers?) barriers? What are the best research methods for What are the best research methods for

answering all these questions?answering all these questions?

Types of context Types of context in TELin TEL

What is context?What is context? “Context is any information that can be used to

characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.” (Dey, 2001)

Any information not explicitly carried by the “surface level“ of an interaction with a computational system

Surface level: user issues a query, user accesses a resource, ...

We will distinguish between Macro-context Micro-context

Macro-contextMacro-context educational level formal and informal learning delivery setting (distance, blended, ...) intended users and user roles ...

Micro-contextMicro-context UserUser

traittrait statestate

InteractionInteraction atomic vs. activity structureatomic vs. activity structure implicit vs. explicit interest indicatorsimplicit vs. explicit interest indicators

Background knowledge / „semantic Background knowledge / „semantic enrichment“; often w.r.t. materialenrichment“; often w.r.t. material

(Berendt, 2007)(Berendt, 2007)

Tags and contextTags and context

What‘s in a tag?What‘s in a tag?

tag

item user • material

• user

• interaction• „writing“• „reading“

Background knowledge

Tags as interest indicators (1) – Tags as interest indicators (1) – or: measuring and using or: measuring and using interaction-contextinteraction-context

Comparative search efficiency:Comparative search efficiency: Search option [A|B|...] Search option [A|B|...] a target LO a target LO Click-through rateClick-through rate aka aka confidenceconfidence (fraction of successful (fraction of successful

searches)searches) Path lengthPath length

In LOR: Search with ...In LOR: Search with ... explicit searchexplicit search (for resources w/o tags or ratings): 4.4 (for resources w/o tags or ratings): 4.4

searches / resourcesearches / resource community browsingcommunity browsing (tagclouds, lists, pivot browsing): 3.9 (tagclouds, lists, pivot browsing): 3.9

searches / resourcessearches / resources social informationsocial information (community browsing or explict search with (community browsing or explict search with

interest indicators (tags, ratings)): 2.8 searches / resourceinterest indicators (tags, ratings)): 2.8 searches / resource Recommendation of search options?!Recommendation of search options?!

Tags provide context to better Tags provide context to better utilize background knowledgeutilize background knowledge

Can tags be used for enriching existing metadata of Can tags be used for enriching existing metadata of educational resources in a multilingual context?educational resources in a multilingual context?

30% of tag applications matched with descriptors from a 30% of tag applications matched with descriptors from a multilingual thesaurus, which had also been used to multilingual thesaurus, which had also been used to index these learning resources (Vuorikari et al., 2009)index these learning resources (Vuorikari et al., 2009) ““Thesaurus tags”Thesaurus tags”

Thesaurus tags provide forThesaurus tags provide for adding properties to tags (e.g., relation to a concept in a adding properties to tags (e.g., relation to a concept in a

multilingual thesaurus, language)multilingual thesaurus, language) linking resources to multilingual thesaurus descriptorslinking resources to multilingual thesaurus descriptors can support retrieval of resources in a multilingual contextcan support retrieval of resources in a multilingual context

Cf. convergence and quality tendencies in tagging in Cf. convergence and quality tendencies in tagging in general (Bollen & Halpin, 2009; Hayes et al., 2007)general (Bollen & Halpin, 2009; Hayes et al., 2007)

What is user context – What is user context – the case of languagethe case of language

User models have many different variablesUser models have many different variables LanguageLanguage (first language, further languages, (first language, further languages,

proficiency, preferences): an important proficiency, preferences): an important descriptor of usersdescriptor of users

Language situationLanguage situation (interaction in a first or (interaction in a first or second language): an important descriptor second language): an important descriptor

Linguistic trait and state context variables!Linguistic trait and state context variables!

Measures / indicators of Measures / indicators of user languageuser language

self-profiling that explicitly addresses the question self-profiling that explicitly addresses the question (“your mother tongue?”)(“your mother tongue?”)

IP addressIP address browser settingsbrowser settings language of the currently used interfacelanguage of the currently used interface language of search terms and tagslanguage of search terms and tags known or inferred language of tagged resourcesknown or inferred language of tagged resources

NB: indicator of the language situation: compare user NB: indicator of the language situation: compare user language with interface/resource language (e.g., language with interface/resource language (e.g., Berendt & Kralisch, 2009)Berendt & Kralisch, 2009)

Tags as interest indicators (2) – Tags as interest indicators (2) – or: tags for measuring (re-)useor: tags for measuring (re-)use

bookmarking and personal collections of digital bookmarking and personal collections of digital learning resources as a proxy for the use and learning resources as a proxy for the use and re-use of resourcesre-use of resources

When coupled with user and resource When coupled with user and resource location / language:location / language:

become proxy for (re-)use across national and become proxy for (re-)use across national and language boundarieslanguage boundaries

(e.g., Vuorikari & Koper, 2009)(e.g., Vuorikari & Koper, 2009)

Why support trans-lingual re-Why support trans-lingual re-use? (1): Situation on the use? (1): Situation on the WebWeb

What would be an „ideal“ language situation on What would be an „ideal“ language situation on the Web?the Web? One H0: ~ amount of materials proportional to One H0: ~ amount of materials proportional to

number of first-language speakers of a languagenumber of first-language speakers of a language Not the case – non-English languages are Not the case – non-English languages are

severely under-represented on the Web due toseverely under-represented on the Web due to Resource-creation behaviourResource-creation behaviour Link-setting behaviourLink-setting behaviour Link-following behaviourLink-following behaviour Attitudes towards resources in different languagesAttitudes towards resources in different languages

(Berendt & Kralisch, 2009)(Berendt & Kralisch, 2009)

Why support trans-lingual Why support trans-lingual re-use? (2): User re-use? (2): User preferencespreferences

Theoretical reasons (cognitive effort), supported by Theoretical reasons (cognitive effort), supported by implicit and explicit user preferences:implicit and explicit user preferences:

Clear tendency to access materials in one‘s mother Clear tendency to access materials in one‘s mother tongue in a medical information systemtongue in a medical information system

17% of users of a LOR had saved only content in their 17% of users of a LOR had saved only content in their native language in their Favourites native language in their Favourites

LORs: pragmatic reasons (curricula)LORs: pragmatic reasons (curricula) Search-engine give better results when queried with Search-engine give better results when queried with

linguistically correct (rather than Latinized) spellingslinguistically correct (rather than Latinized) spellings But: individual differences depending on proficiency in But: individual differences depending on proficiency in

English!English!

(Berendt & Kralisch, 2009; Vuorikari & Koper, 2009; Blanco & Lioma, (Berendt & Kralisch, 2009; Vuorikari & Koper, 2009; Blanco & Lioma, 2009)2009)

Using context for improving Using context for improving trans-lingual re-use: How? trans-lingual re-use: How? (1)(1)

better-organised search result lists better-organised search result lists first a ranked list in the preferred language, first a ranked list in the preferred language, then a ranked list in a second language, etc. then a ranked list in a second language, etc. done: Google for general search results and done: Google for general search results and

by LeMill (lemill.net) for learning resources. by LeMill (lemill.net) for learning resources. organise tag clouds by language or organise tag clouds by language or

country as in LRE (lreforschools.eun.iorg)country as in LRE (lreforschools.eun.iorg)

Using context for improving Using context for improving trans-lingual re-use: How? trans-lingual re-use: How? (2)(2)

for users with weaker language preferences; for for users with weaker language preferences; for „unknown“ users; ...“:„unknown“ users; ...“:

Make exceptions to the foregoing by recommendations Make exceptions to the foregoing by recommendations usingusing travel-well tagstravel-well tags: :

terms with the same or similar spelling in most languagesterms with the same or similar spelling in most languages ex.: technical terms like “mathematics”, place and person namesex.: technical terms like “mathematics”, place and person names How to identify them? How to identify them?

via multi-lingual thesaurus or similarvia multi-lingual thesaurus or similar tag has been assigned by users from different languagestag has been assigned by users from different languages Tag has been assigned to resources in different languagesTag has been assigned to resources in different languages

travel-well resourcestravel-well resources:: Have been bookmarked by people from different languages, country Have been bookmarked by people from different languages, country

contexts, …contexts, …

(Vuorikari & Ochoa, 2009)(Vuorikari & Ochoa, 2009)

Using context for improving Using context for improving trans-lingual re-use: How? trans-lingual re-use: How? (3)(3)

Direct recommendations 1 (see above):Direct recommendations 1 (see above): Recommend content or tags in a preferred languageRecommend content or tags in a preferred language

Indirect recommendations:Indirect recommendations: E.g., recommend bookmark lists of other users with a similar E.g., recommend bookmark lists of other users with a similar

“language preference profile” “language preference profile” Profile similarity: “extensional” and/or “intensional”: degree of Profile similarity: “extensional” and/or “intensional”: degree of

tolerance for mixed-language resources, results, etc. tolerance for mixed-language resources, results, etc. Direct recommendations 2: Direct recommendations 2:

repositories could specifically encourage users who are repositories could specifically encourage users who are competent in “smaller” languages also to author content in competent in “smaller” languages also to author content in their language.their language.

OutlookOutlook

Some questionsSome questions How to best couple „content“ and „architecture“ How to best couple „content“ and „architecture“

researchresearch How to bring together research from Web analytics / e-How to bring together research from Web analytics / e-

Commerce, search-engine research, digital libraries, Commerce, search-engine research, digital libraries, TEL, ...TEL, ...

How to complement the exploratory research with How to complement the exploratory research with experimental research effectively and efficientlyexperimental research effectively and efficiently

How to best combine different methods (validate that How to best combine different methods (validate that the dependent variables used the dependent variables used meanmean something!) something!)

Which privacy/security questions arise?Which privacy/security questions arise? What is specific for TEL?What is specific for TEL?

... and now yours! ... and now yours! (Thx!)(Thx!)

ReferencesReferences Most references can be found in the paper, available at Most references can be found in the paper, available at

www.cs.kuleuven.be/~berendt/Papers/TEL_context_attention_vuorikari_berendt.pdfwww.cs.kuleuven.be/~berendt/Papers/TEL_context_attention_vuorikari_berendt.pdf

In addition:In addition:

Bollen, D., & Halpin, H. (2009). Bollen, D., & Halpin, H. (2009). An Experimental Analysis of An Experimental Analysis of Suggestions in Collaborative Tagging. In Suggestions in Collaborative Tagging. In Proc. Of WI-IAT’09, Milan, Proc. Of WI-IAT’09, Milan, Italy, 15-18 Sept 2009.Italy, 15-18 Sept 2009. IEEE Computer Society Press. IEEE Computer Society Press.

Hayes, C. et al. (2007). ... In Berendt, B., Hotho, A., Mladeni\v{c}, D., Hayes, C. et al. (2007). ... In Berendt, B., Hotho, A., Mladeni\v{c}, D., & Semeraro, G. (Eds.) (2007). & Semeraro, G. (Eds.) (2007). From Web to Social Web: Discovering From Web to Social Web: Discovering and deploying user and content profiles. Workshop on Web Mining, and deploying user and content profiles. Workshop on Web Mining, WebMine 2006, Berlin, Germany, September 18, 2006, Revised WebMine 2006, Berlin, Germany, September 18, 2006, Revised Selected and Invited Papers.Selected and Invited Papers. LNAI 4737. Berlin etc.: Springer. LNAI 4737. Berlin etc.: Springer.