26
ärvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 THEORETICAL ISSUES IN THEORETICAL ISSUES IN CONTEXT MODELLING CONTEXT MODELLING Kalervo Järvelin ([email protected]) University of Tampere Finland

THEORETICAL ISSUES IN CONTEXT MODELLING

  • Upload
    chaela

  • View
    31

  • Download
    0

Embed Size (px)

DESCRIPTION

THEORETICAL ISSUES IN CONTEXT MODELLING. Kalervo Järvelin ([email protected]) University of Tampere Finland. Talk Outline. Lab IR - traditional study designs and contributions Contexts of IR Modeling context - issues in modeling A cognitive approach to modeling - PowerPoint PPT Presentation

Citation preview

Page 1: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 1

THEORETICAL ISSUES IN THEORETICAL ISSUES IN CONTEXT MODELLINGCONTEXT MODELLING

Kalervo Järvelin([email protected])

University of TampereFinland

Page 2: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 2

Talk OutlineTalk Outline

Lab IR - traditional study designs and contributions

Contexts of IR Modeling context - issues in modeling A cognitive approach to modeling Consequent research designs Conclusion - does it matter?

Page 3: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 3

The Lab IR FrameworkThe Lab IR Framework

Docu-ments

Docu-ments

Represen-tation

Represen-tation

DatabaseDatabase

Searchrequest

Searchrequest

QueryQuery

MatchingMatching

Represen-tation

Represen-tation

QueryResult

QueryResult

EvaluationResult

EvaluationResultEvaluationEvaluation

Relevance assessment

Relevance assessment

Recallbase

Recallbase

Page 4: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 4

The most popular focus -it is CS, after all, isn’t it?!

IR Research Goals IR Research Goals

Traditional goals: (a) theoretically (~formally) understanding

IR, (b) empirically testing IR methods, and (c) designing better IR systems

This can be seen in typical IR study designs - by populating the framework with variables

Page 5: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 5

Lab IR Study Types / DesignsLab IR Study Types / Designs

Study types: experimental methodological constructive theoretical

Experimental designs – IR evaluation focus areas with variables designs operate with

these fringe areas controlled

(no analytical variables) dependent variables: ER many hidden variables

DocumentsDocuments

Rep MethodRiRep MethodRiRep SetDiRep SetDi

TopicTopic

Query VersQjQuery VersQjMatching MethMkMatching MethMk

Rep MethodTjRep MethodTj

EvaluationProcedurePeEvaluationProcedurePe

Relevance AssessmentRelevance Assessment

RecallBaseRecallBase EvaluationResult EREvaluationResult ER

QueryResult AijkQueryResult Aijk

Page 6: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 6

LabIR: Hypos, Laws, Theories LabIR: Hypos, Laws, Theories

The dependent variables - typically recall and precision - using various metrics

The independent variables typically are the use or non-use of various techniques

The controlled variables - collection, requests

Thus Lab IR is about the explanation of the variation of recall and precision through IR techniques applied

Page 7: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 7

LabIR: ContributionsLabIR: Contributions IR technology Theoretical growth

theory expansion, e.g., due to enrichment of indexing theory

greater analytical power through formal model building improved empirical support through testing over several

test collections new research questions – within the given set of

variables

Theoretical growth has been limited by not bringing in radically new concepts or relations

Cannot possibly analyze real world applicability

Page 8: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 8

Lab IR Tests Lab IR Tests

IR research typically considers only retrieval tasks which most often are: (a) purely topical (b) content-only (c) well-defined (d) static, and (e) exhaustive

This is like saying that no matter what your situationis, your needs always arepurely topical, content-only, well-defined, static, and …… True, isn’t it?

Page 9: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 9

Small World?Small World?

Page 10: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 10

Psssst ...... IR in Context

Alternatives?

Page 11: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 11

One possible context:Task PerformanceAugmentation through IR

Augmenting Task Performance

Colleagues

Knowledge Methodology

Education/Training

Create Information Seeking

Remember/Use

Tools

Work Task Situation

Other SourcesEducation

Documents

Information Systems

Acquire Knowledge

Find DocsDocs Available

Retrieve Docs

Other Collections

Colleagues ...

DocDB FactDB

Context

Web

...

Page 12: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 12

Socio-organizational& cultural context

Nested Context FrameworksNested Context Frameworks

How much should one cover?

Work Task

WorkProcess

Task Result

Work task context

Seeking Task

SeekingProcess

Seeking Result

Seeking context

Docs

Repr

DB

Request

Query

Match

Repr

ResultIR context

How to model what is covered?

Page 13: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 13

What to cover?What to cover?

Clearly, current IR test designs are not representative of real world IR

In principle, whatever has important relationships with the

application of IR, should be covered however, what has, is not well known answering this requires theoretical modeling,

operationalization and empirical testing so, what is context?

Page 14: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 14

Context? What?!Context? What?! Dervin (1997):

there is no term that is more often used, less often defined, and when defined defined so variously as context - it has become almost a ritualistic invocation

for some, context has the potential of being virtually anything that is not defined as the focus

for others, it is inextricable surround that denies all generalizations

there are endless lists of contextual factors This is why we hate it - it is foreign to CS!

Page 15: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 15

Some Contextual ThemesSome Contextual Themes

Reality is discontinuous, knowledge partial Context is not useful as an independent

entity Context requires a focus on process and

relationships between product and process Need to focus on multiple interdependencies Context is a necessary source of meaning

(Dervin 1997)

Page 16: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 16

How to model? - IR Context How to model? - IR Context DimensionsDimensions1. Work task dimension2. Search task dimension3. Actor dimension4. Perceived work task dimension5. Perceived search task6. Document dimension7. Algorithmic search engine dimension8. Algorithmic interface dimension9. Access and interaction dimension

Each dimensioncontaining multiple

variables

Much more detailin the forthcomingbook by Peter&Kal

Page 17: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 17

http://www.springeronline.com/

Page 18: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 18

InformationObjects

InformationSystems

InterfacesCognitiveActor(s)

Organisat.

Cultural

= Cognitive transformation and influence

= Interactive communication of cognitive structures

Social

&4

3 2 1

5

6

8

7

Context

A Cognitive Model of IS&RA Cognitive Model of IS&R

Page 19: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 19

Context around and withinContext around and within

Informationobjects

IT: EnginesAlgorithms

InterfaceInformation

seeker

Org.

Cultural

SocialContext

Real Social & PhysicalWorld

Cognitive World

PerceivedInformation

objects

PerceivedIT: EnginesAlgorithms

PerceivedInterface

Situationi.e. perceivedtask – need

sources

Social

Org. / Cultural

Perceived context

Page 20: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 20

Augmented IR Research Goals Augmented IR Research Goals

Theoretically understanding the application of IR in / to context; not just engines but also their use in varying situations

Empirically describing / testing the application of IR in / to various contexts

Supporting the design of information systems, services and information management

This does not downgrade IR into Social Science but rather means more / better engineering

Page 21: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 21

New Research SettingsNew Research Settings

Extended laboratory experiments What kinds of work tasks are typical in Domain X? What kinds of search tasks do these generate? What kinds of documents tend to be relevant?

IIR experiments Lab approach but including the seeking actor

Research on interfaces / actors in context interaction of actors and information system

interfaces in diverse socio-organizational contexts

Page 22: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 22

Lab IR settings focused on Lab IR settings focused on domain/task domain/task variables variables and dimensions of and dimensions of ITIT and and information objectsinformation objects

Informationobjects

IT

TaskOrg.Context

Interface Actor(s)

Social

Cultural

Page 23: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 23

Lab IR settings focusing on Lab IR settings focusing on actor actor variables and variables and dimensions of dimensions of ITIT and and information objectsinformation objects

Informationobjects

IT

TaskOrg.Context

Interface

Social

Cultural

Work taskperception

Search taskperception

Actortypes

Page 24: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 24

ConclusionConclusion Context is endless and pervasive

its management requires perspective, goals and interests: what do we want to achieve??

we may see multiple, nested contexts Thus, what is IR about? As technology ...

developing systemsbased on a technological imperative?based on people’s communication / access needs?

As a discipline ... [vs. a technology] understanding ... perhaps ... but what? C-n-c-

l?

Page 25: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 25

Does context really matter?Does context really matter?

We do not know really (if, or how much) Some certainly wish / believe it does not I don’t believe them until shown that

no matter what one’s situation is, one’s information needs are of just one type

no matter of what type information needs are, they are always best served in one already known way

Page 26: THEORETICAL ISSUES IN  CONTEXT MODELLING

Kalervo Järvelin – Issues in Context Modeling – ESF IRiX Workshop - Glasgow 2005 26