ESRC Scottish Doctoral Training CentreInformation Science PathwayTraining day 25th June 2014

Introduction to organisational research and case studies

Professor Hazel Hall

Institute for Informatics and Digital Innovation/School of Computing

This session

Theme is organisational research Context is Information Science Mix of lecture material and short exercises

Session begins with consideration of the distinctiveness of organisational research, then moves on to case studies

But first…

SOME INTRODUCTIONS

Organisational research and case studiesat doctoral level within the Centre for Social Informatics

Organisational case study

Organisational case study

Case studyOrganisational

research

Organisationalresearch

Research cited in this sessionTitle Organisational

researchBusinessresearch

Case study as output

Setting

Intranet implementation in a corporate environment

X X X Professional services firm

Blogs in the classroom X X Edinburgh Napier

E-information roles X IM/KM

Outsourcing research & information services

X Business information services

Research in Librarianship Impact Evaluation Study (RiLIES)

X Information science research

ORGANISATIONAL RESEARCH

Organisational research

What makes organisational research “different/distinctive”? Practical difficulties in accessing sites for data collection

Information sharing practice of drug dealers Strategies for dealing with information security breaches in Company X

Legal and ethical issues when setting up studies Power of the context in sites of data collection

Real-life organisations staffed by humans whose behaviours are influenced by range of factors – culture, politics, power struggles

Intangibility of phenomena under investigation Knowledge, value, social capital, goodwill

Expectation of the organisation to derive value from the study

Information Science and organisational research Borrows from other disciplines

because Information Science is concerned with range of organisational perspectives

technology, culture, functions…

Requirement to read widely Sociology, anthropology, management science… even physics?

Intranet implementation: Galison, P. (Ed), (1997). Image and logic: a material culture of microphysics. Chicago: University of Chicago Press.

As an applied science, organisational partners may expect return on participation

CASE STUDIES

Understanding of the term “case study”

Case study is an approach to research Empirical enquiry that investigates a contemporary phenomenon

within a bounded, real-life context especially when boundaries between phenomenon and context are not clearly

evident• Intranet implementation: “The reasons why they don’t use the intranet to

knowledge share [phenomenon] may be due to cultural issues [context]” uses multiple methods including, but not exclusively, qualitative techniques,

e.g. participant observation, interviews, document analysis• Intranet implementation: interviews; document analysis• RiLIES: interviews; citation “sketching”

Alternative understandings

The case study is the output of research “Story/ies” of the case(s) investigated

The knowledge trap: an intranet implementation in a corporate environment (http://hazelhall.org/publications/phd-the-knowledge-trap-an-intranet-implementation-in-a-corporate-environment/)

Hall, H. & Davison, B. (2007). Social software as support in hybrid learning environments: the value of the blog as a tool for reflective learning and peer support. Library and Information Science Research, 29(2), 163-187. (DOI 10.1016/j.lisr.2007.04.007.)

Enhancing the impact of LIS Research projects

(Text book exercise)

Case study approach for real-life, contemporary research Describe - explore – explain – illustrate – provide

examples Intranet implementation

“Here’s a real information-intensive distributed organisation that hoped an intranet would support knowledge sharing in the firm. I established that it did not to the extent anticipated, and propose reasons why with illustrations and examples.”

Blogs in the classroom “We wondered if claims that blogs can encourage student reflection were

exaggerated. We tested this by analysing the content of recent student blog postings in an educational setting, and demonstrated with examples that reflection is often limited.”

RiLIES “These five case studies of real LIS research projects show how a range of

factors can increase the impact of the research output on the practice of librarians.”

Case study approach for investigating “how” and “why” questions

RiLIES How can LIS research projects be conceived, designed and

implemented to increases the chances that their findings will influence the practice of librarians?

Intranet implementation Why don’t staff in this corporate environment use the intranet for

knowledge sharing?

Case study approach for triangulation

Collect data on specific cases to triangulate with other data collected

Case study (or studies) are just part of the project, e.g. RiLIES Practitioner poll Focus groups Validation survey and case studies

Single/multiple case studies as output

A study can include single or multiple cases

Intranet implementation: 1 (big) case study Blogs in the classroom: 1 case study RiLIES: 5 case studies

In case of multiple case studies, each should stand on its own

Rationale for single case study

A critical case – likely to have strategic importance for the general population Intranet implementation: focus on culture

‘If it is valid for this case, it is valid for all (or many) cases’. See http://heim.ifi.uio.no/~in166/h00/criticalcase.pdf

An extreme or unique case RiLIES: 5 case studies chosen were amongst the most frequently

cited in the practitioner poll as having influenced practice

A new/revelatory case Blogs in the classroom: no empirical studies conducted previously

(although plenty of claims made!)

A prelude to further study, to test ideas For example, a pilot case

Case study research design process

Five elements

1. Identify research questions to be explored

2. Determine propositions or hypotheses Bearing in mind that case studies themselves often generate hypotheses and

models to be tested in the future – by you, by other researchers

3. Select clear units of analysis

4. Analyse data in a logical fashion so that it can be tied back to propositions

5. Interpret findings

Case study research design process

Five elements: Intranet implementation

1. What is the role of an intranet in knowledge sharing?

2. External and internal organisational factors determine role This proposition was based on an analysis of sociotechnical literature that dated

back to the 1970s

3. Interviews and document analysis

4. Data analysed and reframed using actor-network theory (more on this later…)

5. Findings interpreted to uncover underlying explanations of practice

Case study research and “rigour”

Accusations of bias and lack of rigour in case study research because data from which findings derive belong to a specific context Poor reliability

Can you be certain that you would report the same findings if you ran the same study at another time or location?

• Intranet implementation: Perhaps not, but research protocol is such that the process could be repeated in another large information-intensive professional services firm (i.e. method is reliable)

Doubtful validity How can this/these case(s) be generalisable to the wider context? To what

extent is your case study “representative” of the population as a whole?• Intranet implementation: It can’t, but it does not seek to “generalise”• RiLIES: Multiple case studies can address this to an extent

Other “weaknesses” of case study research

Causal inferences cannot be made, and it’s not possible to “test” in a “traditional” sense Chemistry would give you Na2O + 2HCl = H2O + 2NaCl

In case studies only associations and correlations can be made

Processes can be time-consuming and cumbersome Organising access, non-disclosure agreements Requirement to be on-site Willingness of “participants” to participate Labour in transcribing interviews…

Events cannot be controlled

Intranet implementation: access agreed first week of September 2001 for interviews to start 1st October 2001…

Value of case studies

In-depth studies “Power of good example” derives from “rich” data

Intranet implementation

Particularly useful for new areas of research, where there is little/no extant literature and previous empirical evidence

is lacking Blogging in the classroom

Generate new hypotheses for future testing Blogging in the classroom

Often inexpensive Depends on depth of study (and how you transcribe interviews)

Resources

Research methods textbooks in business and management are useful for organisational research in general

Most general research methods textbooks include chapters on case study research

Three particularly useful texts Eisenhardt, K. (1989). Building theories from case studies.

Academy of Management Review, 14, 532-550. Flyvbjerg, B. (2001). Making social science matter. Cambridge:

Cambridge University Press. Yin, R.K. (2013). Case study research (5th ed.). London: Sage.

Flyvbjerg, B. (2001). Making social science matter. Cambridge: Cambridge University Press.

Yin, R.K. (2013). Case study research (5th ed.). London: Sage.

Intranet implementation: Flyvbjerg (2001) helpful to justify case study approach.

What are the main “questions” you would need to ask? Which methods could you use to collect data? Who would you collect data from? How will you organise and analyse the data that you have

collected?

An investigation into the impact of UK information science research

Exercise

Analysis of data for organisational research

Professor Hazel Hall

Institute for Informatics and Digital Innovation/School of Computing

In this section

Data analysis as part of the research process Data, evidence and findings Role of coding data in data analysis Coding exercise

Data analysis as a process Design methods gather evidence present case

Move from description of elements, e.g. object, people, phenomena “observed” to explanation, i.e. analysis.

Output is tied to purpose of the research and related research questions, with scope for extension discover what the research is really about new research questions may emerge example: intranets and information sharing power issues and knowledge

management

Refine & develop concepts – critical treatment

Research Established “theory”

Intranet implementation

Poor understanding of knowledge sharing with technologies

Blogs in the classroom

Blogs promote reflective learning

Contribution (Action)

Knowledge sharing practice is local.Efforts to knowledge share is influenced by power bases

(Adopt communities approach to KM in case study organisation)

Blogs do not promote reflective learning to the extent reported

(Pay attention to weekly blog hints to engineer reflection)

Refine & develop concepts – critical treatment

Research Established “theory”

Intranet implementation

Poor understanding of knowledge sharing with technologies

Blogs in the classroom

Blogs promote reflective learning

Contribution (Action)

Knowledge sharing practice is local.Efforts to knowledge share is influenced by power bases

(Adopt communities approach to KM in case study organisation)

Blogs do not promote reflective learning to the extent reported

(Pay attention to weekly blog hints to engineer reflection)

Data analysis

Data analysis

Belief in your research results Research findings are expected

to be grounded in evidence not to be based on speculation, nor on weak inference

Therefore decisions on data analysis are important Example from e-information roles study: apparently more opportunity in the public

and voluntary than the corporate sector. However: less obligation in corporate sector to advertise posts public and voluntary sector organisations could be playing “catch up” with the corporate

sector

We could not be confident that this finding was grounded in evidence because our data collection was not extensive enough

Inevitability of “too much” data

Assume it’s murder safety net

Not all data collected will be analysed - data collectively “emphasised” to serve as evidence to build a case

Tension present a set of understandable findings yet acknowledge the complexities of the social world under investigation

You can’t just say “this

way of working is better

than the old way”You can’t just say “this

way of working is better

than the old way”

Data, evidence and findings

Data + interpretation = evidence Evidence = social product/artefact of work completed

Evidence findings

Data cannot (normally) speak for itself, so data ≠ evidence evidence ≠ mere illustration evidence is built from multiple data sets

research design should permit multiple collection of “same” data for triangulation purposes

obligation falls on the researcher to check alternative claims for the evidence collected

Links between findings & research design Outcome of data analysis (findings) must be understood in the context

of the methods adopted Example from e-information roles study: globalisation the strongest driver in the

creation of new job roles in the corporate sector. However: Research design determined sample selection focused on large, multinational companies

So we were confident that the finding was grounded in evidence, within the context of our sample

Obligation to provide detailed and comprehensive account of both findings and basis on which they were obtained

Data analysis: some examples

Research Format of data Coding & analysis

Intranet implementation

Recorded interviews; interview notes; archive of company documents

Interviews transcribed; interview data coded using Ethnograph; archive details organised into historical sequence & coded manually – “content analysis”

Blogs in the classroom

Students’ blog entries; survey

Content analysis of blog entries; survey results not incorporated

e-information roles Focus group notes; job adverts; job descriptions; survey; telephone interview notes

Combined mind-mapping of focus group notes & job data; survey & telephone data analysed using Excel

Outsourcing research & information services

Interview notes; provider web sites

Interview data coded & analysed manually – total of 11 data sets

Data analysis options

Analysis using software Standard packages

Word Excel Access

Dedicated software SPSS - http://www.spss.com/ Ethnograph - http://www.qualisresearch.com/ Nvivo http://www.qsrinternational.com/products_nvivo.aspx Atlas.ti - http://www.atlasti.com/

Manual analysis

Use of Excel to analyse survey & interview data for the e-information roles project

Use of Excel to analyse survey & interview data for the e-information roles project

Relative ranking of the importance of employee backgrounds: computing, business, librarianship

Column M records comments

Date Data Source

8 November 2001

History – investmentBudget changes

Named meeting minutes

Use of Word to analyse document data for intranet implementation

Source of information

Date of activity/development

Activity/development

H: Who controls 422 the Intranet content, is it 423 controlled by you in XX … rather than 424 from the centre, from the KM group …? 425

#-CONTROL $-RELS KMGP: Well, in terms of what tools and what 427 -#-$ facilities are made available to us, 428 | | that's obviously controlled by the 429 | | central group. But in terms of the XX 430 |-$ content and the XX presence, that's 431 | entirely controlled by me … simply 432 | because it wouldn't be relevant to go 433 | through a central group. 434 -#

H: Yeh, OK. You've told me about 436#-INT BUY-IN ownership. How … it sounds as if 437 -# you've got really good buy-in from 438 | your own set of people … 439 | |P: Absolutely. 441 -#

#-INT BUY-INH: What about the Intranet as a whole in 443 -# the UK? What are your perceptions of 444 | buy-in there? 445 | |$-KM SPONSP: I think it varies. I mean, I'm very 447 -#-$ fortunate in that I report into the 448 | KM partner, who's also one of the 449 | senior partners … 450 -$

H: Which, who …? 452

Use of Ethnograph to analyse interview data for intranet implementation

“Translating” data for analysis - coding Coding

records instances of occurrence organises data into categories comprises part of the analysis stage in qualitative research

Attention to coding in research design Design of research tool has determined predefined codes

Indicate the best day of the week for team leader meetings:

A. MondayB. TuesdayC. WednesdayD. ThursdayE. FridayF. Don’t knowG. No preferenceH. Not applicable

Note also the importance of the last three options: there is a difference between not having a preference and not knowing; if this forms part of a survey of staff who have nothing to do with team leader meetings, there needs to be an option for their response. Attention to coding at the design stage can help with asking the “right” questions.

Note also the importance of the last three options: there is a difference between not having a preference and not knowing; if this forms part of a survey of staff who have nothing to do with team leader meetings, there needs to be an option for their response. Attention to coding at the design stage can help with asking the “right” questions.

Coding down

Data is coded according to predefined categories

identified in range of work brought together in literature review identified in a single piece of work commonly deployed, e.g. age breakdowns used in national statistics

Dimension Code Interpretation Evidence

Reflection C Content-free Comment makes no reference to points in the original entry.

U Non-reflective (U=’unreflective’)

Comment makes reference to the original blog entry, the module content or the general context in order to state an opinion, emotion or a point of fact or theory.

R Reflective Comment addresses points from the main blog entry and demonstrates a consideration of the validity of the content, the process or the underlying premise.

Propositional stance A Agree Comment actively supports the point made in the original entry.

I Indifferent Comment neither supports nor challenges original entry.

D Disagree Comment takes up a contradictory position to the original entry.

Affective P Positive Comment is encouraging, approving, accepting, etc.

E Even Comment appears affectively neutral.

N Negative Comment is hostile, discouraging, dismissive, etc.

Scheme based on Kember, D., Jones, A., Loke, A., McKay, J., Sinclair, K., Tse, H., Webb, C., Wong, F., Wong, M. & Yeung, E. (1999). Determining the level of reflective thinking from students’ written journals using a coding scheme based on the work of Mezirow. International Journal of Lifelong Education, 18(1), 18–30.

Example coding down: blog posting data coding scheme

Coding up

Data is coded according to categories suggested by the data

Revise codes as new insight is developed through the process of coding - further discovery of what the research is really about

Example from intranet implementation project: seven broad categories related the intranet under investigation

Content Functionality History KWorld Policy Staffing Uptake

Some data in this spreadsheet fits with predefined codes, i.e. in columns D-L. However comments need to be coded up.

Some data in this spreadsheet fits with predefined codes, i.e. in columns D-L. However comments need to be coded up.

Relative ranking of the importance of employee backgrounds: computing, business, librarianship

Column M records comments

H: Who controls 422 the Intranet content, is it 423 controlled by you in XX … rather than 424 from the centre, from the KM group …? 425

#-CONTROL $-RELS KMGP: Well, in terms of what tools and what 427 -#-$ facilities are made available to us, 428 | | that's obviously controlled by the 429 | | central group. But in terms of the XX 430 |-$ content and the XX presence, that's 431 | entirely controlled by me … simply 432 | because it wouldn't be relevant to go 433 | through a central group. 434 -#

H: Yeh, OK. You've told me about 436#-INT BUY-IN ownership. How … it sounds as if 437 -# you've got really good buy-in from 438 | your own set of people … 439 | |P: Absolutely. 441 -#

#-INT BUY-INH: What about the Intranet as a whole in 443 -# the UK? What are your perceptions of 444 | buy-in there? 445 | |$-KM SPONSP: I think it varies. I mean, I'm very 447 -#-$ fortunate in that I report into the 448 | KM partner, who's also one of the 449 | senior partners … 450 -$

H: Which, who …? 452

Value of software packages for coding and generating reports for analysis

Speaker

Code

Line numbers

Data coded

Advice pointers Be disciplined and systematic when analysing data

especially important to keep records of what you do if you dip in and out of your research work

Be prepared to account for what you have done in the report of your work another reason to keep good records

When designing data collection tools, look forward to data analysis good decisions at this stage may save a lot of work at data analysis stage – as will

be demonstrated in the class exercise!

The class exercise is based on the responses to questions 3.1, 3.2 and 3.3 in the e-information roles survey

The class exercise is based on the responses to questions 3.1, 3.2 and 3.3 in the e-information roles survey

Ability to align work activities to business strategy

Ability to connect with developments

Ability to cope with changeAbility to see the big pictureAbility to translate the needs of

the business at all levelsAbstractingAdaptabilityAnalytic mindBusiness acumenBusiness awarenessBusiness developmentBusiness focusCataloguingChange managementClassificationCollaboration – non-technicalCollaboration – technicalCommercial awarenessCommunicationComputer literacyConfidenceContract/supplier managementCreativityDiplomacyE-learning facilitationEmpathyEngaging audiencesEnterprise content management

EnthusiasmEvaluation of information sourcesFacilitationFlexibilityGrammarImaginationIndexingInfluencingInformation analysisInformation deliveryInformation governanceInformation literacyInformation retrievalInnovationIntegrityIntellectual property knowledgeIntelligenceInterviewingIT savvyKnowledge harvestingKnowledge of e-information

arena, new technologiesKnowledge of government policyKnowledge of information

sourcesKnowledge of lawKnowledge of public sector

vocabularyLanguagesLeadershipLiteracy

ManagementManagement of individualsManagement of teams MarketingMulti-taskingNegotiationNetworkingNumeracyOrganisationOutgoing personalityPolitical awarenessPresentation skillsPrince 2Problem solvingProfessionalismProject managementRecords managementRelationship buildingRelationship managementRepackaging informationResearchSelf-managementSmall business knowledgeSocial computingSpellingStakeholder managementStrategic thinkingSynthesising informationTaxonomy developmentTechnical abilityTime management

TrainingUnderstanding of

technical toolsValidation of information

sourcesWeb authoringWeb developmentWeb usability testingWorking under pressureWriting

How would you group these responses for coding?

How would you group these responses for coding?

Analytical tools and frameworks for organisational analysis

Professor Hazel Hall

Institute for Informatics and Digital Innovation/School of Computing

In this section Focus on tools and techniques for organisational and case study

analyses through a consideration of:

Purpose and output of frameworks Actor-network theory as a framework – with example of its application in the

research into the intranet implementation

NB there is a wide range of tools and techniques for research in general. Some (or elements of some) are more applicable to organisational and case study research than others. Actor-network theory is just one example for illustration purposes here.

Purpose of frameworks Frameworks

help make sense of data collected, and thus of phenomena (e.g. organisational dynamics) observed

act as a tool for diagnosis

and thus aid the processes of:

acquiring knowledge reflection action for change (if appropriate, for example in an action research setting)

Output of frameworks Frameworks provide you with a means of formatting your findings

e.g. as a graphical representation of the organisation under investigation

In using a framework you are encouraged to (re)organise your data understand what it is that your data represent present your findings in a format that is understandable to others – the

representation can be used as a short-cut to shared understanding

Actor-network theory as a framework - example

Background

Optimism associated with the development of systems to promote knowledge sharing is misguided.

Examples in the literature go back to 1980s. “Culture” often takes the blame.

Case study organisation wanted explanations as to why the efforts of its knowledge management staff to promote information systems for knowledge sharing were sub-optimal.

Actor-network theory as a tool of analysis History

Developed in 1980s Michel Callon and Bruno Latour

Key concepts Non-humans, as well as humans, are actors Relationships between actors shift as they compete for organisational resources,

from tangible, e.g. office space, to intangible, e.g. corporate attention Actor-networks grow through successful “translation” Actor-networks diminish/disintegrate when ties in the network loosen

Relevance of actor-network theory to this case

The organisation was understood as a mesh of competing actor-networks.

The success/failure of corporate initiatives was suspected to be related to the degree to which particular groups enhanced or diminished their organisational power-base.

Service delivery could be examined with reference to historical and social context of the organisation.

The approach provided opportunities to reflect, learn, act.

Actors in the organisation

External consultantsExternal consultants

Senior staff with KM responsibilities (not KM specialists)

Senior staff with KM responsibilities (not KM specialists)

Knowledge sharing as a concept

Knowledge sharing as a concept

IntranetIntranet

RepositoriesRepositories

Shared collaboration space

Shared collaboration space

Mission statementsMission statements

Specialist KM staff members in centralised unit

Specialist KM staff members in centralised unit

Specialist KM staff members in business units

Specialist KM staff members in business units

Senior sponsors of KM (not KM specialists)

Senior sponsors of KM (not KM specialists)

External systems vendors

External systems vendors

Intranet usage statistics

Intranet usage statistics

“Ordinary” staff (not KM specialists)

“Ordinary” staff (not KM specialists)

KM strategyKM strategy

KM as a conceptKM as a concept

Analysis phase 1

Mission statements

Mission statements

KM as a conceptKM as a concept

Senior sponsor of KM (not a KM specialist)

Senior sponsor of KM (not a KM specialist)

IntranetIntranet

Specialist IT/KM staff member in centralised unit

Specialist IT/KM staff member in centralised unit

Senior specialist IM/KM staff member in centralised unit

Senior specialist IM/KM staff member in centralised unit

Analysis phase 2Mission

statementsMission

statements

KM as a conceptKM as a concept

Senior sponsor of KM (not a KM specialist) Senior sponsor of KM (not a KM specialist)

IntranetIntranet

Specialist IT/KM staff member in centralised unitSpecialist IT/KM staff member in centralised unit

Senior specialist IM/KM staff member in centralised unit

Senior specialist IM/KM staff member in centralised unit

Senior specialist IM/KM staff member in centralised unit

Senior specialist IM/KM staff member in centralised unit

Specialist IM/KM staff members in centralised unit

Specialist IM/KM staff members in centralised unit

Some specialist IM/KM staff members in business units

Some specialist IM/KM staff members in business units

“Ordinary” staff (not

KM specialists) “Ordinary” staff (not

KM specialists)

Analysis phase 3Mission

statementsMission

statements

KM as a conceptKM as a concept

Senior sponsor of KM (not a KM specialist) Senior sponsor of KM (not a KM specialist)

IntranetIntranetSenior specialist IM/KM staff member in centralised unit

Senior specialist IM/KM staff member in centralised unit

Senior specialist IM/KM staff member in centralised unit

Senior specialist IM/KM staff member in centralised unit

Specialist IM/KM staff members in centralised unit

Specialist IM/KM staff members in centralised unit

Specialist IM/KM staff members in business units

Specialist IM/KM staff members in business units

“Ordinary” staff (not

KM specialists) “Ordinary” staff (not

KM specialists)

Some findings Central position of intranet, and its proximity to KM as a concept,

account for confusion over what KM represented in the organisation. Distance between policy documentation and “ordinary” staff explained

lack of engagement with KM, and what it implied in terms of behaviours.

Ties between KM staff in business units and “ordinary” staff strengthened over time at the expense of their relationship with the central KM team and the main tool of the KM implementation. As a result their commitment to KM weakened, as did that of their “ordinary” colleagues.

ESRC Scottish Doctoral Training CentreInformation Science PathwayTraining day 25th June 2014