Systematic Approaches to Literature Reviewing

The Literature Review ?

“Literature reviews …… introduce a topic, summarise the main issues and provide some illustrative examples.”

Agree? Disagree?

If they are to be considered a reliable source

of research evidence, literature reviews should record provide the reader with sufficient to be able to judge whether all of the relevant literature is likely to have been found, and how the quality of research considered was assessed.

1 Agree? 5 Disagree? 10

The Literature Review ?

Discussion

What is a Literature Review? Why do we do Literature Reviews? What are the main challenges?

Literature Matters

connection to findings

coverage

working understanding

critical appraisal

Holbrook, A., Bourke, S., Fairbairn, H., & Lovat, T. (2007). Examiner comment on the literature review in Ph.D. theses. Studies in Higher Education, 32(3), 337–356. doi:10.1080/03075070701346899

“importance of coherent argument”

“more than acknowledge the literature and pull it together… includes ‘working understanding’ ‘critical appraisal’ ‘ connection to findings’ and disciplinary perspective’”

Literature Matters

disciplinary perspective

scholarliness

Knowledge of range of literature

DEMONSTRATES

Systematic Review Means you need to be:

Critical Evaluate what you read

Analyse Extract differing information from what you read

Synthesise Show relationships between studies/sources; differing

definitions, concepts, theories etc. Evaluate

Methodological approaches/tools and techniques used

Systematic Review – Objective is to produce more than a summary A review of a clearly formulated question That uses systematic and explicit methods To identify, select and critically appraise

relevant research, And to collect, analyse, synthesise and

evaluate the research that is included within the review. 

Systematic Review Need to apply the same level of rigour to

reviewing research evidence as you will apply to producing your research evidence

Workflow for Literature Reviews

Search Assess Read Write

How do I start?

Start with an area you are interested in or need to find out about.

Look for hot topics, trends, emerging ideas, key questions, key issues.

Example Suppose I have searched for a topic I’m

interested in: Education Higher Level Computer Science Teaching and Learning Approaches For some purpose

What are key issues of interest ? To Ireland, Third Level in Ireland, DIT in particular

Peer Tutoring In Computer Science Programmes In Tertiary Education and its Effects On First Year Student Retention.

Clear context, clear statement of scopeClear context, clear statement of scope

Locating and Finding Research

Where do I start?

Need to decide what I need to tell the reader about.

Breaking my topic into pieces

So what do I want to find out?Student

retention

In CS @ 3rd level

Peer Tutoring

Effects on

students

Combine the results to make a case

Locating and Finding Research

Student retention

•What do I mean by this,

why is it a problem, why

are people interested in

it

•Why do students drop

out

For third level computer

science

•Why are people

interested, what is the

level of problem in the

area

•Why do student drop out

Locating and Finding Research

Peer Tutoring

•What is it

•How can it help students – all ways

•How can it help student retention in particular

Where do I start?

Previous Theses Review Papers Journal Papers Conference Papers Supervisor

Phase 1- Identify the Research

A broad but defined, systematic sweep Defined search terms

record recall and precision Recall is the ratio of the number of

relevant records retrieved to the total number of relevant records in the database. It is usually expressed as a percentage.

Precision is the ratio of the number of relevant records retrieved to the total number of irrelevant and relevant records retrieved. It is usually expressed as a percentage.

Defined search arena e.g. databases, citation indices, reference

lists from primary and review articles, grey literature, conference proceedings, research registers, the internet, individual researchers/practitioners

Other broad search limits, e.g. language, date,

TIPS! Document the

search protocol and record what

research was found

Systematically manage the

search output, e.g. using zotero,

endnote

Phase 2- Selection

Select from research using criteria related to your research question

Develop inclusion or exclusion statements, these might relate to purpose of the paper, study outcomes, research design, methods used, population worked with etc.

E.G. Review paper E.G. Paper by key author E.G. Year long studies E.G. Pure computer science

programmes only

TIPS! Document the

statements and

their purpose (might be

pragmatic or research related)

Search Log

Date Database Keywords Results

1/11/13 2pm ACM Digital Library Peer, tutoring, undergraduate

10 articles

4/11/13 SpringerLink Peer tutor*, “computer science”

5 articles

1 book

10/11/13 Scopus Student retention, undergraduate

0

Running the search How many titles and abstract can you check? How easy will it be to decide to accept or reject a

record? Record the reason for rejection for “Excluded

research”

Don’t stop searching when you’ve stopped searching

3. Critical appraisal of studies

“Assessing the quality of methodology is a critical part of the systematic review process”

No standard approach but there are hierarchies in fields of study

Critical appraisal of research

What would be appropriate to consider when critically

appraising research in your area?

25

4. Collect data & analyse

Evaluate Synthesise results of literature review

Tables to compare Descriptive

Author Year Aim Methods Conclusions Critique

Write up literature review - Structure Background Purpose/Research question Method Findings Discussion Implications/Recommendations

Voice – Guide the Reader Through

1. Assess the value of the literature

2. Explain the context research takes place

3. Emphasise limitations of existing research

4. Tell a story

Writing StyleIntroduction

What I will show you?Why?

BodyWhy this area?Don’t leave reader to fill gaps

ConclusionWhat we have seen?How this is relevant to research?

Evaluation and Literature Review Towards the end of your dissertation you will

refer back to literature review Do your findings confirm those of others? Does your work extend that of others? Does your work provide new meaning to the work

of others? Does your work break new ground? Does your work raise issues about the

methodological choices made in previous studies? Does your work challenge existing ideas on your

subject?

Comprehensive Literature Review

Coherent synthesis of past and present

research in the domain of study

Source: Dr Hazel Hall, Napier University

What are the main conclusions on

previous research in this area?

What are the key areas of debate in this

area?

Which aspects of this work are of most relevance to my

study?

What are the key concepts in this area?

What have been the main research

questions?

Where is existing knowledge “thin”?

How is this topic approached by

others?Where are the gaps in

literature?

What are the main

perspectives on this topic in

previous research?

Do parallel literatures exist for this topic?

Which discussions?

Which sub-themes? Which writers?

Which work is subject to

challenge?

Who are these

“others”?

Which existing work could be

extended?

In which subject areas has the topic

been studied?

Challenges in Conducting Literature Reviews Where Knowing where to start, (e.g. wide

then narrow, or narrow then wide?) and what to include

Knowing when to stop literature searching Knowing when to stop “perfecting” the

review Knowing how far to venture into the

literature of associated domains Your supervisor will help you through these