Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Queen's Printer and Controller of HMSO 2007 Page 1
Understanding the Implementation and Integration of e-Health Services.
Research Report
Produced for theNational Institute for Health Research
Service Delivery and Organisation programme
January 2009
prepared by
Professor Frances S Mair
� Section of General Practice and Primary Care, University of Glasgow
Professor Carl May
� Institute of Health and Society, Newcastle University
Dr Elizabeth Murray
� Department of Primary Care and Population Health, University College London
Dr Tracy Finch
� Institute of Health and Society, Newcastle University
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 2
Dr George Anderson
� Section of General Practice and Primary Care, University of Glasgow
Dr Catherine O’Donnell
� Section of General Practice and Primary Care, University of Glasgow
Professor Paul Wallace
� Department of Primary Care and Population Health, University College London
Professor Frank Sullivan
� The Health Informatics Centre, University of Dundee
Address for correspondence
Professor Frances S Mair
Section of General Practice and Primary Care
Division of Community Based Sciences
1 Horselethill Road
Glasgow
G12 9LX
E-mail: [email protected]
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 3
Contents
Research Report .........................................................................1 Produced for the National Institute for Health Research Service Delivery and Organisation programme ..........................................................1
January 2009.............................................................................1
Contents.....................................................................3
Acknowledgements ....................................................8
1 INTRODUCTION .................................................9 1.1 Background .........................................................................9 1.2 Aims 10 1.3 Plan of investigation............................................................ 10
2 Developing the Conceptual Model ....................14
3 Work Package 1 (WP1): Summary and synthesis of literature on the implementation of e-Health ......................................................................18
3.1 Background ....................................................................... 18 3.2 Stage 1: Scoping Review – Aims/Objectives........................... 19 3.3 Stage 1 – Methods.............................................................. 19 3.4 Stage 1 – Results ............................................................... 21
3.4.1 Limitations of scoping exercise ....................................27 3.4.2 Scoping exercise conclusions.......................................27
3.5 Stage 2 – Aims/Objectives ................................................... 28 3.6 Methods ............................................................................ 28
3.6.1 Criteria for considering studies for this review ...............28 3.6.2 Search strategy for identification of studies...................29 3.6.3 Identification of papers for inclusion.............................31
3.7 Stage 2 - Results ................................................................ 32 3.7.1 Papers identified........................................................32 3.7.3 Thematic Coding of Systematic Review..........................39
3.8 Discussion and conclusions .................................................. 43
4 Work-package Two (WP2) Barriers and Facilitators to the Implementation of E-Health Services: The Perspective of Health Professionals....46
4.1 Background ....................................................................... 46 4.2 Aims/Objectives ................................................................. 46 4.3 Methods ............................................................................ 47
4.3.1 Design .....................................................................47 4.3.2 Task Groups..............................................................47
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 4
4.3.3 Setting and sample....................................................47 4.3.4 Key Informant Interviews. ..........................................50
4.4 Data Analysis. .................................................................... 52 4.5 Results .............................................................................. 53
4.5.1 Patient benefits and clear purpose. ..............................53 4.5.2 Collaborative working.................................................56 4.5.3 Standards.................................................................58 4.5.4 Ease of use ...............................................................60 4.5.5 Professional attitudes .................................................62 4.5.6 Clear rationale...........................................................64 4.5.7 Cost.........................................................................65
4.6 Results Using the NPM as a Coding Frame.............................. 67 4.7 Discussion. ........................................................................ 72 4.8 Conclusion: Recommendations for Implementation. ................ 75
5 Work package three (WP3) Development and Validation of the Technology Adoption Readiness Scale (TARS) ............................................................77
5.1 Background ....................................................................... 77 5.2 Aims/Objectives ................................................................. 78
5.2.1 Structure of the Work Package Report ..........................78 5.3 Phase 1: Item development & expert survey .......................... 78
5.3.1 Aim of phase 1 ..........................................................78 5.3.2 Methods ...................................................................78 5.3.3 Item construction and piloting.....................................78 5.3.4 Online survey of experts .............................................79
5.4 Results .............................................................................. 81 5.4.1 Description of sample.................................................81 5.4.2 Data analysis/results..................................................82 5.4.3 Key messages from Phase 1........................................82
5.5 Phase 2: Development of TARS ............................................ 83 5.5.1 Aim of phase 2 ..........................................................83
5.6 Developing TARS Generic..................................................... 83 5.6.1 Methods ...................................................................83 5.6.2 Results.....................................................................83 5.6.3 Testing TARS Generic with health care professionals ......84 5.6.4 Results TARS Generic .................................................85 5.6.5 Key messages from Phase 2........................................86
5.7 Phase 3: Testing and Validation of TARS Specific in study sites. 86 5.7.1 Aim of Phase 3 ..........................................................86 5.7.2 Methods ...................................................................86 5.7.3 Results.....................................................................88
5.8 Discussion ......................................................................... 90 5.8.1 Main findings ............................................................90 5.8.2 Methodological issues.................................................92
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 5
5.8.3 Comparison of findings with existing literature ..............93 5.8.4 Recommendations for further development & application
of TARS....................................................................93 5.9 WP3 Outcomes................................................................... 94
6 Work-Package 4 (WP4) The implementation of e-Health systems: perspectives of e-Health implementers on the integration of new technologies into everyday work. .................................................95
6.1 Background ....................................................................... 95 6.2 Phase 1 Aims and Objectives................................................ 95 6.3 Phase 1 Methods ................................................................ 96 6.4 Results .............................................................................. 98
6.4.1 Identification of Case Studies and Interviewees. ............98 6.4.2 Findings from case studies........................................ 100 6.4.3 Case Study 2: Picture Archiving and Communication
System .................................................................. 103 6.4.4 Case Study 3: Clinical Nurse Information System........ 106 6.4.5 Summary of overall findings...................................... 110
6.5 Discussion and conclusions ................................................ 111 6.5.1 Discussion and Conclusions....................................... 112
6.6 Phase 2 Development and Formative Evaluation of the e-Health Implementation Toolkit...................................................... 113 6.6.1 Introduction and Background. ................................... 113 6.6.2 Aims and objectives ................................................. 113 6.6.3 Methods ................................................................. 114
6.7 Results ............................................................................ 116 6.8 Discussion and Conclusions................................................ 116
7 Taking Analysis To A Higher Level..................118 7.1 Background ..................................................................... 118 7.2 Aims/Objectives ............................................................... 119 7.3 Methods .......................................................................... 120 7.4 Results ............................................................................ 120
7.4.1 Coherence .............................................................. 122 7.5 WP2 Analysis Using NPT .................................................... 125 7.6 WP3 Analysis using NPT..................................................... 129 7.7 The “Added Value” Provided by the NPM and NPT and a multiple
work-package approach..................................................... 130 7.7.1 Discussion .............................................................. 133
8 Conclusions/Key Findings ..............................135 8.1 Overview ......................................................................... 135
8.1.1 Implications for Policymakers .................................... 135 8.1.2 Implications for e-Health implementers. ..................... 138 8.1.3 Implications for Research.......................................... 140
References .............................................................142
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 6
Appendix 1 NPM Definitions ...................................150
Appendix 2 List of papers included in WP1, Phase 1 scoping review .......................................................154
Appendix 3 WP1 Papers included and excluded at full paper stage ......................................................168
Appendix 4 WP1 Details of included reviews..........176
Appendix 5 DATA EXTRACTED FOR THEMATIC ANALYSIS - SYSTEMATIC REVIEW..........................186
Appendix 6 WP2 Information letter Task Group .....262
Appendix 7 WP2 interview Invitation letter ...........266
Appendix 8 WP2 Summary of Key Findings ............271
Appendix 9 WP2 Task Group Schedule ...................274
Appendix 10 WP2 Interview Schedule....................276
Appendix 11 WP3 Sample characteristics of expert survey participants ................................................278
Appendix 12 WP3 Expert sample: Factor rating scores: frequencies, means and standard deviation279
Appendix 13 WP3 Phase 2: QUESTION ANALYSIS BY NPM CONSTRUCT....................................................282
Appendix 14 WP3 Analysis of rating items for Phase 2.............................................................................285
Appendix 15 WP3 Final set of TARS items mapped against WP2 data themes.......................................289
Appendix 16 WP3 TARS GENERIC...........................293
Appendix 17 WP3 Tars Generic: Means, standard deviations and frequencies.....................................300
Appendix 18 WP3 TARS SPECIFIC (Site 1) .............304
Appendix 19 WP3 TARS SPECIFIC (Site 2) .............310
Appendix 20 Sample characteristics for TARS Specific (Site 1 and Site 2) .....................................317
Appendix 21 WP3 TARS Specific Site 1: TARS items, means, standard deviations and frequencies..........320
Appendix 22 WP3 TARS Specific Site 2: TARS items, means, standard deviations and frequencies..........323
Appendix 23 WP3 Comparison of correlations between TARS Specific samples .............................327
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 7
Appendix 24 WP3 TARS Specific Site 1: Cross tab analysis of agreement with TARS items by perception of level of routineisation.......................330
Appendix 25 WP3 TARS Specific Site 2: Significant differences in comparison of Nursing/health staff (NH) with call handlers (CH) on TARS items, specified as percentages of professional group......333
Appendix 26 WP3 TARS Specific Site 2: Comparison of perception of not/partly routine (NP) with completely routine (C) by TARS items (bracketed figures refer to % within perceived routinisation grouping) ...............................................................334
Appendix 27 WP4 Interview Schedule for Case Study 1 (Choose and Book)..............................................336
Appendix 28 Salient Features of Selected Case Studies ...................................................................338
Appendix 29 WP4 Print-out of the e-HIT. ...............342
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 8
Acknowledgements
"This presentation/report/article/press release presents independent research commissioned by the National Institute for Health Research (NIHR) Service Delivery and Organisation (SDO) Programme. The views expressed in this publication/presentation are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. The NIHR SDO programme is funded by the Department of Health."
� We would like to acknowledge the contribution of Jo Burns, former researcher at Department of Primary Care and Population Health, University College London.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 9
The Report
1 INTRODUCTION
1.1 Background
E-Health has become more prominent in health policy over the last decade as policymakers throughout the world have recognised the possibilities it offers to transform the way healthcare is delivered. It is also a way of addressing health care challenges relating to changing demographics and the increasing prevalence of long term conditions. The United Kingdom Department of Health (DoH) estimates that 17.5 million people may be living with a chronic condition (DoH, 2004) while the World health Organisation (WHO) has suggested that by 2020 long term conditions could be the leading cause of disability (WHO 2002). E-Health may be able to address such problems, as well as improving quality, by facilitating self care and error reduction. It may also increase access to care, for example, by reducing inequalities resulting from geographical location.
The NHS plan (DoH 2000) required the development and implementation of Information and Communication Technologies to provide patient-centred services. Since the publication 10 years ago of Information for Health (NHS Executive 1998), many policy documents have been published with the aim of promoting e-Health (NHS Executive 2001; US National Institute of Medicine, 2000; Wanless 2002;Wanless 2004; DoH 2001;DoH 2002a; DoH 2002b; EU Commission 2004; EU Commission 2005). Within the UK, Connecting for Health (NPfIT/CfH) has been prominent in this activity.
However, despite rapid technological development the professional, organizational and institutional terrain into which this technology has been introduced has not accepted change readily. A key problem has been that of integrating e-Health services into professionals’ patterns of work, so that e-Health has rarely become part of routine service delivery. Utilisation of e-Health by health care professionals is complex as such services can change the dynamics of professional roles and relationships and affect the organisation of clinical work. Professional resistance to new technologies is often cited as a major barrier to progress but our previous work in the field of tele-Health suggests that such a view is overly simplistic (May et al. 2001a). Understanding barriers to implementation of e-Health services is crucial, however, if we are to overcome the apparent inertia of the NHS when it comes to introducing new technologies.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 10
1.2 Aims
In response to the commissioning brief for EH94 our study aims were to:
1. Explore a range of health professionals’ attitudes towards e-Health systems and to identify, describe and understand those factors that promote or inhibit the use of e-Health systems and other computerised tools across different health care professional groups and sectors
2. Identify promoters, inhibitors and factors which have little impact on e-Health implementation.
3. Develop a structured instrument to test the readiness of health professionals to use new e-Health systems and their potential for integration in particular clinical settings to be known as the Technology Adoption Readiness Scale (TARS)
4. Develop a model implementation toolkit which would draw on the results obtained from answering aim 2, to be known as the e-Health implementation toolkit (e-HIT).
5. Make practical recommendations regarding strategies that will facilitate and enhance the utilisation of e-Health systems by health care professionals across the UK.
1.3 Plan of investigation
To meet these aims we divided the project into four distinct but linked work-packages. Each was led by one of the Glasgow, Newcastle or London teams respectively, but all of those working in the project contributed to each work-package. Sharing the results of each work-package within the group helped us to better understand the processes of e-Health implementation.
Work-package 1 (WP1) had two components, an initial scoping exercise and a systematic review of the literature, both aimed at providing a “review a reviews”: essentially a summary of what was known about e-Health implementation. We also hoped to identify the gaps in this literature, so that we could make clear recommendations about the agenda for further research in this field.
Work-package 2 (WP2) examined health care professionals’ views about factors that affect implementation and integration of e-Health services, particularly their perspectives on barriers and facilitators to implementation. Participants were asked to comment on the findings from WP1 in order to see how professional’s perspectives resonated, or not, with the literature on barriers and facilitators. The objective of WP2 was to help us better understand the challenges involved in implementing and integrating e-Health services into routine health care delivery. That would enable us to make clear recommendations regarding the implementation and integration of e-Health services.
The purpose of work-package 3 (WP3) was to use information from the other work-packages to refine our robust normalisation process model (NPM). This would increase our understanding of professional and organisational capacity to implement and integrate e-Health services within routine health service delivery contexts. The main deliverable from WP3 was intended to be a structured instrument known as the Technology Adoption Readiness Scale (TARS).
Finally, the purpose of work-package 4 (WP4) was to identify, describe and understand promoters, inhibitors, and factors which had little effect, on the
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 11
implementation of e-Health systems. We planned to draw on these findings to develop an e-Health implementation toolkit (e-HIT) which could be a practical guide to those involved in implementing e-Health services.
Whereas much research has focused on barriers to implementing new technologies in specific clinical contexts, in this project we have taken a “whole systems approach”. We have used a theoretical model, the Normalisation Process Model, which attends closely to the processes through which innovations are played out and focuses on the “whole-system” in which they are located - consistent with the recommendations made by Greenhalgh (Greenhalgh et al. 2004) in their innovations literature review. Furthermore, while much research into the uptake of new technologies has tended to focus on the processes of technology diffusion and adoption, it is worth highlighting from the outset that this project has focused on workability and has explored the problem of normalisation, that is, the routine embedding of newly introduced technologies into everyday practice.
The NPM has been the conceptual model underpinning the work of this study (May 2006; May et al. 2007a; May et al 2007b) and has played a pivotal role within the study. The NPM consists of four constructs: interactional workability (IW); relational integration (RI); skill set workability (SW) and contextual integration (CI). Interactional workability refers to issues such as the ease of use of systems, while relational integration refers to issues of confidence and accountability. Skill set workability covers issues of training, workload and roles and responsibilities; while contextual integration refers to organisational issues such as resource allocation. A fuller description of each of the constructs within the model is included in Appendix 1. In this context, an objective of the study was to develop an evidence-based conceptual model of new technology adoption by NHS professionals that uses professional responses to e-Health technologies as a vehicle for analysis. Such a model will have added value if, like our other work, it applies across a range of technologies and contexts.
All work-packages in the study contributed to this objective, either by conceptual development (WP1); empirical investigation of specific operational contexts (WP2); development of a predictive instrument, the TARS (WP3); and pilot work for development of the e-HIT. The strong theoretical grounding of our research will maximise the transferability of our predictive instrument and implementation package and of our evaluation findings across e-Health systems and health care contexts. Importantly, while the NPM has been used to help us study the processes of implementation, the analysis and interpretation of findings from the individual work-packages has in itself contributed to the further refinement of the NPM (May & Finch 2009). The next section describes in further detail the theory development work undertaken during this project.
Defining e-Health
A challenge for those researching e-Health is to define the term, since the topic encompasses a broad array of electronic technologies related to giving and managing health care. We had no wish to repeat work previously commissioned by the SDO (Pagliari et al. 2005), and for the purposes of this project we adopted the definition of e-Health provided by Eng (Eng 2001):
‘e-Health is the use of emerging information and communications technology, especially the Internet, to improve or enable health and healthcare.’
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 12
However, to conduct our work-packages successfully we needed to have a clear way of defining and characterising the different types of e-Health. For example, within WP1 it was crucial that we have clear inclusion and exclusion criteria for our systematic review, and the first stage in this process was to develop clear definitions of the terms to be used. Furthermore, an understanding of how we were defining the different types of e-Health was essential to ensure consistency in terminology across the work-packages, thus ensuring that the research team approached participants in a uniform way and conceptualised the field of study in a clear and consistent manner.
The project team therefore, via an iterative process, devised a conceptual framework to guide data collection and analysis. It was not our intention to provide a new definition of e-Health, rather to be able to clearly define and categorise different types of e-Health services for the purpose of presenting a framework that was both broad in scope and tightly focussed.
We therefore divided e-Health into four domains, and the descriptions of these four domains were agreed via consultation with all co-applicants and researchers. The definitions of the domains were worked and re-worked periodically throughout the first quarter of the project in parallel with the initial scoping exercise as concrete examples of different types of service were identified and discussed by the research team and the precise way to categorise different types of e-Health services was refined. The intention was to develop categories that could be easily distinguished from one another, which were straightforward to describe to health professionals and which, encompassed those technologies most relevant in the contemporary UK National Health Service.
The four domains were:
1. Management systems. Including electronic medical record systems (for example the systems being implemented within NPfIT (Connecting for Health).
2. Communication Systems. Including synchronous and asynchronous information and communications systems used for clinical information sharing, ranging from e-mail through telemedicine and telecare systems.
3. Computerised decision support systems. Including computerised/automated systems used to support health professionals in working within clinical guidelines and care pathways or providing best evidence based care.
4. Information systems. Including web-based resources or e-Health portals for information retrieval
The definition of these domains is shown in Box 1.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 13
Box 1
Management Systems
A computer based system for the acquisition, storage, transmission and display of patient administrative or health information (including images) from various sources that can assist with administrative or clinical activities. Management systems are therefore intermediaries between users and a body of stored data with specific applications (for professionals or patients).
Communication systems.
This refers to the use of telecommunications systems as intermediaries where users are separated in space and/or time. This communication can be between health professionals or between health professionals and a patient or carer(s). This type of communication can be synchronous or asynchronous and involves the targeted exchange of information between specific individuals or individuals with specific roles for diagnostic, management, advice, educational or support purposes.
Computerised decision support systems.
A computerised decision support system acts as an intermediary between users (and future actions). It refers to a computer system that is interactive and rule based and uses two or more items of patient data to generate case or encounter-specific advice that can aid clinical decision making. Such systems would be automated, generally operate in real time and involve decision support from an artificial intelligence (e.g. computer system or software programme) rather than an individual (either in person or remotely located).
Information resources.
This refers to the use of internet technology by the public, health workers and others to access health and lifestyle information, services and support, where information resources are intermediaries between users and a body of stored data for general information (i.e. information is generic and not person specific) that is available for use by individuals or groups.
It is acknowledged that some e-Health services may fall within one or more categories but this was not viewed as problematic as the key requirement of this categorisation was that it was clear, concise, easy to understand, and that the research team were able to use it to categorise e-Health services in a standard fashion.
Section 2 describes the theory development undertaken during the project, while the methods and findings from each work-package are presented in Sections 3-6.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 14
2 Developing the Conceptual Model
The call for EH94 required applicants to locate their study designs and aims in an explicit theoretical framework. Although there were several theories to which our work could be related, at the outset, we took the view that this study was one of a number of concurrent opportunities to develop new theory that answered in a practical way some of the core questions that have been consistently raised by policy-makers and clinicians who seek to operationalize ways of thinking about, organizing, and enacting, new techniques and technologies for health care delivery. The development of the Normalization Process Model (NPM) has been a component in a series of sociological studies—mainly led by May— into the operationalization of professional knowledge and practice in healthcare. In particular these studies have been concerned with the implementation processes that lead to particular outcomes for complex interventions, rather than focusing on evaluating the outcomes of these processes. Of course, other theoretical perspectives explore elements of the same questions, and we deal with these elsewhere in this report. The purpose of this section is to describe the development of NPM and show how it both contributed to this study and was developed and refined within it.
Work leading to the development of the theory began in the late 1990s when May and Mair collaborated with others on two important ethnographic studies of the development, implementation and evaluation of telemedicine systems. The first of these, funded by the NHSE between 1998-2000 focused on design and implementation problems, and led to analyses of problems of workability and integration in practice (May and Ellis, 2001b; May et al., 2001a; Mort et al., 2003). The second, funded by the DoH Policy Research Programme, between 2000-2002, used the evaluation of telemedicine systems as a vehicle to explore the relationship between rigorous practices of knowledge production (randomised controlled trials) and the organizational contexts in which these are located. An important outcome of these studies was a paper offering a framework for the structured analysis of the relationship between evaluation processes and communities of practice (May et al., 2003a).
This analysis located ‘normalization’ as an endpoint of implementation processes, defining it in terms of a relatively stable set of practices that led to these health technologies becoming routinely operationalized in everyday work (embedding), and sustained in practice (integration). One result of this work was a further collaboration, this time with the Virtual Outreach Trial team led by Paul Wallace at UCL during 2002. In this work (May et al., 2003a, we sought to synthesise the results of a large corpus of qualitative data—using constant comparative analysis (Strauss, 1987)—to define the conditions that would lead to the normalization of telemedicine services in practice. We now had two conceptual working models of normalization processes, one focusing on how evaluation studies act as a vehicle that makes stable clinical practice possible, and the other focusing on the policy and organizational processes. These formed the theoretical basis of our application for EH94.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 15
Although these papers have been regularly and supportively cited, they formed only a conceptual model. This mapped a set of processes and relations but said little about what these meant in practice, or how they worked. Meanwhile May was working with colleagues on a series of papers that synthesized other areas of research. These added conceptual models of interpersonal relations in the clinical encounter (May et al., 2004); the production and reproduction of new patterns of work organization and allocation (May, 2005); and the development of robust socially patterned knowledge in healthcare systems (May et al., 2006). Each of these papers shared a common method: synthesising prior qualitative studies, and reanalysing their data. They also shared a common product: a set of analytic propositions or theoretical assertions which all focused on the organization and practice of professional work, across a range of contexts.
The results of these synthetic analyses were drawn together in work leading to an ESRC Research Fellowship for May (2004-2008). An explicit objective of this fellowship was the development of a middle range theoretical model that would assist in the process evaluation of large Health Technology Assessment trials. The development of this explicitly theoretical model—the Normalization Process Model (May, 2006)—coincided with the early phases of EH94, but at this stage it was not clear that it could be applied to this or other studies. Theory development nevertheless continued, with a review and critique of theory in the area (May, 2007), and major collaborative papers (May et al., 2007a; May et al., 2007b) that set out the model as a coherent and robust set of tools that fulfilled the three primary requirements of a theory (May et al., 2007a):
1. Accurate description. A theory must provide a taxonomy or set of definitions that enable the identification, differentiation, and codification of the qualities and properties of cases and classes of phenomena. The analytic assertions and propositions developed in earlier synthetic analyses of qualitative data perform this function.
2. Systematic explanation. A theory must provide an explanation of the form and significance of the causal and relational mechanisms at work in cases or classes of the phenomena defined by the theory, and should propose their relation to other phenomena.
3. Knowledge claims. A theory must lead to knowledge claims. These may take the form of abstract explanations, analytic propositions, or experimental hypotheses. They may also map relations with other phenomena that are believed to possess similar qualities and properties.
Further, it fulfils a fourth, but not mandatory, component of a theory in that it proposes a means of testing its knowledge claims:
4. Investigation. A theory must be testable. Such tests may be abstract (i.e. formal logical representations, simulations, or thought experiments); or concrete (empirical investigations).
As described in Section 1.3 the NPM consists of 4 constructs: interactional workability, relational integration, skill set workability and contextual integration: Appendix 1 provides full details.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 16
This model was empirically ‘tested’ by the co-applicants in a series of data clinics where we jointly interpreted and analysed qualitative data collected in this study. Parallel processes of collaborative data analysis using the NPM also took place in other studies. This made it clear that the NPM was robust and could provide an analytic advantage in interpreting data collected in this study and in developing a readiness assessment instrument and implementation toolkit. The NPM was therefore used to inform development of the work-packages and also to analyse data within these work-packages.
The NPM in its published form at the end of 2006 (May, 2006), was sufficient as a tool to analyse specific processes—the material practices through which the operationalization of e-Healthcare systems is accomplished—but it was insufficient to account for other important processes. This was demonstrated by applying the model empirically to experimental (Wilkes, 2007), qualitative (Gask et al., 2008; Mair et al., 2008), and review (Elwyn et al., 2008) data. This showed that it was useful in explaining factors that promoted and inhibited collective action in operationalizing practices. For example, the interactional workability construct addressed the ways in which e-Health services helped or hindered professionals in performing various aspects of their work; while relational integration addressed professionals’ confidence in the safety or security of new systems; skill set workability examined how e-Health services affected workload and training requirements; and contextual integration looked at issues of resource allocation, infrastructure and policy.
It did not, however, explain how those interventions were formed in ways that were sustained, how actors were enrolled into them, or how they were appraised. May and Finch therefore collaborated to extend the model to a middle range theory by exploring new domains of coherence (the work of making a complex intervention hold together and cohere to its context), cognitive participation (the work of engaging and legitimising a complex intervention) and reflexive monitoring (the work of understanding and evaluating a complex intervention in practice). We describe in detail elsewhere the methods by which the theory was built (May and Finch, 2009; May and Finch, forthcoming).
The development of a set of explanatory ideas around normalization has shifted from an initial set of empirical generalizations presented as synthetic propositions or assertions (May et al., 2003b), to a robust conceptual model that presents generalizable propositions (May, 2006), and finally to a middle range theory that offers a set of mechanism-based explanations for processes of implementation, embedding and integration (May and Finch, 2009). Our aim in doing this work has been to develop a robust explanatory model that can underpin structured, prospective, and policy relevant studies, and which are genuinely open to interdisciplinary inquiry. Unusually for a theory building process, this has been a collaborative one, in which co-investigators in this study and others have made important contributions to the development of explanation.
Due to delays encountered within WP3 it became possible to incorporate elements of the extended NPM in this work-package, while reanalysis of collected data permitted exploration of its value in work-packages 1 and 2. The added value of this is discussed in Section 7.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 17
This section has shown that the project has benefited greatly from being part of a wider programme of work involving the co-applicants, which has both contributed to further theory development of the NPM and increased the breadth of analysis possible within this project.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 18
3 Work Package 1 (WP1): Summary and synthesis of literature on the implementation of e-Health
3.1 Background
How best to implement new systems of practice – new technologies and their associated ways of working – is a core problem for health service managers and clinicians. The large literature on the topic reflects the challenges of implementing, integrating and embedding e-Health systems into routine health care delivery (see Section 1.1). This literature is often inaccessible – either because it ranges across disciplines in a way that makes appropriate papers hard to find without specific subject expertise, or because it focuses on systems in a highly specific way – so that people charged with implementing e-Health systems in the NHS find it difficult to locate the appropriate evidence base. If they do, it is often difficult to determine its relevance to current circumstances. In order to identify factors which promote and inhibit e-Health implementation processes, and therefore to address one of our main research objectives, we conducted a review of reviews. This allows us to synthesise and highlight the key messages from this literature in an accessible way.
A secondary review is preferable to further primary systematic reviews as it allows us to provide an overview of this field of research and to identify strengths and weaknesses in the literature. A theory-led review also demonstrates gaps in the published literature and produces clear guidance for the direction of future research. The aims of WP1 were therefore to:
� summarise the e-Health implementation literature;
� identify and understand barriers and facilitators to e-Health implementation, using both thematic and framework analyses of the review data;
� produce a taxonomy of problems that could serve as the focus for discussion by expert task groups and key stakeholders within WP2;
� inform the development of the predictive instrument and implementation enhancement packages within WPs 3 and 4.
To achieve our study aims rigorously and efficiently we summarised and synthesised the literature in two stages.
(a) Scoping Review: We used free text search terms to identify papers, rather than more formal searching techniques. The objective of this stage was to identify a broad range of papers relating to the implementation of e-Health, and so to understand how scholars have generally framed implementation problems. This would inform the development of a thematically derived taxonomy of barriers and facilitators. This taxonomy
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 19
then informed the interviews in WP2, and the development of the structured instrument, the Technology Adoption Readiness Scale (TARS) in WP 3. Many of these papers could not have met the rigorous methodological benchmarking of a true systematic review, but nevertheless offered important insights.
(b) Formal systematic review of reviews: We focused on structured systematic and narrative reviews of studies of implementation of e-Health interventions. We conducted a rigorous systematic review of this material, but in a novel move then analysed these papers using (i) the thematic framework developed in the scoping review, and (ii) the NPM. This provides a unique and extremely robust process of theoretical triangulation in a systematic review that allows us to make clear statements about the gaps in the literature and recommendations for future research.
For both reviews the definition of e-Health was that outlined in Section 1.3 and the literature was also categorised into the four e-Health domains (management, communication, computerised decision support and information systems) described in Section 1.3, Box 1.
3.2 Stage 1: Scoping Review – Aims/Objectives
The aim of the scoping exercise was to identify the main barriers and facilitators to e-Health implementation that could be used to inform other work-packages within the project.
Objectives
� To undertake a broad examination of the e-Health implementation literature, including papers that would not meet the strict scientific criteria required of a systematic review.
� To undertake a thematic analysis of the e-Health implementation literature.
� To produce a list of barriers and facilitators to e-Health implementation, based on the identified literature which could be used to inform the work to be undertaken within other project work-packages.
3.3 Stage 1 – Methods
Scoping Exercise Search Strategy
The study team developed a broad search strategy, designed to include all e-Health domains, and entered this into Medline for 1990-2006. This period covers key developments in e-Health, as well as pertinent developments in health and technology (e.g. the spread of the World Wide Web after 1995), and it is noteworthy that the term “e-Health” did not come into widespread use until after 2000 (Pagliari et al 2005). As we were undertaking this scoping exercise to provide, amongst other things, a framework of
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 20
knowledge to discuss with health professionals, it was most appropriate that we searched a database orientated to this group. Medline, more than other databases, is aimed at health professionals as much as it is at academics, so papers identified in such a search would be most likely to reflect an e-Health knowledge base with which professionals are already, to some extent, acquainted. Also, a MedLine review was deemed sufficient to permit a rapid overview of the main issues to inform other WPs, because of its broad scope within a medical focus, its ability to filter reviews from original research papers, and its frequent use in other systematic reviews (de Lusignan 2005, Hanson et al 2001, Jaatinen et al 2002).
The free text search terms used were:
Barriers OR facilitators OR implementation OR adoption OR translation OR
stabilization OR normalisation OR readiness AND:
e-Health OR "electronic medical records" OR "computerised decision support" OR "health informatics" OR telecare OR tele-Healthcare OR tele-Health OR telemedicine OR teleinformatics OR telecommunications health (OR + medicine
OR + nursing) OR tele* OR + computers + health (OR + medicine OR + nursing) OR software + health (OR medicine OR nursing) OR information + communication + technologies OR "the internet" + health OR "electronic mail" OR e-mail/email OR "chat room" OR "bulletin board" OR CD-ROM OR multimedia OR hypermedia OR "virtual reality" OR "audiovisual aids" OR "cellular phone" OR mobile tele*
We also restricted our search to English language articles involving human subjects.
Inclusion criteria and exclusion criteria
Papers were included if they contained a quantitative or qualitative secondary synthesis of evidence, and if their conclusions were empirically substantiated. Ideally, the reviews would be systematic, clearly explaining what they reviewed, why, and how the material reviewed was obtained. Reviews of particular systems and narrative reviews were also included if they contained a secondary synthesis of evidence. Our inclusion rules for secondary synthesis were much broader than those of a formal systematic review, since our aim was to gain wide insight into barriers and facilitators. We therefore included papers that did not employ systematic search strategies so long as the commentary drew upon syntheses of: (a) the literature or (b) case studies, pilots, services, or trials to substantiate the narrative.
Studies were excluded if they contained no secondary synthesis of evidence, little or no reference to empirical work, were driven by speculative argument or measured outcomes rather than reviewing real data. Also, as this study was focussed towards the experiences of professionals, (as mandated by the commissioning brief) papers that exclusively focussed on patient experience were excluded.
The project researcher (GA) screened all identified abstracts for potential inclusion. Full papers were obtained for those identified as potentially
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 21
relevant and assessed against the inclusion and exclusion criteria. The PI (FM) reviewed marginal papers.
A thematic analysis of reviews
Data was abstracted from eligible studies using a specifically designed data extraction tool previously piloted on 10 papers. Data extraction was in two stages: (1) details of the paper, setting, methods, aims and conclusions were recorded; and (2) each paper was assessed for content in relation to pre-defined themes based on the research brief and a preliminary review of the literature. The themes arose from the aims of the review - to identify barriers and facilitators to the implementation of e-Health systems - and also included the impact of e-Health technologies on the relationship between patients and professionals. and the efficiency and ease of use of the technology.
The development of the thematic review was guided by the methodological literature, which indicated that thematic analysis should involve identifying prominent or recurrent themes, summarised under each thematic heading (Dixon-Woods et al, 2004, 2005). The content of these headings was used to identify and define overarching thematic categories, leading to a greater understanding of the topic. Thus, the relevant literature was examined and identified under each thematic focus, identifying both unifying concepts within each theme and discrepant issues. These were used to construct a narrative analysis for each theme. These were then reviewed across each theme, again to identify possible unifying concepts, as well as discrepant issues.
The analysis was conducted primarily by GA. Verification of the thematic analysis was obtained by data sharing and discussion with three other members of the team (FM, TF, CM) and by sharing the findings with members of the project team (KO, EM, PW, JB, FS) and with an Expert Advisory Group of international authorities in the field.
3.4 Stage 1 – Results
Identification of the literature
We identified 719 potentially useful citations and 62 of these were included in the thematic review. Of the 657 excluded, the main reason was that they were not focused on e-Health systems. Others were not secondary syntheses of primary evidence (they did not "review" original studies). More than half (34/62) of the included papers originated in the US; 8 were from the UK; 6 were from Canada. The rest originated from Europe, Hong Kong, Australia or New Zealand. Appendix 2 provides the full list of papers included in the scoping exercise with accompanying basic details. Of the 62 papers, 9 were generic; that is, they covered more than one e-Health domain; 8 were concerned with management systems; 29 with communication systems; 7 with computerised decision support systems and 9 with information systems. The quality and detail of the literature was mixed. The methods and focus of the reviews also varied: 12 were classifiable as systematic, 23 were narrative literature reviews while the remaining 27 were essentially reviews of systems and national policies with reference to relevant literature. Some papers were chiefly concerned with software design and specific systems, with only a brief mention of barriers and facilitators to implementation (Hussein et al 2004, Carrino et al 1998),
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 22
while others focused on these exclusively (Bond 2006, Weinstein et al 2001). However, the literature at all levels of detail presented similar emergent themes.
Summary of the main factors
The scoping review identified seven main factors that affected the implementation of e-Health systems. These were:
1. Conditions prior to implementation.
2. Cost.
3. The need for and impact of validation and evaluation.
4. Professional attitudes.
5. Ease of use of the system.
6. Security, confidentiality and standards.
7. Education and training.
There was some over-lap between these factors, and they are not presented here in a hierarchy of importance or in frequency of occurrence. Rather, they may be regarded as the key distinguishable barriers and facilitators emerging from the thematic analysis of the literature, though the extent of this distinction varied, as is demonstrated below.
Conditions prior to implementation
The literature showed that conditions prior to implementation were regarded as integral to the selection of a site for an e-Health intervention, in all e-Health domains. These conditions had two dimensions: those within the organisation itself (be it a health trust or a GP surgery); and broad societal conditions (e.g. the level of political support for the implementation of an e-Health system). For example, May et al (2003c) asserted the importance of a positive link between an organisation and a policy sponsor in the successful implementation of telemedicine.
An organisation’s willingness and readiness to adopt and operate an e-Health system along, with that of the wider community (Jennett et al 2004) were important facilitators to implementation. Many reviewers recommended adequate preparation for an intervention. This may be in terms of strategic planning (Anderson et al 1997, Jerant 1999), establishing a vision as to what is to be achieved, and preparatory needs analysis of the site(s) in question (Cook and Dupras 2004). Yellowlees (1997) recommended that “telemedicine applications and sites should be selected pragmatically, rather than philosophically”. In short, good planning and prior knowledge of the site in question were identified as important facilitators to e-Health implementation.
Other sources suggested that the initial planning of system implementation may depend on a positive relationship between agencies. Established inter-agency co-operation was a precondition of successful system implementation; however, it was also integral to the process and sustainability of such success and is hence more than merely a prior condition. However, a system could not be implemented in isolation (Sheng et al 1997) and required input from providers, financers, and all agencies concerned (Cook and Dupras 2004). Recommendations from experts in the field may be important (Jerant 1999). There might also be a need for sponsorship (Anderson and Aydin 1997) and good marketing of the system, depending on the financial structure of the health service concerned.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 23
However, successful implementation of any e-Health system seemed reliant on a broader acceptance of principle. The implementation of a system from one domain of e-Health may be more likely to succeed if those from other domains were already established, or were implemented simultaneously. As Jennett et al (2004) concluded in relation to tele-Health:
“Tele-Health applications are more likely to be successful, in terms of cost and sustainability, if they are considered to be part of the larger domain of e-Health. Programs implemented and evaluated as independent initiatives are at greater risk of failure.”
While prior conditions in an organisation clearly seem to affect the likelihood of e-Health success, such conditions could not be separated from the broader socio-political climate. There may be significant differences in professional cultures between societies (Uhlenhopp et al. 1998) and the infrastructure of health services and indeed telecommunications infrastructure varies significantly between places (Anonymous 1999). If there were shortcomings in the funding and infrastructure of a health service, an intervention was more likely to fail (Rudowski 2003), whereas if the health service is well funded and supported, new interventions were easier to implement (Elford 1997, Guler and Ubeyli 2002, Weinstein et al. 2001).
Cost
The cost of a system as a factor in implementation was mentioned frequently in the literature across all four domains (Mairinger 2000, Wallace et al.1998, Elford 1997, Falas et al. 2003, Whitten and Love 2005, Al –Qirim 2005, Goldberg and Dwyer 1995), though it was not described in any depth. It was suggested that the more effective the technology, the more costly it may be in financial terms (Angood 2001). In terms of telecommunications, there was also the cost of regular usage to consider as well as that of initial implementation, since network tariffs may apply (Mun et al 1995); in terms of other systems there would also be costs associated with procurement and maintenance.
The need for and impact of validation and evaluation
Validation and evaluation of e-Health systems were important factors in their widespread implementation. Again, this was the case with all four domains of e-Health, albeit less so with information systems, perhaps because the world wide web is a completely normalized tool.
Without strong data demonstrating that a system works, improves standards of care, can be used efficiently and easily, and is cost-effective to implement, it is unlikely to win the confidence of policy makers and users. Indeed, lack of validation and evaluation was frequently presented as a barrier to system implementation (Lehmann 2004, Anderson and Aydin 1997, Yellowlees 1997, Jadad 2002, Swinglehurst 2005), while continued monitoring and evaluation was a facilitator (Elford 1997, Cook and Dupras 2004, Styra 2004, Jerant 1999).
The need for validation and research into e-Health systems was inextricably bound with the effectiveness of the technology and its impact on practice. Vreeman et al (2006) claimed that:
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 24
‘Further research is needed to better characterize the effect EHR (Electronic Health Records) can have on the process and outcomes of care and to elucidate what “active ingredients” are necessary to achieve these benefits.’
Related to the above was the issue of whether validation and evaluation already available to implementers and users served as barriers or facilitators. This would depend heavily on the content of the evaluation – a positive evaluation would support the enthusiasm of those seeking to implement it elsewhere. Yellowlees (2005) explained that when telemedicine system fails in a site:
‘…the whole practice of telemedicine itself attracts an unenviable reputation, and it may be years before another attempt is made to provide telemedicine services in a region where telemedicine has failed once already.’
There was also an aspect to evaluation and validation that was cyclical: good results encouraged further implementation, implementation led to further research. Because e-Health systems, particularly management systems, could retain clinical data in an easily accessible way, with their implementation came the ability to validate them quickly and efficiently (Lehmann 2004, Anderson 2000).
Professional attitudes
Accounts of professional attitudes to the implementation and integration of e-Health systems suggested a broad array of barriers and facilitators to success. These attitudes related to perceptions of the changing relationship between professionals and patients (Wallace et al. 1998, Mandl et al 1998) and perceptions of the effectiveness of the technology. There seemed to be a general anxiety towards the use of technology (Childs et al 2005), or clinicians felt that their roles were undermined by it (Lehmann 2004), with the introduction of new systems implying shortcomings in their own performance (Whitten and Love 2005). The latter related to how the e-Health system affected the allocation of tasks and how an organisational structure changed once technology was adopted. It could also relate to unease regarding patient safety, particularly in terms of how professionals could manage a critical situation at a distance.
Anderson (2000) summarised the impact implementation may have on professional attitudes:
‘Computer-based information systems change traditional practice patterns. They structure how clinical data are recorded, organized and displayed. Their implementation frequently affects professional relations between individual caregivers and professional groups within the organization. The responses of physicians and their health care professionals are shaped by their perceptions of how the system affects their work relations and accomplishment of their professional goal.’
Aside from concerns about particular aspects of the systems, there was also the issue of “ownership” in terms of professional willingness to use, manage, and be associated with them (Jennett et al 1996, Elford 1997). A related facilitator here was leadership (Anderson and Aydin). If authority and leadership were clearly designated (someone is willing to take responsibility), it was more likely that professionals would broadly accept an e-Health system. The fundamental precondition to confidence in the system was acceptance in the first instance. Maglogiannis (2004) claimed that acceptance by doctors is the most important factor in the successful implementation of telemedicine systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 25
Ease of use of the system
The ease of use of an e-Health system, the interaction between the technology and its users, was self-evidently a factor in successful implementation (Goldberg and Dwyer 1995). The system needed to be appropriate to the work, not imposed for its own sake (Lehmann 2004, Mairinger 2000, Cook et al 2004, Mandl et al 1998, Sable 2001), be compatible with existing systems (Hussein et al 2004), and reliable (Stanberry 2000).
A system’s ease of use was closely related to its original design. Angood (2001) explained how the design of a system must reflect clinical need:
‘Over the past two decades, medical informatics has provided valuable lessons by showing that attempts to use information technology will often fail when the underlying motivation is the application of technology for its own sake rather than satisfying the need for providing solutions to clinical problems or an improved efficiency of patient care.’
E-Health systems were implemented because they were deemed to be impressive technologically rather than beneficial to clinicians and patients. Hence, a well-designed system was not necessarily a clinically effective one. Good design in itself was not a sufficient facilitator of successful implementation; rather, that a system’s design was compatible with existing work patterns was the integral factor. Safran and Goldberg (2000) raised the following issue with some electronic medical records systems:
‘Some EPRs (electronic patient records) have limited remote access capabilities or assume that no two clinicians would need to look at the same patient information at the same time.’
This was an issue of the system’s workability. If only one clinician could
view a record at any given time, it became more difficult to share
information instantaneously. Related to this were factors involving time and
reimbursement. The implementation of an e-Health system would not
necessarily reduce clinicians’ workload (Jaatinen et al 2002). If workload
increased clinicians would not always be reimbursed for the extra time taken
(Mairinger 2000, Rudowski 2003). One study (Shiffman et al 1999) also
explained that using the system was regarded as a tedious and unrewarding
process by some professionals. However, if an e-Health system replaces
face-to-face consultation, clinicians travel time would reduce (Whitten and
Love 2005).
Related to the general ease of use of a system was its ability to order and manage information. If the technology disrupted established working practices, and increased the time and effort required to record, communicate, and find information (and indeed arrive at a clinical decision), successful implementation was unlikely (De Lusignan 2005). A management or information system could contain insufficient, misleading, or too much information (Maulden 2003, De Lusignan 2005, Tierney 2001). However, if a system was flexible and adaptable to existing working practices, and efficient in ordering existing information in a way that was easily accessible, it was more likely to succeed (Yellowlees 2005).
Security, confidentiality, and standards
Security, confidentiality, legality and standards were almost exclusively issues of accountability and safety. For successful implementation, a system needed to be not only workable, but also secure. It needed to be safe for
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 26
both patients and professionals; for patients, in terms of the security of their personal data, and for professionals in terms of not compromising their standards of practice. Falas et al (2003), in a review of decision support systems, stated:
‘Ethical issues regarding the privacy of the patient’s data when transmitted over a telecommunications network, the risks involved when the patient is treated remotely, and the patient’s readiness to accept telecare devices in the home are very important.’
The implications of e-Health systems on patient privacy and confidentiality were cited frequently (Link et al 2001, Mairinger 2000, Jennet et al 1996). The extent to which these issues affect the success of an e-Health system depended on the security of, and access to, the technology. This was the case in all four domains of e-Health where patient data was concerned. Communicating patient data in any way involves some risk. However, when the means of communication were electronic, the risks of compromising a patient’s privacy, while no more or less severe, could be different to those of verbal and written communication.
There were several aspects of e-Health systems that could induce anxiety in health professionals. One related to the expectations of the patient, and to patient and professional autonomy. Lehmann (2004) gave an example relating to decision support technologies used in diabetes care:
‘…standalone use of a program by patients- without reference to a health carer- is the sort of thing that gives health-care professionals cause for concern, because of issues over patient safety.’
The perceived risk was that should a professional recommend a patient to use a computerised decision support programme, and the programme made an error which ultimately compromised the safety and well-being of the patient, it was the professional who remained accountable. In short, Lehman’s implication was that programmes that appear to increase patient autonomy could potentially decrease that of the professional, leading to anxiety over liability.
Since technology in health could detract from professional control in the care of the patient and so increase anxieties over liability, clear standards for use were essential. Indeed, the need for clear standards of practice regarding the use of all types of e-Health system was mentioned frequently (Jennet at al 2004, Rudowski 2003, Hussein et al 2004, Grams and Moyer 1997, Anderson 2000, Tierney 2001). The implication was that the use of such systems should be engineered to be within existing practice standards. However, because these technologies were relatively new, the standards and guidelines required to manage them had the potential to evolve through trial and error. Standardisation was hence a double-edged sword: while it provided clarity and safety to the users, it could stifle the technology’s potential. Hence, in a climate of defensive practice, if a programme was compatible with existing practice standards, its implementation was more likely to be successful.
Education and training
It was perhaps not surprising that education and training featured frequently in the literature (Uhlenhopp et al 1998, Carrino et al 1998). The success or failure of a new system depends on the capabilities of those using it and new technologies had to be learnt in the first instance (Pellegrino and Kobb 2005). Lack of skills and knowledge was frequently cited as a barrier to success (Childs et al 2005). Education could be as
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 27
simple as the communication of information between professionals and organisations for the purpose of influencing system implementation and usage (Mairinger 2000) to more formalised practices, such as officially sanctioned system-specific training courses (Anderson and Aydin 1997). Weinstein et al (2001) explained how education could help practitioners not only learn the system, but use it to its optimal effect:
‘Educating pathologists about certain limitations of telepathology will increase their awareness of the small but manageable list of problem diagnoses and provide them with strategies to circumvent such problems.’
Education was hence not simply about what a system can do, but also what it cannot. This in turn emphasised to the professionals what specific skills were required of them; the limitations of a programme would highlight their own capabilities as professionals. Education and training were hence not restricted merely to the learning of IT skills; rather, they could instigate improvements in clinical practice itself related to the use of e-Health systems.
3.4.1 Limitations of scoping exercise
The scoping exercise was not intended to be a comprehensive systematic review, and is thus likely to have missed some relevant publications. Inclusion decisions, and data extractions were conducted by only one reviewer, to ensure completion of the review in time to inform WPs 2, 3 and 4: this is likely to have produced some inconsistencies and inaccuracies. However, we are confident that the scoping exercise was sufficiently robust to meet its aims and objectives and in any case the rigorous systematic review described in Sections 3.5 – 3.7 provided the opportunity to examine the robustness of these initial findings.
3.4.2 Scoping exercise conclusions
The literature reviewed here presented an array of barriers and facilitators to the implementation of electronic health systems. Though there were many different types of system and different scenarios recorded, certain commonalities remained. Whether the factors identified here mirrored the real-life experience of those working on the ground remained to be seen. However, in the thematic analyse of current literature identified within this scoping exercise, it was apparent that, in order to increase the likelihood of an implementation being successful, certain dynamics needed to be considered. Good conditions prior to implementation, manageable cost, awareness of the most appropriate systems, professional willingness to adapt, an easy to use and secure system and adequate education and training were all significant factors in both the initial implementation and subsequent sustainability of an effective e-Health intervention.
A common factor for all factors identified in the scoping review was the relationship between the e-Health systems themselves and the organizational contexts in which they were applied. Hence, the factors identified as either impediments or contributors to successful implementation hint towards a holistic approach. E-Health systems, if adopted by an array of collaborative agencies, facilitated both the successful implementation of the system and the efficacy of inter-agency co-operation. The practice of one agency must be receptive to that of another. Hence, the overall receptiveness of an organisation to the implementation of an e-Health system may be determined by an array of factors, including those beyond the context of the inner workings of that organisation itself.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 28
3.5 Stage 2 – Aims/Objectives
Stage 2 consisted of a formal systematic review of reviews of implementation of e-Health interventions into health care systems conducted by the co-applicants.
The aim of this review of systematic reviews was to identify and synthesise available reviews of implementation of e-Health interventions into health care systems.
Objectives
� To identify published reviews of e-Health implementation
� To synthesise the findings of identified reviews: 1) via an analysis of the papers which using the key themes identified in stage 1 as a coding framework; and 2) using Normalisation Process Model (NPM) as a direct coding framework.
� To compare and contrast the results of these two methods of data analysis.
� To summarise the available evidence, highlighting the key findings, with a view to both informing current and future e-Health implementation programmes and identifying outstanding research questions.
3.6 Methods
Standard, Cochrane approved systematic review methodology was used.
3.6.1 Criteria for considering studies for this review
This review focused on reviews of e-Health implementation. For the purposes of this study we defined these three components as follows:
Definition of a review
‘A review paper provides an analytic account of the research literature related to a specific topic or closely related set of topics. It is intended to contribute to knowledge by answering a research question.’
Inclusions
1. Systematic review: where relevant literature has been identified by means of structured search of bibliographic and other databases; where transparent methodological criteria are used to exclude papers that do not meet an explicit methodological benchmark, and which presents rigorous conclusions about outcomes.
2. Narrative review: where relevant literature has been purposively sampled from a field of research; where theoretical or topical criteria
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 29
are used to include papers on the grounds of type, relevance, and perceived significance; with the aim of summarising, discussing, and critiquing conclusions.
3. Qualitative metasyntheses or meta-ethnographies, where relevant literature has been identified by means of a structured search of bibliographic and other databases, where transparent methods had been used to draw together theoretical products, with the aim of elaborating and extending theory.
Exclusions
1. Secondary analyses (including qualitative metasyntheses or meta-ethnographies) of existing data-sets for the purposes of presenting cumulative outcomes from personal research programmes.
2. Secondary analyses (including qualitative metasyntheses or meta-ethnographies) of existing data-sets for the purposes of presenting integrative outcomes from different research programmes.
3. Discussions of literature included in contributions to theory-building or critique.
4. Summaries of literature for the purposes of information or commentary.
5. Editorial discussions that argue the case for a field of research or a course of action.
Where the abstract stated it was a review, but there was no supporting evidence in the main paper, such as details of databases searched or criteria for selection of papers (either on methodological or theoretical grounds), the paper was excluded.
Definition of e-Health
We used the definition of e-Health outlined in the introduction (Section 1.3):
‘e-Health is the use of emerging information and communications technology, especially the Internet, to improve or enable health and healthcare’
We classified e-Health interventions into 4 domains: management systems; communication systems; computerised decision support systems; and information resources as described in Box 1 of Section 1.3 and as used within the initial scoping exercise.
Definition of implementation
We used the definition of implementation provided by Linton (2002):
‘Implementation involves all activities that occur between making an adoption commitment and the time that an innovation either becomes part of the organizational routine, ceases to be new, or is abandoned (...) [and the] behavior of organizational members over time evolves from avoidance or non-use, through unenthusiastic or compliant use, to skilled or consistent use. (p 65)’
3.6.2 Search strategy for identification of studies
We searched the following electronic bibliographic databases:
� MEDLINE
� EMBASE
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 30
� CINAHL
� PSYCINFO
� The Cochrane Library, which included Cochrane Database of Systematic reviews, Cochrane Central Register of Controlled Trials, DARE, NHSEED (NHS Economic Evaluation Database), Health Technology Assessment Database.
The search was supplemented with papers known to any of the co-authors, and with papers identified in the earlier scoping review, undertaken by our researcher (GA). Our definitive search used thesaurus terms, while the scoping review had used similar search terms, but as text words. By combining these two modes of searching we aimed to identify the largest possible number of papers.
The search strategy for MEDLINE used the three concepts: e-Health; implementation; and review.
Thesaurus terms which referred to e-Health interventions were:
� Medical-Informatics-Applications
� Management-Information-Systems
� Decision-Making-Computer-Assisted
� Diagnosis-Computer-Assisted
� Therapy-Computer-Assisted
� Medical-Records-Systems-Computerized
� Medical-Order-Entry-Systems
� Electronic-Mail
� Videoconferencing
� Telemedicine
� Computer-Communication-Networks
� Internet
Where appropriate, thesaurus terms were exploded to include all terms below the searched term in the thesaurus tree. The lowest term was always exploded.
There are no thesaurus terms for implementation, so this concept was searched for by looking for these text words in title, keywords or abstract:
� Routin*
� Normali?*
� Integrat*
� Facilitate*
� Barrier*
� Implement*
� Adopt*
These two concepts were combined, and then the search was limited by publication type = review or meta-analysis.
Initial searching of the MEDLINE database was done setting date limits from 1990 – 2007, and no limitation of language. None of the non-English language citations were relevant, and there were no relevant publications published prior to 1995. Hence the MEDLINE search was re-run with date limits from 1.1.1995 – 31.12.2007, limited to publications in English, and these limits were used for searching all other databases.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 31
The search was translated for the other databases, using each database’s thesaurus terms.
3.6.3 Identification of papers for inclusion
All citations were downloaded into a database in Reference Manager 11, and screened by two independent reviewers. EM screened all citations; CM and TF double screened 50% each. If either reviewer could not confidently exclude the paper based on the abstract or citation, the full paper was obtained. Papers that could not be definitely excluded at this point were obtained for further scrutiny. All papers obtained were double screened. CO’D screened all papers; EM, CM, TF, and FM double screened 25% each. If there was a disagreement about whether a paper should be included or excluded, it was read by all reviewers, and agreement reached through discussion (see Appendix 3 for summary of exclusions and inclusions at this stage).
Data extraction, analysis and synthesis
The aim of the analysis and synthesis was to determine where there is adequate current knowledge of particular processes, and where there are gaps in the current research literature, with a view to informing future research commissioning.
Data extraction consisted of two stages. First we used a simple but standardised data extraction instrument to categorise papers on the basis of: country of origin; e-Health domain; publisher and date of publication; review aims and methods; databases searched within the review; inclusion and exclusion criteria of review; and number of papers included in the review (See Appendix 4). EM, CM, TF, CO and FM all contributed to this exercise.
Secondly, for every paper a judgement was made (FM, CO, TF, EM, CM) as to whether material relevant to the four constructs of the NPM was present or absent. As this was a qualitative analysis, no attempt was made to quantify the weight put on any one NPM construct in a given review; the analysis was limited to whether material relevant to the construct was present or absent.
Having determined which constructs were represented in each paper, we then extracted all the process oriented key words or phrases in each paper, and code them to the relevant construct of the NPM. Two independent reviewers coded according to the constructs of the NPM. If a key word or phrase could not be coded to the NPM this was stated, as a way of identifying gaps in the model. FM; EM, TF, and KO’D each extracted data from 25% of papers. FM then rechecked all data extraction to ensure every action-oriented statement was identified and coded accurately according to the constructs of the NPM (May 2006). Data, which were statements treated as attributions, were thus extracted from all included papers, and these attributions were then recoded, independently by May.
Dual coding in this way permitted economy of effort and enabled differences in coding and interpretation to be identified and discussed. Any areas of disagreement, which were minimal, were resolved through discussion. If any areas of disagreement remained then a final reviewer (TF or EM) served as arbiter.
At the same time FM conducted a separate thematic analysis of the identified literature, examining to what extent emergent themes matched those identified through the scoping exercise. This then presented an
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 32
opportunity to explore the effects of different approaches to coding, that is, what differences, if any, were noted through using the NPM as a coding framework versus using results from an emergent thematic analysis, on our final conclusions regarding the key findings from the review.
3.7 Stage 2 - Results
3.7.1 Papers identified
Between them the searches yielded 6,585 unique citations (Figure 1). Of these, 6,439 could definitely be excluded on the basis of the title or abstract, leaving 146 citations where the full paper was needed before a decision could be made. Of the 146 full papers, 19 met the criteria for inclusion (Appendix 4).
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 33
Figure 1. WP1 Quorum Flow Chart
Reasons for exclusion at full paper
stage:
Not a review: n = 105
Not on e-health: n = 1
Not on implementation: n = 20
Only published as an abstract: n = 1
(Where more than 1 reason applied,
only the first was counted).
Medline
3,123
Embase
3,373
CINAHL
40
Cochrane
9
PsycINFO
40
References in database
6585
Order
146
Exclude
6,439
Include
19
Exclude
127
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 34
Quality of included reviews
The methodological quality of many of the included reviews was very poor. Some reviews did not even mention which databases were searched and search strategies were often rudimentary, and many reviews had not specified their inclusion and exclusion criteria (Appendix 4). Furthermore, for eight (42%) of the nineteen papers reviewed it was impossible to ascertain how many papers had been included as this information was not explicitly provided.
Description of included reviews
Of the 19 included reviews, 13 originated from the USA, 4 from Canada, and 1 each from the Netherlands and Malaysia. Reviews tended to cover one or more e-Health domain, with 8 reporting on management systems, 7 on communication systems, 5 on decision support, 1 on information systems.
Relationship with the Normalisation Process Model
Each statement from a paper relating to findings regarding barriers or facilitators to e-Health implementation was treated as an “attributive statement.” By this method 411 attributive statements were coded directly against the NPM. All the included papers contained data which could be coded according to the NPM. They also all included data which could not be coded to the NPM, such as that about technical specifications, beliefs, attitudes or the need for evaluation.
Table 1 shows the constructs of the NPM that were addressed within each review. It aggregates the number of attributions included for each construct within each paper. If the exact same attribution was made more than once for any given paper it was counted as a single attribution. It can be seen that the majority of the reviews contained a great deal of information that related to contextual integration issues (142/411), namely the degree to which the e-Health innovation related to organisational structure and goals. Rather less attention was paid to the impact of the interventions on health professionals’ interactions with patients (interactional workability)(55/411) and inter-professional relations (relational integration)(72/411). Skill set workability was largely considered as a training issue, rather than as a legitimate concern, with 42/142 comments falling into this category. These findings are illustrated clearly in Figure 2. It should be noted that Table 1 and Figure 2 are not meant to have statistical properties but rather serve to illustrate the distribution of comments relating to barriers and facilitators across the NPM. We have found it a valuable means of presenting this content analysis in an effort to determine whether coding to the NPM helps identify any gaps or issues given particular emphasis, by the published literature.
The following section illustrates how findings related to the NPM.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 35
35
Table 1. Number of Attributive Statements Within Each Category of NPM
Interactional Workability Relational Integration Skill Set Workability Contextual Integration Outside NPM
1. Broens 2007 3 4 5 6 7
2 Chaudrhy 2006 0 1 1 18 1
3 Hebert 2006. 0 1 1 3 5
4 Hilty 2002 4 4 5 6 4
5 Jennett 2004 0 3 1 8 13
6 Jennett 2005 1 2 0 2 2
7 Johnson 2001 0 4 3 5 5
8 Kawamoto 2005 7 1 0 1 2
9 Kukafka 2003 1 3 3 6 2
10 Leatt 1 4 4 6 5
11 Lu 2005 10 12 3 9 6
12 Ohinmaa 1 3 1 8 7
13 Papshev 2001 2 3 1 12 6
14 Peleg 11 7 2 13 16
15 Shekelle 2006 2 2 1 4 3
16 Studer 2005 1 3 1 12 1
17 Vreeman 2006 2 7 4 11 4
18 Yarborough 3 6 3 7 7
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 36
36
19 Yusuf 6 2 3 5 4
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 37
Figure 2. Radar Plot of Distribution of Attributions within the NPM
Interactional Workability
Interactional workability issues accounted for less than 14% of the coded data relating to barriers and facilitators of e-Health. Most of the focus in the literature in relation to this construct concerned issues regarding “ease of use”, which could be presented as either a barrier or facilitator of implementation as the following examples demonstrate.
‘a common theme among all four features is that they make it easier for clinicians to use a decision support system….as a general principle then our findings suggest that an effective clinical decision support system must minimise the effort required by clinicians to receive and act on system recommendations.’(Kawamoto 2005)
‘the literature suggests that utilisation of PDAs saves clinicians time in regard to accessing, retrieving, and recording data’ (Lu et al .2005)
‘physician and organizational resistance due to the perceived negative impact on the physician’s workflow.’ (Shekelle 2006)”
Although ease of use was the main aspect of interactional workability identified, there were a few comments on other aspects, such as perceived negative effects on health professional – patient interaction:
‘…the unique relationship between a physician and his or her patient is different from that found in other industries. Many physicians value this interaction and are hesitant to give it up or take time away from the experience due to new information technologies.’ (Yarborough 2007).
Distribution of Attributions
050
100150
IW
RI
SW
CI
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 38
Relational Integration
Security and safety issues were a key issue within this construct. Such concerns could act both as facilitators or barriers. On the one hand people may expect e-Health technologies to reduce errors, which would encourage uptake:
‘Greater efficiency and increased error reduction will be achieved if data capture or data entry are permitted wherever and whenever it is available’ (Lu et al. 2005)
On the other hand security and safety concerns could serve as a barrier to widespread utilisation:
‘EMR system downtime: exposed physicians and practices to risk of patient injury and liability’ (Studer 2005).
‘concerns regarding the privacy and confidentiality of patient information stored in an EMR’ (Studer 2005).
Skill Set Workability
Skill set workability was concerned mainly with the need for adequate training for personnel who were going to be involved in any e-Health implementation, although issues relating to division of labour, that is, who does what when were also mentioned:
‘potential affect on the division of work among care providers’ (Peleg 2006)
‘A need for training users how to use these novel types of systems. Such training is needed at all levels’ (Broens 2007)
Contextual Integration
Contextual integration refers to infrastructure and organisational issues. The literature emphasises the need for adequate resources, particularly finance. Issues such as administrative supportive, policy support, standards and interoperability also fell within this construct.
‘factors within the organisational infrastructure required to enable usage eg available resources, supportive policies and accessibility to the IT system.’ (Kukafka 2003)
‘Barriers to implementation include funding’ (Jennett 2005)
‘Many programmes fail because of inadequate financial and other administrative support from the leadership, or the fact that telemedicine is not a match for the overall mission of the organisation’ (Hilty 2002).
‘challenges for adoption of electronic health records…..included cost, technical issues, system interoperability.’ (Shekelle 2006).
OUTSIDE THE NPM
Many issues, 24% (100/411) fell outside the model. These mostly related to technical issues, attitudes, and issues of engagement and evaluation. Sometimes items were unable to be coded to the NPM because they were very generic and non specific, making it unclear as to the “work” that was being done. Examples of items which did not fit the model were:
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 39
‘the importance of evaluating CDS’
(Peleg 2006)
‘Ownership, attitudes of participants, professional connections remote location’ (Ohinmaa 2006)
‘recommendations: include end users, especially clinicians into in the system development activities.’ (Vreeman et al. 2006).
It is important to note that the majority of data did fall within the constructs of the NPM and using this model helped to improve understanding of the processes at play and to identify strengths and gaps in the literature.
3.7.3 Thematic Coding of Systematic Review.
As well as coding the systematic review using the NPM as a coding frame we also continued to develop our thematic coding, that began during the initial scoping exercise. Most of the data in the systematic review mapped onto these themes, and the table of all coded data is shown in Appendix 5. It is worth noting that some additional codes were added which had not been in the original coding framework - technological issues, communication issues, organisational issues and ‘other’.
Issues that fell into the themes that matched the initial scoping exercise codes were similar and are not repeated here (although shown in Appendix 5) but examples of items that fell within the new themes are illustrated in the following section. Table 2 shows which themes were present within each paper.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 40
Table 2. Thematic analysis codes (full list of papers with data mapped onto each theme included in Appendix 5)
Conditions Prior To implementation
Cost
Need and importance of validation and
Attitudes
Ease of Use of System
Security,
Confideniality
Education and Training
Technological Issues
Communication Issues
Organisational Issues
Other
1. Broens 2007 � � � � � � � � �
2 Chaudrhy 2006 � � � �
3 Hebert 2006. � � � � � �
4 Hilty 2002 � � � � � � � � � �
5 Jennett 2004 � � � � � � � � �
6 Jennett 2005 � � � � � � � � �
7 Johnson 2001 � � � � � � �
8 Kawamoto 2005 � � � � � �
9 Kukafka 2003 � � � � � �
10 Leatt 2006 � � � � � � � � � �
11 Lu 2005 � � � � � � � � �
12 Ohinmaa 2006 � � � � � � � � � �
13 Papshev 2001 � � � � � � � � � �
14 Peleg 2006 � � � � � � � � �
15 Shekelle 2006 � � � � � � � �
16 Studer 2005 � � � � � � � � � �
17 Vreeman 2006 � � � � � � � � �
18 Yarborough 2007 � � � � � � � � � �
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 41
19 Yusuf 2007 �
�
�
�
�
�
�
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 42
Technological issues
These ranged from system design issues to generic comments on product quality:
‘Technical issues under-reported including bandwidth, audio quality, FPS, size of transmitted video image speed of computed and name and make of CODEC and other equipment’ (Hilty 2002)
‘inefficient technology pg 5 discussion’ (Jennett 2005)
‘systems that prompted clinicians to record a reason when not following the advised course of action were significantly more likely to succeed than those that allowed the system to be bypassed without recording a reason to’
(Johnson 2001)
‘Products and technology’ (Papshev 2001).
It is therefore clear, that while technology proponents may frequently suggest that “the technology is not the problem”, the literature suggests that technical issues continue to be cited as either an important facilitator or barrier to implementation.
3.7.3.2 Communication
Communication issues related to the e-Health implementation itself and to communication using the system – such as during professional - patient interactions. Sometimes these were quite generic statements whose meaning was unclear.
‘it is also important to maintain bidirectional communication throughout the process to enable staff to see current measures indicating cost savings and error reductions, as well as to ensure that staff are providing input about the system’s usefulness and potential modifications’
(Leatt 2006)
‘Physicians’ concerns that system would interfere with or negatively impact the physician-patient encounter’ (Leatt 2006)
‘Benefits in communication’(Lu et al. 2005)
‘Communication between clinicians has also improved’ (Ohinmaa 2006).
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 43
Organisational
Organisational issues included, for example, the extent to which e-Health services matched organisational priorities, their effects on workflow, and how they might benefit organisations.
‘Alignment of decision support objectives with organisational priorities and financial interests’(Kawamoto 2005)
‘a shift toward specialised and focused system, interacting systems that are integrated into the clinical environment and workflow’(Peleg 2006)
‘the importance of considering work flow design as part of the EMR implementation process’ (Studer 2005)
‘Improved reporting capabilities …cited the capability for more comprehensive reporting that integrated clinical and administrative data as a key benefit. …Helped clinicians and administrators to be aware of the current departmental workload which helped provide the rationale for how pts and therapists were scheduled.’ (Vreeman et al. 2006)
3.7.3.4 Other
There were only two statements that fell out with our thematic coding and these were:
‘Context: stability of client’s condition and nurses ability to co-ordinate more than one visit’ (Hebert et al. 2006)
‘phase 5: system use inducing strategies focuses on developing and implementing approaches that are proactive and specifically targeted to influencing favourably the predisposing, enabling and reinforcing factors identified in phase 4.’ (Kukafka 2003).
3.8 Discussion and conclusions
A thorough and systematic search of the literature identified 19 reviews which addressed the implementation of e-Health technologies into health care settings. The majority of these reviews originated from North America (the USA and Canada). With a few exceptions, they were of poor quality, and their findings must be treated with caution. It is worth noting that very few reviews focused on information systems, e.g. use of the internet, which is surprising in view of the growing importance and potential of this area of e-Health.
Recommendations
1 Findings from existing systematic reviews about e-Health implementation should be viewed with caution as they are misleading.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 44
2 Any future e-Health implementation reviews must address each of the 4 domains of e-Health.
It appears that the fundamental assumptions underpinning research in this area are unsatisfactory in that there is undue focus on organisational issues and insufficient attention being paid to problems of workability, that is, the work that health professionals, as individuals or teams, must undertake to make these systems function satisfactorily in practice. However, despite caveats about the lack of robustness of the findings of the reviews and the undue focus on organisational issues included in our review of reviews, certain conclusions can still be drawn concerning the identified barriers and facilitators to e-Health implementation.
First, the Normalisation Process Model (NPM) was shown to be a valuable analytical tool, which provided a good theoretical framework for synthesising the available data and proved more informative than an atheoretical thematic analysis would have done in highlighting the gaps in the literature and illustrating areas where extra work is necessary. If one were to draw only upon the thematic analysis one might gain the impression that the literature was fairly broad ranging in its coverage with few gaps. However, the use of the NPM reveals that the literature emphasises issues of CI, with an emphasis on the execution and realization of systems at the expense of understanding the work of embedding them in practice. Far less data was present on the impact of e-Health initiatives on interactions with patients (interactional workability issues), inter-professional relationships (relational integration issues, and fit with existing staff skills and roles (skill set workability). The published literature therefore provides an unbalanced account of barriers and facilitators to e-Health implementation, neglecting the wider social framework that must be considered when introducing new technologies. Thus, if e-Health implementers are being guided simply by the existing literature it helps explain why e-Health is not becoming normalised. Use of the NPM helps to highlight where the gaps in the existing literature are located and subsequently informs the development of clear recommendations as to how to address these problems.
Coding to the NPM also shows gaps in the model itself. Material that could not be coded within the model included technological issues, issues relating to professional beliefs and attitudes (the focus of other theoretical frameworks), and importantly issues relating to enrolment and engagement of health professionals and also to the ways individuals and groups appraise e-Health services as being beneficial or not. This has contributed, along with other related studies, to further development of the model as will be addressed in Section 7 taking analysis to a higher level.
3. The research team believes there is little to be gained at present from commissioning a further systematic review in this field.
Although the literature has methodological limitations, the findings resonate with two other reviews recently conducted by the co-applicants. Based on this fact, and our knowledge of the primary literature, we believe it is unlikely that a further review, even if robust, would add to our knowledge in this area. We would therefore not recommend this as a good use of scare research funds. However, we do feel there is a requirement for further primary e-Health implementation research of the kind suggested below.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 45
Recommendations
4. Primary researchers of e-Health implementation need to broaden the scope of their research so that they give greater consideration to the impact of e-Health technologies on:
� interactional workability, that is, the effects of e-Health services on health professional – patient encounters;
� relational integration issues such as confidence in the safety of a system and the effects on efficiency;
� skill set workability issues which include not simply training and support issues but also examination of how new technologies affect roles and responsibilities;
� enrolment and engagement with participants;
� how new practices are evaluated and judged by participants.
Thus, greater attention to a broad range of socio-technical issues is merited. Researchers need to explore all these issues in greater depth. It was clear from the thematic analysis that some issues such as cost and training were addressed in quite a superficial and simplistic way, whereas much deeper examination of these issues is merited.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 46
4 Work-package Two (WP2) Barriers and Facilitators to the Implementation of E-Health Services: The Perspective of Health Professionals.
4.1 Background
A key challenge for those wishing to encourage the use of e-Health services is how to move to mainstream service delivery. Understanding barriers to the implementation of e-Health services is crucial if this is to be achieved. Professional resistance is frequently cited as one of the most significant of these barriers. However, earlier suggests that implementation remains a problem even when health care professionals support the use of new technologies (May et al 2003a, Lapointe and Rivard 2006). It was not our intention to replicate this work in this work-package; instead our purpose was to examine health care professionals’ views about:
1. Factors that affect implementation and integration of e-Health services;
2. How to facilitate integration of e-Health service in different health service contexts
3. The effects, if any, of e-Health services on health care professional/patient interactions and clinical activities; and
4. Training and educational requirements.
4.2 Aims/Objectives
The aim of the work-package was to identify informants’ views of implementation and integration of e-Health services across the four e-Health domains outlined in Section 1.3, Box 1.
Objectives
The objectives of work-package 2 were to:
1. Increase our understanding of barriers to and facilitators of e-Health implementation.
2. Highlight training/educational requirements.
3. Help refine our conceptual model of e-Health normalization and thereby inform other work-packages.
4. Inform the development of recommendations regarding implementation and integration of e-Health services.
5. Compare the findings from a thematic analysis and an NPM-based analysis in order to determine, whether there were particular advantages or disadvantages to either approach, and which method would prove most helpful in conceptualising the gaps or strengths of the data.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 47
4.3 Methods
4.3.1 Design
Work-package 2 used two complementary methods of qualitative data collection - task groups and semi-structured telephone interviews.
4.3.2 Task Groups
Task groups are a hybrid focus group intended to generate both conventional qualitative data and recommendations for action grounded in the experience of group members. They were used not only to obtain the views of participants on the issues affecting e-Health initiatives in their area, but also to identify their views on the issues that will affect the implementation of e-Health projects.
To obtain a wide range of experience we sought members from health professions such as primary and secondary care physicians, nurses and other allied health professionals from three contrasting NHS contexts. We held five task groups, each preceded by informal discussion and the distribution of information to participants, in particular the results of the initial scoping exercise within WP1.
4.3.3 Setting and sample
Identifying case study sites and participants
We aimed to identify three case study contexts which differed in respect of:
� Geographical location/coverage
� Levels of utilisation and normalisation of e-Health services
� Domains of e-Health Used
� Different service and policy contexts
We believed these criteria would allow us to examine health professional perspectives on e-Health implementation from those located in deliberately different environments – and so contribute to the objectives described in Section 1.2.
Criteria for task group participants were that they should be:
� health professionals from a range of backgrounds, including physicians; nurses; professions allied to medicine and pharmacy.
� from different sectors, particularly primary and secondary care.
� people with real experience of using e-Health services.
� managers and informatics specialists involved in running and/or developing e-Health services.
We wanted to recruit participants from different disciplines to obtain a broader view of barriers and facilitators to e-Health implementation than a uni-disciplinary approach might permit. We also wanted the views of individuals with practical experience of e-Health as opposed to hypothetical beliefs. We thought that the task groups would benefit from the experience of members who were directly involved in overseeing service delivery.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 48
Having determined our selection criteria the project team selected sites that they believed would be suitable based on the criteria set out above. In order to protect participants’ anonymity, as promised in our consent forms, we are not providing identifiers of the study sites within this report. Suffice to say, that:
• � One site was an exemplar of an English NHS organisation with well integrated e-Health services mapping to the management and computerised clinical decision support system domains of our classification, involving both primary and secondary care sectors.
• � One site was a health board in Scotland which did not have an integrated electronic medical records system. However, it had several e-Health initiatives and services, ranging from specialist paediatric telemedicine services, to the use of electronic medical record systems in primary care (85% of Scottish GPs use the GPASS software). As elsewhere in Scotland the Emergency Care Summary project extracts recent medication and allergy information from GP electronic medical record systems and makes this available to out-of-hours clinical staff. It therefore used e-Health services that match the management and communication domains of our e-Health classification system.
• � The third site was an NHS call centre with services that included all four e-Health domains, as currently it uses an electronic medical records system, asynchronous electronic communication systems to relay patient information and triage decisions to integrated partner organisations, an electronic clinical decision-support system and has a web presence.
Task Group Recruitment.
Task group recruitment was facilitated by contacts established within each site, generally individuals well known within the organisation with respect to e-Health initiatives. Nevertheless, recruiting the task groups was a complex task. As well as initial ethical approval (attained from an MREC early in the project), we also needed project approval by the research governance bodies of the respective sites. At one study site we achieved this very quickly but for the other two this proved more challenging. In both cases staff turnover was a factor and our initial points of contact for governance left before the process was complete. Although we had received verbal assurances of approval from both contacts and had completed the relevant paperwork, we required an official written record of this approval in order to proceed both safely and accountably. After delays of several months we received written approval and honorary contracts for GA to conduct the task groups.
Potential participants, identified by our internal contacts, were sent a standard information and invitation letter via initial e-mail contact (see Appendix 6). Those who responded and expressed an interest were asked to suggest suitable dates and times. Once two or more participants offered compatible dates and times, these were offered to those remaining. One week before the task group, participants were sent a summary of the findings from the initial scoping literature review (Appendix 8). Since much of the task group was to be centred on the extent to which the findings of
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 49
the literature reflected the experience of professionals, it was important that they were given this initial frame of reference.
A factor in recruitment was igniting enough interest and willingness amongst the professionals within each site to participate in our project. Although we were given considerable help and support by our contacts, the response rate from potential group participants was mixed. At one site the response rate was initially very good. This may have been, at least in part, due to the fact that the organisation had a long-established electronic health system with which all professionals had some level of involvement. Hence, a broad array of professionals working at this study site would feel qualified to comment on their experiences. Despite a positive numerical response from these professionals, arranging a suitable time in which all who wished to could attend remained a challenge, as is often the case in any busy health service environment. However, the first group ran with the maximum number of 8 attending.
Arranging the second group at this site proved more challenging. Though we had had a very positive response rate for the first task group, our contacts had already utilised their influence and were limited as to how many new recruits they could provide. We contacted two professionals who had been unable to attend the first group. One of these agreed to attend if a group could be conducted at a suitable time. After further enquiries and advice from our contacts, we were able to attract the interest of three more. Despite further efforts we were unable to recruit more participants, so ran the second group with the willing four. Had we further delayed the running of this group, we may have found more willing participants, but it was equally likely that those we already had enrolled would lose interest.
Recruitment at the second site, where research governance approvals had been obtained with relative ease, was surprisingly difficult. Fifteen professionals were contacted directly, and others indirectly through our contacts. Three who initially expressed willingness subsequently ceased replying to correspondence, two declined to take part, and four did not respond to our initial (e-mail) approach. The remaining six agreed to take part, but could not all attend on the same date. They were subsequently split into two groups of three, though one who was to be a member of the second group was unable to attend on the day due to unforeseen professional commitments. He was subsequently interviewed as a key informant.
The nature of our NHS call centre study site meant that it was necessary to conduct the group at their working base during official working hours. The group consisted of four members of the night-shift at that particular time. Though eight were initially approached on our behalf by our recently established internal contact, this was done with the understanding that it was unlikely all would be able to attend, unless it happened to be an unusually quiet shift. GA hence travelled to the call centre early in the evening shift and four nurse advisors attended the task group.
Task groups were conducted between June 2007 and May 2008. The composition of each is shown in Table 3.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 50
Table 3. Composition of Task Groups
GPs Secondary Care Doctors
Nurses Pharmacy/Allied Health Professionals/Health professional advisors
IT or Clinical Managers/Technical Staff
Task Group 1
2 3 2
Task Group 2
2 1 1
Task Group 3
4
Task Group 4
1 1 1 (former nurse)
Task Group 5
1 1
4.3.4 Key Informant Interviews.
To develop and elaborate on conclusions from the task groups, additional telephone interviews (16) were held with key stakeholders involved in e-Health implementation throughout the UK. These were used to validate and elaborate on the key issues highlighted within the scoping exercise (WP1) and the recommendations and principles from the task groups.
Recruiting strategy.
As with the task group participants, interviewees were sent an initial invitation and information letter, (Appendix 7). The interview schedule was almost identical to that of the task groups (see below); however, there were key qualitative differences both in the running of the interviews and subsequent data analysis. As the interviewees were chosen because they were involved in e-Health implementation at a more senior level than the task group participants, and the content of the interviews reflected this, they were analysed separately. They were interviewed as implementers and experts rather than as frontline users. Potential interviewees were identified initially from the prior knowledge of members of the research team (mostly by FM and FS). They were also identified by the interviewees themselves. Hence, the list of those eligible grew with interview, though as with the task groups, many of those approached (via e-mail) did not respond. Of the thirty five people approached by us directly, sixteen were interviewed. Others either refused or did not respond, or expressed interest and willingness but did not reply to further correspondence.
Three of the interviews were conducted face-to-face, though most were conducted by telephone. As our research was concerned more with content than interaction, nothing significant was lost in telephonic interviewing. Interviews were conducted between August 2007 and May 2008 and covered representatives of a range of professional groups with different
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 51
backgrounds and experiences of e-Health, including senior representatives from nursing, primary and secondary care (including clinical directors/leads and national programme leads), academia, royal colleges, and informatics specialist groups.
Task groups and interview schedules.
As explained above, prior to the task groups and interviews being conducted, all potential participants were sent an invitation and information letter, explaining our project and why we thought their contributions would prove useful. A few days before a group was due to be held, those who had confirmed they would participate were sent a short summary of the key findings of the WP1 literature review (the seven key barriers and facilitators to e-Health implementation illustrated in Appendix 8). The rationale for this approach was to give the participants enough time to acquaint themselves with the ideas which would guide the group or interview, but not so far in advance that they would either forget the material or present the group with overly-prepared answers. The key barriers and facilitators were presented in text boxes placed at random levels on the page; this was so as to avoid giving the impression that we had imposed upon them a hierarchy of importance. Informed consent for participation and to tape-recording of the meeting was confirmed for all participants.
The task groups began by gathering information from each participant about their roles, their experience of using e-Health systems, and the systems they had used. The group then explored the main factors identified in the review, and how they may impede or facilitate the implementation of e-Health systems. The same pattern was employed in the key informant interviews; the interviewee would be asked to describe their background in e-Health and appraise the findings of the literature review.
A schedule was developed for the task groups and interviews to guide the group facilitator/interviewer (GA) and to ensure that all the important areas were covered (Appendices 9 & 10). Participants were asked to consider specific systems in which they had been involved, and which factors, if any, they believed the most relevant to their experience and the type of system in which they had been involved. They were then asked about each factor specifically; whether they had personal experience of it (and could they report real examples) and whether they agreed with the findings of the review that the factor was an important barrier/facilitator. Finally, they were asked if they felt any important factor was missing from our findings.
Task groups and interviews were conducted in a semi-structured manner, with the schedules acting as a guide for the researcher. This encouraged participants to talk at length. As well as considering whether the factors covered in the literature were an accurate reflection of their personal and professional experience, participants were also encouraged to describe their working practices. In short, we were concerned with ascertaining how, as well as whether the literature findings were relevant.
A further benefit of this semi-structured approach was to allow respondents to determine the direction of the interviews and task groups. They were able to make their own linkages between, and venture their own understandings
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 52
of, the findings rather than have these imposed by the facilitator/interviewer. We wanted them to order our findings, it was their decision as to which they wished to concentrate upon. It was also possible that the answers given by participants in response to initial prompting would cover areas that would have emerged later in the schedule; the semi-structures approach thus avoided repetition.
The task group and interview schedules and the initial task group and key informant interviews were made available for review by the whole research team at a joint data meeting. The approach to running the interviews and task groups were then amended slightly to ensure that we achieved the aims and objectives of this WP. We believe this approach has improved the quality of data collection and facilitated achievement of the WP aims/objectives.
4.4 Data Analysis.
We analysed the textual data from our task groups and interviews in an identical fashion. The method involved analysing both sets of data in two distinct ways, one an atheoretical thematic analysis and the second a framework analysis using the NPM as the coding frame. This dual approach to data analysis was undertaken because:
1) we wished to explore whether free coding would highlight issues that might be less obvious or missed entirely within the constraints of a predefined coding framework;
2) our experience with analysis of the data in WP1 had shown that atheoretical thematic coding could conceal gaps or strengths in data, whereas using the NPM as a coding frame might help us conceptualise strengths and weaknesses within the data more clearly. It would also help to clarify where issues resided within a broad implementation model and thereby provide valuable information that would inform the recommendations arising from our results.
1) An initial thematic analysis was undertaken, assigning general themes to each segment of text (Kvale 1996) until the point of saturation (no new themes were emerging). Here, it was important to acknowledge the previous thematic findings of the scoping exercise, but not to impose them unnecessarily. Once a code had been assigned, it could them be re-assigned upon encountering further data to which it was appropriate. It was thus possible to ascertain quickly the extent to which specific phenomena appeared in the data as a whole. The initial coding was fairly general: large segments coded with key themes. However, as the analysis progressed, it was possible to embark upon more detailed coding of segments of data and relate certain codes to one another. All the data was initially coded freely. “Free” codes had no immediate relationship with other codes, hence their name. They could then be attached to other free codes to which they could be related, often in a conceptual hierarchy. When they were attached, they would become part of a “tree”; having a key concept at the root (e.g. ease of use) with various branches growing from it (e.g. compatibility and efficiency). In short, “trees” function as a tool with which to hierarchically order concepts (themes). Hence, many “root” codes would be sub-coded with related emergent themes. This was invaluable when choosing the data upon which to concentrate in writing the results. Segments rich in recurrent
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 53
themes and encompassing examples of a variety of significant phenomena would hence serve as effective illustrations of the dynamics of e-Health implementation as a whole. This is not to say that atypical phenomena would be discarded. On the contrary, the atypical nature of such themes made them potent subjects for inclusion. To find a case “unusual” serves to emphasise the typicality of other cases; what Silverman (1993:44), adapted from Becker et al. (1961), calls “deviant case analysis”.
Once we had identified the emergent themes within the data, they were mapped onto the NPM to identify higher order themes. In doing this, it was possible to consider the place of these emergent barriers and facilitators within a broader process, relating each theme to the constructs within the model, and considering illustrative examples. However, the free coding drove this first approach to data analysis.
2) Once the thematic analysis was complete, a second, entirely distinct analysis was undertaken where the statements made within the task groups and interviews were treated as attributive statements and coded using the NPM directly as a coding frame. Attributions were therefore coded to one or more of the four constructs of the model, or if they were deemed not to fit within the model were classified accordingly. Two reviewers coded the statements independently, enabled differences in coding and interpretation to be identified and discussed. The distribution of attributive statements within the model, were then examined. This approach was expected to facilitate understanding of where, within the model, the bulk of data resided.
4.5 Results
The main themes emerging from our thematic analysis of the task groups and key informant interviews relating to barriers and facilitators to implementation were:
1. Patient benefits and clear purpose.
2. Collaborative working.
3. Standards; including a) universality, b) clinical risk, legality and safety.
4. Ease of use; including a) familiarity (with the system/interface), compatibility, and ease to learn, b) efficiency and c) technical support.
5. Professional attitudes; including a) unity of purpose and b) sense of user empowerment and relationship with suppliers and designers.
6. Clear rationale.
7. Cost; including a) time, convenience and physical space and b) financial cost.
4.5.1 Patient benefits and clear purpose.
Perceived patient benefit was a theme apparent throughout the data, and often cited by participants as a theme missing from the findings of the literature review. Clear purpose was inextricably related to patient benefit, and it related to what health care professionals perceived as the patient’s understanding of what new systems were attempting to achieve for them. An ambulance service redesign manager spoke of how the system used by
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 54
the paramedics enabled the quick and remote transfer of information between paramedics and hospital based staff, to the ultimate benefit and safety of the patient.
‘… rather than have the ….case of every time someone phones up for an ambulance, we turn up, we do something and then (take them)…to the department……. we don’t have to do that ……..we can keep them in the community………..’
While this form of communication and decision support could reduce unnecessary hospital admissions without compromising the safety of the patient concerned, the redesign manager added that there was reluctance to do this because of safety concerns. This demonstrated one of the dilemmas faced by health professionals in relation to new technologies: the confidence to use them without in any way compromising patient care. Thus, even if the technology enabled more efficient working, remote working, or both, it would not necessarily be immediately embraced. The need for professionals to understand the benefits to patients derived from new technologies was not exclusively about protecting their own accountability. In the same group, the former nurse, spoke of the benefits of electronic communication in the rehabilitation service.
‘I mean, it’s purely speculation, I think that the true value that they make to the patient systems are possibly for the people that don’t attend rehab, the people that we don’t catch now. It’s a different medium and it is maybe a bit more different from a hospital and that might be why it might catch them. ……..’
What was significant here was the availability of a new means of communication between the patients and professionals, and indeed the potential to assist previously hard to reach patients. It was suggested that while some patients may have initially been reluctant to engage in a face-to-face encounter with a doctor or a nurse, if they could communicate remotely (and with relative anonymity) they might receive assistance that would not otherwise be available. It was suggested that professionals welcomed this (rather than feeling undermined) and that the technology could increase the connection between professionals and patients.
What was immediately significant about this theme is that it did not emerge in the literature. Indeed, it seemed that researchers have underestimated how important patient benefit was to professionals. Aside from the two examples presented above, the need for a demonstrably positive impact on the patient was frequently cited by professionals as a facilitator to the implementation of new technologies.
It was not simply the practicalities of giving benefit to the patients through new technologies that was a concern of task group members, but also the patients’ understanding of its uses. A consultant spoke of patient satisfaction.
‘And yet you can’t quantify patient satisfaction. That’s the hardest thing. Patients may actually feel more secure knowing that ……, after what you said, I know now, if I’m going in an ambulance, I’ll feel much more confident because I know that these things are going on and I think you can’t assess that, you don’t know if your survival’s better because you know that!’
The consultant was referring back to the ambulance redesign manager’s explanation of the communication that could now take place between
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 55
paramedics and hospital staff. In putting herself in the role of a patient, she demonstrated how patient understanding of these new technology-enabled practices could improve patient confidence and ultimately their satisfaction with the services available. However, the other side of patient understanding was patient misconception, which was also an emergent theme within the task groups. The nurses within the NHS call centre spoke at some length of the public perception of their service:
‘N3 The way they advertised it has clearly gone way off the mark, and that’s why the public’s expectation is way beyond, it’s ridiculous, even in a perfect world I don’t think you could meet their requirements, but that’s partly the way it’s been put forward to them, because you will have 24 hour access to professionals.’
Clearly, the nurse’s assertion was that the advertising of the service had led to an imbalance between patient expectation and a deliverable service. While the call centre provided 24 hour telephone triage they were still, in many respects, advisors. While they could potentially diagnose problems and offer a course of action to the patients, they were in no position to offer hands-on medical care. Hence, the service’s abilities had, in her eyes, been exaggerated to the detriment of the service’s relationship with patients.
Patient benefit was also cited as important by the interviewees. A nurse described one of the main findings of her research:
‘……the focus of the system, in a sense, has to support … with the electronic health record, for example, it must be a patient-centred approach, as opposed to a professional-centred approach. Because otherwise, you won’t get the integration, and the benefit, and… in the same way. So, if it’s viewed from the patient’s point of view, and how the professional interacts with that patient, we’re more likely to get something that supports the patient’s journey, which is, after all, what we’re trying to do.’
The point made here was that although the patient-centred approach was, in her view, most important, the professional’s unity of purpose in providing patients with the best possible service was inextricably linked to it. These sentiments were echoed explicitly by other interviewees, and were implicit throughout.
While it was clear that the benefits to the patient were highly significant factors in successful implementation of new health technologies, the above quotations showed that there were several aspects to this: professional accountability and confidence; the ability to increase both the level of care and the number of patients available to care for; and the relationship between the patients and professionals derived through a clear and mutual understanding of a system’s purpose.
There were clear links to the NPM in the phenomena described above. The ambulance service redesign manager’s reference to confidence in the new systems was an issue of relational integration in that it suggested an impact on the pre-established interpersonal relationships both between
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 56
professionals and professionals and between professionals and patients. It demonstrated how the professionals’ sense of their own accountability to the patients was reflected in their willingness (or reluctance) to utilise and rely upon a newly established system. Hence, while the technology in itself was undoubtedly important, professional trust in such technology was the over-riding factor.
There was also an issue of the interactional workability of such technology. If a new system enabled better communication between professionals based in different locations (e.g. a hospital and an ambulance), there was clear benefit for both professionals and patients; the technology could be said to be interactionally workable. However, without the related sense of relational integration (confidence in the technology), such workability may not have been fully realised. Likewise, without the workability, integration was less likely. The Normalization Process, in terms of knowledge between agents, at least in the context of patient benefits and clear purpose, was hence cyclical.
The more general impact of the new technologies in terms of the overall practices of an organisation was also apparent when considering patient benefits. Certain changes in professional roles enabled by new technologies would certainly serve to benefit the patient, e.g. paramedics being able to communicate better with the hospital base and hence being able to offer a greater level of patient care in transit. This highlighted the role of skill set workability in patient benefit. A relatively subtle change in the division of labour may potentially have had significant benefits for both patient care and patient confidence, as perceived by health care professionals.
Finally, there was a broader factor related to a more general notion of patient expectation. This was illustrated most explicitly by the call centre nurse when she reported on the “bad press” they had received. This may have been a factor which transcended e-Health and was an issue for both health and technology as a whole; hence, within the NPM, it was clearly an issue of the contextual integration of the service or system (how the system fitted into the broader context of current healthcare). This served to demonstrate the “double-edged” nature of new technologies: these new systems may have enabled professionals to do more for patients, but the technology itself may not have met the fullest expectation of its users (be they patients or professionals). With technology came an expectation for significantly improved services, and should such expectations be proved unrealistic, it was the reputation of the related services which suffered.
4.5.2 Collaborative working.
The data suggested that the organisational changes necessary for successful system implementation had to take place on a holistic basis. One participant said:
‘We have a bit of difficulty though, because we have a separate database for collecting information on the paediatric units, which has no link at all with any of the other systems.’
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 57
This demonstrated one of the potential problems created by the variety of available systems. While one system may have been highly effective in the organisation of inpatient care, another may have proved more efficient to other clinical uses, in this case, paediatricians. Even if these two systems were geared towards achieving the same end (be they recording the relevant information in such a way that it could be retrieved quickly and efficiently, or suggesting an appropriate course of action on inputting a specific list of symptoms), if these systems were designed for different patient groups or different parts of an organisation, they may not have been compatible with one another. What the paediatrician implied is that the data recorded by the system that he used was difficult to integrate with that collected on the other systems within the organisation. While there were clearly benefits to having specific systems for specific disciplines, problems emerged if these systems were not in some way linked, leading to the problematic isolation of disciplines. This issue of compatibility between systems was one of interactional workability: how the compatibility between systems affected the exchange of information between professionals in real time.
Collaborative working seemed particularly important at the initial stages of implementation. A clinical manager from a task group explained:
‘…you’ve got to get the clinicians on board, you’ve got to get the staff on board, you’ve got to make the staff look at the system and make it such that it’s user-friendly, so that the content is such that it works for users, otherwise we’ll have the first point of resistance here and some of the systems should be such that it’s flexible and can be changed for user requirement.’
The clinical manager argued here that professional agreement was an integral factor in successful implementation, and that this agreement had to be reached early, preferably in the planning stages. He also implied that while agreement on the general use and purpose of the system should be as universal as possible for maximum impact, there should also be scope to tailor such a system to the needs of specific professional groups. This need for a balance between universality and user-specific requirements emerged frequently in the data, in many different contexts. Here, however, it was clear that collaborative working must exist in the first instance if such a balance was to be attained.
Interviewees frequently cited collaborative working as an important facilitator to technological integration. The former telemedicine manager asserted:
‘Technology is not an issue per se. I think cost is not an issue per se. I think the biggest hurdle and the biggest thing to overcome is get the people to talk to one another. You need two people. We made the mistake at the beginning. We had one person. The people at the other end were not too keen. So you cannot play by yourself. You have got to play with somebody else.’
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 58
The implication here was that without collaborative working in the first instance, other factors such as technology and cost were of less significance, since the system was less likely to get to a stage in its development and continued usage where these factors became the primary concern. Several interviewees expressed the same sentiments in similar contexts - a system could not be successfully implemented unless those whose work was directly affected by the implementation were collaborative in their will to use it. Collaborative working was essential to the successful relational integration of a system. If it was to be utilised fully by all, it was undoubtedly beneficial for the relevant professionals to have a sense of shared ownership of the system.
4.5.3 Standards
a) Universality.
The need to balance standardisation with flexibility of individual requirements was a recurrent theme. A GP in a task group described the delicate balance between universal standards and individual requirements:
“………. with NPfIT, the government wisely decides to produce a one single unified system, it’s not a problem so long as that unified system can be tailored and bespoke to whatever, however you use it, be it the hospital, be it clinicians, nurses, admin staff…………...’
Any system that was overly rigid was also restrictive to both users and patients. Once again, this tension between the need for similar working practices to ensure compatibility between systems and collaboration between individuals and organisations using them, and the ability to tailor that system to individual needs was evident.
As with the theme of collaborative working, there was an issue here of both the interactional workability and relational integration of a system. For a system to be successful it must be workable for the individual user, but relational in terms of professional confidence in its ability to work with the systems of other users. There was also an aspect of contextual integration, in the broader political will to standardise systems.
b) Clinical risk, legality and safety.
Risk, legality and safety were frequently posed as complicating factors when using e-Health systems. One consultant explained how many clinicians had trouble remembering the passwords that they were compelled to use. This had led to the unsafe practice of writing passwords down, and some consultants were simply not using the systems because they could not remember all their passwords. Hence, while these security measures were a standard means of reducing the risk of unauthorised access to data, they could create new security problems, and deter people from using the systems.
Two nurses illustrated another aspect of risk and professional standards:
‘N3 You get the regular caller who knows exactly what they say, but because they say I’ve got a pain that’s going down my arm, etc, they know they will get an ambulance, you have that side of it as well. We’ve had 30
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 59
calls in the last week that are rubbish, but they may not be, so you have to send the ambulance, because we can’t see them. We can’t say, you’re lying to me, you have to go with the words they’re saying, so that’s difficult.’
This demonstrated one of the problems faced by health professionals working remotely from the patients. While remote working could reduce waiting times, time spent in hospital, and travel time for both patients and professionals (as explained above by the ambulance service redesign manager), it could also compel the professional to take an unnecessary action due to lack of information. With no physical or visual cues upon which to make a decision, the nurse advisor was obliged to send an ambulance. The comments of a second nurse illustrated this problem further.
‘N2 You can ask to speak to the patient, listen to them, if they’re really gasping, you can hear that. You get patients who say they can’t breathe, or you get a mother that comes on and says their child’s got projectile vomiting, and our version is it hits the wall, their version is it’s a lot. They use words that are incorrect because they’ve heard them bandied around, but for what we mean versus what they mean, they are not correct. We can’t assess colour, temperature, we only have their version.’
This was an issue of different “languages”, as patient and professional understandings of medical terminology may differ. Again, this was an issue of interactional workability which transcended the use of new technologies, but may have been emphasised by them. Attaching different meanings to he same words can lead to an unnecessary course of action. Computer-based decision support systems cannot distinguish between these different understandings.
Informants also frequently talked of safety. GA discussed security at some length with one of the clinical directors for electronic medical records systems:
‘GA Is a computer any less secure than a filing cabinet?’
‘Obviously, in many respects, it’s more secure. The snag is that if there’s one corrupt person in a GP’s surgery then obviously that is a rare phenomenon and obviously relatively a limited number of people. Whereas if you’ve one corrupt individual and they’ve got access to five million records then obviously (there is) the potential for …… loss of data. So, that’s where obviously one has got to build in securities and access and regular checks about who has been accessing. But if there was a robust system that will only allow the patient access and obviously the technology is there to do that with smart cards and appropriate passwords, then obviously, to a large extent, that gets around the issue because the only person who can access it are the primary care team and the patient. And the patient then would be the guardian of the security and would only pass that onto those looking after him if he wished. What’s always, of course, the argument against that is what if the patient was unconscious? Now, that’s a very rare phenomenon for actually people to be admitted unconscious and in that situation there has to be some sort of break the glass solution but obviously,
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 60
they’re rare and therefore you could have actual individual monitoring to make sure that whenever that happened we would follow up to see that it was justified.’
This illustrated the two-sided nature of the security surrounding patient data. On the one hand, it could only be viewed by a small number of people, and those who viewed it were clearly accountable. However, the rigidity of security measures could also (albeit very rarely) potentially compromise patient safety if the required information was inaccessible. Hence, the rigidity of standardised and secure record systems may not always be to the patient’s benefit - hence the need for a “break-glass” solution. This was an issue of interactional workability: the system must be secure, but not to the point at which it would compromise access when needed.
Security was clearly an issue of relational integration, both in terms of confidence (that a system was secure, that the given professional had the right to access the data in the right circumstances) and accountability (that professional integrity would not be compromised by an insecure system). This linked to an issue of skill set workability in that the technicians whose role it was to keep the system working safely and securely, potentially shared in the accountability of the medical professional. If a system was breached, it may have been the case that both the health professionals and the technicians were accountable.
4.5.4 Ease of use
a) Familiarity, congruence and ease to learn.
This theme arose both in the literature review and in the original data, since all users of e-Health systems share the concern about ease of use: either a system was easy to use (a facilitator to successful implementation) or it was not (a barrier). Familiarity was seen as beneficial, as a GP in one of the task groups described one of the advantages of the system used in his practice:
‘(The) system is different to some of the others as it’s Windows-based and it’s very good, and with it being Windows-based it’s got potential to share a common language with potentially other systems………….”
The implication here was that if certain aspects of the system were already known by virtue of their general usage, such a system could be learned more quickly. Professionals who were new to the e-Health system would probably have used Windows, so a Windows-based system was likely to be compatible with their established knowledge and expertise.
Interviewees also cited congruence as an important factor. A clinical director:
‘The change to their working lives in terms of sitting down in much more structured manner than what they would have done previously they always structure time in their day to do review of films but it was not infrequently interrupted but not being able to get the films to them and what have you,
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 61
but that doesn’t happen any longer…… and for the NHS in Scotland I mean just the fact that we can store things electronically, transfer things electronically compare images etc it’s a major service benefit.’
The implication here was that as well as being congruent with existing working practices, the use of technology could enhance such practices, creating stronger working structures and reducing the time and effort taken to conduct certain tasks. It almost suggested that technology could potentially create a certain idealised structure, allowing professionals to work to their optimum level. As the evidence has shown thus far, the reality was far from as simple as this, albeit the possibility of this ideal being fulfilled was undoubtedly a facilitator to successful implementation.
In general, the ease of use of a system was in its ability to accomplish tasks more quickly, or by offering an extra benefit not possible without the system. All three respondents quoted here referred to the interactional workability of the respective systems. However, while ease of use fitted most obviously with interactional workability, the clinical director’s reference to the structural changes enabled by it showed skill set workability to be an outcome. If the implementation of a new system impacted positively on the structure of working, it was likely to facilitate successful integration. This linked to a further aspect of ease of use: efficiency.
b) Efficiency.
Related to congruence was efficiency. The call centre nurses were in general agreement about certain problems in the efficiency of the programme they used, and its compatibility with the requirement for real-time action:
‘N2 I think there are very good things in there, if you had the time to play with them. For us using it at the speed that we’re expected to use it at, it’s extremely tiresome.’
‘N3 Too much scrolling up and down, down here and over there.’
‘N2 We need a lot of information, but sometimes it’s not in the right place, it’s hard to find, it’s difficult. Somebody showed me how to pinpoint something on a map, and then actually to hone in on it, to give you the actual name, and I can’t remember how to do that now. I went to the person who was showing me, and nobody knows how to do that. …… I still don’t know how to give people directions to their health centres. There’s a lot of information in there, but it’s cumbersome to use.’
What emerged here was the problem of too much information. It was clear that this particular aspect of the system (which told the nurse advisors of the nearest health centres to the patient’s location) could save a significant amount of time between telephone triage and treatment. However, if a large amount of information was instantly available, the nurses and the patients had to filter it themselves. While this may have sounded like a minimal disruption in itself, in a telephone triage service it may lead to patient dissatisfaction. Again, this was an issue of the real-time interactional workability of a system.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 62
The need for an efficient system which complemented existing working patterns was cited frequently by task group members. A pharmacist spoke of his system:
‘It’s also quick and responsive in the main, which again, is easy for people to sign on, they can move through things fairly fast. So the idea is that hopefully it complements our workload rather than heeds it and to me that is one of the biggest strengths of what we’ve got.’
This linked back to the GP’s comment about familiarity with the system. It also suggested that the actual perception of the system being easy to use would lead to a greater willingness to use it. However, whether or not a system was in fact easy to learn and efficient to use, depended in part on suitable support staff being available.
c) Technical support.
Attitudes towards technicians varied, as shown by these comments from a laboratory technician:
‘…the person who is in IT support wants ease of use when something goes wrong they could do something about it, because users think they’re trying to be obstructive, but they’re not, they’re not really. And the person using it who’s used it for a long time, wants it to work in their way, and the person who’s coming in new wants to walk in and they want you to say, oh that’s obvious, I’d do it like that.’
These comments demonstrated how, while technical support was obviously necessary for users of the system, it could also be seen as an intrusion. The use of technology necessitated a reliance on those familiar with such technology, so that technicians become involved in the work of health professionals. This related to the skill set workability of a system: if it could not be consistently used by a health professional without technical support, giving technicians a significant role in the delivery of healthcare. While the health professionals who made up the task groups were on the whole grateful for this input, there was also a certain sense that this reliance on technicians could undermine their sense of autonomy. This linked to another frequently recurring theme within both the literature and the data: professional attitudes.
4.5.5 Professional attitudes
a) Engagement.
The necessity for engagement with health professionals using new e-Health systems was mentioned frequently within the task groups. The following comment was made in a task group.
‘X Absolutely, and that… you kill off the enthusiasm. I’m still so angry that they’ve slowed down such a lot and your very point; is that within a hospital setting everybody listens. It might not be implemented but you are given a reason why it’s not. Whereas nationally; there’s no reason. There doesn’t seem to be any ownership of anything anymore.’
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 63
This suggested that a lack of engagement and communication led to an undermining of professional enthusiasm for the new systems. If there was no clear rationale and no recognisable “ownership” of the system, the will of the workers on the ground to use it may have been diminished. This further related to confidence and successful relational integration.
b) Sense of empowerment and relationships with suppliers and designers.
Task group members and key informants were concerned with their relationships with those involved in design and implementation. While task group members were most concerned with having constructive dialogue with managers and implementers, key informants were more concerned with suppliers and designers. However, this desire for a constructive relationship with those involved in the earliest processes of implementation related to a wish for a sense of empowerment on the part of both implementers and users and the ability to influence the development and use of new technologies.
A pharmacist spoke of the need to involve clinicians in the initial planning prior to system implementation.
‘One of the big changes for me, just looking at what succeeded here before, and the way we’re going about it now, is obviously the different cultures, as well. The approach seems to me, 20 years ago, was to get clinicians at the heart of the project. We pulled doctors, pharmacists, radiologists, therapists, but things have moved on and it’s very much more like a project management approach in the treatment of the project. And I’m not sure that the field…test that we’re, even in our own organisation and nationally, gives enough to pull the clinical community on board as well.’
This suggested a link between empowerment and cultural change. As the technologies involved became ever more complex the relationships between the designers, implementers, and users may have become more remote.
Following on from this, a health professional advisor explained how implementation could potentially lead to a sense of disempowerment.
“It was just the whole concept of moving into this unknown, unseen world; I think really was the attitude that was prevalent. I think that it’s the whole concept of finding themselves straight-jacketed. And I found that more through the development of the data standards, where we want to be able to get them to deliver information in a way that doesn’t contain a lot of free text information. It’s squeezing them into boxes, and giving them… they would have a pick list that they would wish to have under particular categories”
This linked back to the rigidity of working structures that a technology could impose. A system which relied on certain algorithms for entering data may not have presented the user with all the variables they required. This is another example of a tension between the need for a standardised approach
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 64
to data and the needs of the individual clinician. The implication was that systems may be designed by those whose agenda was to get the best technological solution, rather than the best enhancement to the clinicians’ duties.
Some key informants identified their relationship with the suppliers and designers of new technologies as an important factor in implementation – and one that was not covered in our literature review. One of the clinical directors interviewed explained:
‘To me, the biggest constraint I have is relying on suppliers, which I don't think is on here (in reference to the literature review). My programme, xxxxx, is dependent on the delivery from several suppliers, all of which are critical for success. And yet levers we have in order to move them are minimal. So access to the suppliers is actually one of the biggest problems I have.’
While professional attitudes were obviously related to professional confidence in the system and relational integration, there was also a more general cultural factor illustrated here. The link between empowerment and cultural change, and indeed the political dimensions of relying on suppliers and/or managers was an issue of contextual integration. Implementation had to be understood in the broader context of health policy and management.
4.5.6 Clear rationale.
Though task group members sometimes referred implicitly to the need for a clear rationale, this theme was more prevalent in the key informant interviews. The rationale also had to be locally, not just nationally, recognisable.
A clinical director gave the following example of a clear purpose for an e-Health management system:
‘There are a number of reasons,…..our existing paper systems …. had problems of legibility and these were long standing problems’
In this setting, the clinical director explained clearly why a computerised system was necessary for his service: legibility of patient information in the notes which apparently were maintained by several different professional groups. The initial problem was one of how to decipher handwritten data . This rationale could be appreciated by all concerned and was used to justify standardisation in practice. By implication, such a widely appreciated rationale would inspire at least an initial general confidence in the new system. Hence, a clear rationale for the use of a system was inextricably linked to its relational integration.
As well as justifications which reflected individual and collective experiences, there were also those related to evidence presented to the implementers. This evidence could take the form of published research and/or knowledge from other implementations. The proliferation of this evidence would shape the context in which implementation was to take
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 65
place. A health informatics group member, when asked about justifications and knowledge about how implementations had worked elsewhere, explained:
‘I suppose we’re coming back to what is the definition of an e-Health system. If you’re talking about distant systems, tele-medicine systems and so on, then there’s quite a lot of work in the Nordic countries about the effectiveness of those. But the whole principle that you’ve just described of something that works well, that’s particularly the American experience because the Americans will build a system in their university hospital and, because they’re building it in their local hospital, then they’re making sure that it fits their local hospital. This applies both to electronic medical record systems and the distance orientated systems. It would be the same if you built a system for your local health environment; it would work very well because you’d be sitting and talking to them and you’d build in a way which fits with their ways of working. But if you then tried to transfer it to xxxxx, it wouldn’t work half as well.’
This individual referred to evidence available about successful implementations in other contexts. He explained that certain systems had been purposefully developed for very specific contexts. The implication was that, while this would produce evidence of a successful implementation, it would not necessarily be universally applicable. Hence, while evidence of previous successes could act as an initial rationale for implementing a specific system, it was not necessarily the best rationale. Once again, this emphasised the tension between universal standards and individual needs.
4.5.7 Cost
a) Time, convenience and physical space.
There were two essential aspects to the theme of cost, the first of these being that relating to the cost in terms of working practices; time, convenience and physical space. A task group GP explained concisely the impact of a “paperless practice” on the physical work space.
‘Well, we’ve only come to computers over the last six years or so and the one thing that we, we need a bigger building already, so we can’t be as flexible… it means you can’t work anywhere unless there’s a workstation there, and so you can’t just sort of sit in the corner of an office and do some paperwork. You need the computer.’
This showed how the presence of a new technology altered the working practices of health professionals. The implication was that the necessity to use a computer led to professionals being confined to a particular work space for certain tasks, and that the need for these particular spaces led to the need for an increase in the overall space available (which in turn had financial implications).
While the change in the structure of working related to skill set workability, the related change to the actual “hands on” work of the professional was interactional. A nurse in the same task group complemented the GPs comments regarding physical space by stating that “you can’t just use a pen now.”
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 66
b) Financial cost.
The financial cost of a system was less important to the professionals than it was to the implementers. As a task group doctor explained:
‘Cost is only relevant if that cost if we as a trust save X amount of money by decreasing x-rays, which means we can spend it on something else. If you save money and nothing happens, then cost becomes irrelevant. And cost seems to be irrelevant to this, because they seem to be throwing money left, right and centre, and if you can do something cheaper, one they’re not interested, and two they won’t give you refund anyway. So cost has become less relevant whereas it shouldn’t have.’
Here, the doctor demonstrated a belief in the apparent indifference with which cost was regarded by the trust. This belief was reflected further in the key informant interviews. It was interesting to note that while financial cost was undoubtedly an issue that key informants were aware of, many explained at some length that it was not as important as might be supposed, at least not to the users of the systems. One of the clinical directors explained:
‘I think as far as constraints on the ground as far as the health service is concerned, costs, although it's very important for the people who are managing the NHS, for the user isn't an issue, because they're not paying for it. So I think that, where it used to be an issue in general practice, even that's gone, because we've taken over the funding. So actually cost, where it was a lever, is now not a lever and people take it for granted.’
The clinical director described how the ownership of financial cost had, in some respects, shifted, particularly in relation to general practitioners, whose funding had been taken over by the health trusts. Hence, while the financial cost of a system was once an issue that was a concern of many involved in new implementations, changes in the management of finance within the health service, combined with an implied proliferation of innovations, had led to an increasing distance between concern with cost and the actual system users.
The informant from the health informatics professional group brought together all these points:
‘Well, that’s one of the other points I was going to raise because you’ve got a box for cost without a lot of description about it later on and of course cost covers a multitude of sins. And what nobody ever seems to think about is the total cost of ownership. That’s actually the only real way to make an effective judgement about cost. And, again, it’s slightly changed in England now because of the national program, although I’m not so sure it’s changed quite so much in Scotland. The cost, because it’s being born by the health service, is less of an issue, but of course it is for the health service staff in PCTs or whatever who are implementing it. And they tend to look at just
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 67
the headline cost and of course that’s not an efficient or effective way of ensuring adequate implementation.’
This respondent here linked cost in terms of time and space with the financial cost to an organisation; the “total cost of ownership”. He described the different levels of concern regarding cost between implementers and users. His assertion as to the “headline cost” implied the availability of prior knowledge of the overall financial cost, contrasted with a lack of how best to distribute such, and the broader implications for organisational structures. Hence, the contextual aspect of the overall financial cost of an implementation was inextricably linked to the cost in terms of the workability of the organisation and individual users.
4.6 Results Using the NPM as a Coding Frame.
Data from the transcripts were divided into 712 discrete attributive statements relating to implementation issues. These were then coded using the four constructs of the NPM described below.
Interactional Workability (IW)
As in the previous section items coded to the interactional workability construct included generic ease of use issues but also issues relating to whether technologies being implemented were deemed to be “fit for purpose” for users, or not, in terms of whether they facilitated their work.
An example includes this comment from one of our task groups:
‘I’m a user and …you’re right I could get any system in the account, but I can’t use that account on my computer. I have to go to a separate computer. So then I have to carry two computers around to do that. It’s just a waste of resource. So in effect the technology is not sufficiently sophisticated, if you like, to allow us to make use of the solutions that have been deployed.’
Suppliers often say that “technology is not the problem” however, users on the ground, often disagreed and experienced all sorts of practical problems in relation to the use of new system as this interviewee explains:
‘The other thing that has come out for me from that particular application is that the technology was always assumed to be fine. The written spec (company X) had said that it’s all done and dusted and they (had) done some basic testing in the XXX but then when they went out on the ground they discovered that actually firstly it didn’t do as it said on the tin, when they got it to extract records from the real live GP system, the system fell over endlessly and corrupted the data and all sort of things went wrong with the upload, in fact they’ve uploaded very few records already because it was just ill prepared ……………..Interviewee 2’
Such difficulties, relating to the effectiveness of new technologies when put into practice, influence professional perceptions in a negative way. There were many examples showing how systems which failed to deliver as anticipated led to disappointment and disillusion amongst users.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 68
Relational Integration (RI)
Statements relating to this theme included issues of confidentiality, security and confidence in the systems in use. An illustrative example:
‘There was something in it for everybody, really. You know, the nurses got the decision support; so, they felt more confident in their decisions. The training, I think, helped with that as well, the clinical training. The patients, obviously… it was the service they were more interested in, I think, rather than in terms of how the service affected them. I mean, obviously, they hoped having a decision taken by a professional’s a good one anyway. So, I think, the impact of the IT was less obvious to them, it was much more the impact of the change in the service that was obvious.
But, so, for the nurses it really was a major improvement. And, I think, interestingly enough, the doctors felt more confidence in the decisions taken by the nurses, as well.’
This shows that professionals did not necessarily display negative attitudes - a recurring theme from our work is that “professional attitudes” do not represent a “fixed state” or unfounded “belief”. Rather they are shaped by the way an e-Health initiatives affects professionals’ perceptions of their work.
Skill Set Workability (SW)
Skill set workability issues related to training, workload and roles and responsibilities. The comments from respondents illustrated for example that the issue of training and support was a more complex issue than those implementing new e-Health services appreciated:
‘But then, on the other hand, if you say education and training, I think that’s a huge issue as well. I think people don’t really understand how best to use it, to get the best out of it.’
“The basic training in terms of using x is good. But the problem is if you’re only an occasional user, and most of my colleagues use it even less than I do, then it’s very difficult because there isn’t an indoor help system in here. So in all, if you only ever order blood tests when your SHO is on leave, you end up having to phone the helpline, which is bad enough, when it was a x helpline. But now we have this x line who don’t answer the questions. They say someone will phone you. So you phone and just hope they’ll get back to you. And I think I’m probably now comfortable enough with x not needing very much, but for anyone new in our department, because we don’t use it everyday as the busy paediatric service would, the training is all very well, but you actually need to practice.”
‘It’s not a training issue, it’s educating people to understand what the context is, the whole new remit of IT, and the usefulness of it, if there is a usefulness, or the deterrents of it. When you want to come into training, as far as I’m concerned I’ve been on endless technical courses, and I never remember a thing. I go in, I do it absolutely perfectly, I walk out of that
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 69
room and I’ve forgotten every single thing. The only time you ever start remembering anything is when you start doing it, and having to do it. So to me, I tend to look at the word training as it’s not really training, but you need somebody to come in and work with you in your environment. Somebody you can go to, oh, how do you do this, and then you go back and try it, and then come back, it’s not working, and can you just show me? And you feel so bad because you’ve asked them to stop what they’re doing, to come and show you, that you learn, and that to me is the real training. I think you can spend fortunes in classroom training, and getting them in, and you don’t get anywhere. Interviewee 13’
This shows clearly how simply providing e-Health “training” courses is perceived as inadequate and not meeting the needs of e-Health users. “On the job” training and support as well as help with “problem solving” which is readily available seems a far greater priority for health professionals in practice.
Contextual Integration (CI)
Contextual integration issues included broader organisational and contextual issues such as:
‘Yes, there's a very interesting dimension there because, where the user groups have a financial driver over the suppliers, it's a very different relationship, if the chequebook is in the hand of the user. And when it's not, in other words a third party operates the contract and the users are at the mercy of that third party and they have no control over the system supplier, that whole dimension changes. So it's quite an interesting one.’ Interviewee 6.
‘So, in terms of, there are many factors within the sustainability in our health care system, such as the shortage of future manpower, clinicians, and so forth, that we just have no alternative but to look at how technology can support delivery of health care. So, while cost is a big issue, it’s the realisation that we have no choice really, but to look at how e-Health can actually help us.’Interviewee 5
This illustrates how contextual integration concerns not only financial resources but wider issues affecting how and why organisations deliver services – such as personnel issues or the challenges being presented by the populations they serve.
Outside the NPM
Issues that fell outside the model included the need for champions and user engagement, as well as strictly technological issues. An example:
‘I think there’s very much a need for evaluation of things particularly with regard to the cost effectiveness. One of the sad things at the moment is that this concept of a new central record like the Spine seems to be, not only attractive, and somehow, politically considered the way forward in England but sadly also in Scotland. And for some reason, I think it’s partly
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 70
because the concept of building on a GP record is not acceptable to secondary-care clinicians and I think that’s been very unfortunate. So, I think with regard to objective analysis and assessment there’s very much a need for the same degree of effort going in to evaluating policies and implementations…… as goes into the assessment of a new drug. We would never implement a new drug unless it had gone through very stringent, and yet we seem to be very willing to suddenly adopt health policies without any evaluation at all.’
The distribution of attributive statements across the NPM constructs can be more easily visualised by means of a radar plot as illustrated in Figure 3. The axes represent the numbers of attributive statements and each of the constructs interactional workability (IW), relational integration (RI), skill set workability (SW) and contextual integration (CI) are shown at each of the poles. Table 4 provides the numbers of attributive statements in each category that were used to develop the radar plot. As in Section 3.7.2 these tables and figures have no statistical properties but merely demonstrate the distribution of comments relating to barriers and facilitators across the NPM. It is just a graphical way of presenting this content analysis in an effort to determine whether specific aspects of the model are ignored by health professionals or given particular emphasis, by being frequently commented upon. The findings from this WP reveal that health professionals provide a more balanced view of implementation, touching on issues of interactional workability (ease of use issues), relational integration (confidence, security and safety issues), skill set workability (training, workload, roles) and contextual integration (organisation resources and effort).
Table 4 shows the distribution of codes within each construct from each of the transcripts.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 71
Table 4. Distribution of Coding of Attributive Statements
Interactional Workability
Relational Integration
Skill set workability
Contextual Integration OUTSIDE
Task Group 1 20 12 15 16 5 Task Group 2 13 20 6 20 13 Task Group 3 4 10 4 8 17 Task Group 4 11 16 6 6 22 Task Group 5 5 18 9 23 41 Interview 1 2 5 3 3 9
2 4 4 6 3 11 3 8 2 2 5 4 4 2 7 6 3 7 5 7 9 10 16 17 6 3 4 1 5 3 7 1 4 2 0 10 8 2 4 1 3 5 9 1 7 4 4 8
10 9 2 2 1 6 11 1 6 3 9 5 12 1 1 2 11 7 13 4 6 3 7 5 14 0 3 4 8 3 15 0 3 1 0 8 16 2 5 8 1 4
100 148 98 152 210
Figure 3. Distribution of WP2 Coding Against NPM
Distribution of WP2 attributive statements across the NPM
0
50
100
150
200Interactional Workability
Relational Integration
Skill set workability
Contextual Integration
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 72
This clearly shows that the health professionals gave a more rounded view of implementation barriers and facilitators than the literature review as their comments covered a broader range of issues relating to socio-technical as well as organisational concerns. Figure 3, above, illustrates how the comments address all four constructs of the model.
4.7 Discussion.
In this work package, we collected an array of professional views on the barriers and facilitators to the implementation of e-Health systems. These findings suggested that, though there were strong correlations with certain themes in the literature review (such as professional attitudes and ease of use), there were also significant differences. First, professional “attitudes” did not simply represent resistant behaviours but often reflected experience of using e-Health systems. Second, the professionals’ need for a clear sense that a system would benefit patients (and not just themselves) was a prominent theme. Third, cost was considered in a broader sense, explicitly including staff costs and the total cost of ownership idea, while the literature, for the most part, assumed cost to be primarily a question of (financial) resources.
The themes which emerged from the task groups and interviews covered different, and in many instances, broader ground than those which emerged from the literature, and were obviously more personalised.
Mapping the emergent themes onto the Normalization Process Model was particularly useful in demonstrating the relationship between themes and how the various themes and sub-themes could be recognised as part of a broader organisational process. Table 5 maps each identified theme onto the NPM:
Table 5. The Normalization Process Model applied to emergent themes in task group and interview data.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 73
Interactional Workability.
Relational Integration.
Skill-set workability.
Contextual integration.
Patient Benefit.
Remote working enabling better professional/patient communication.
Confidence in the system shared by professionals and patients.
Impact on working practices and roles; e.g. paramedics being able to do more outside the hospital.
Linked to a general idea of patient expectation; e.g. NHS 24 receiving a “bad press”.
Collaborative Working.
Compatibility and incompatibility between systems.
Collaborative working between professionals and organisations generally.
Standards.
Problems in accessing secure patient information.
Confidence that the system is safe and secure.
Involvement of IT technicians in formulating clinical standards.
Political will to “standardise”, broader societal concerns with data protection.
Ease of use.
Ease of use of the systems, familiarity and compatibility in general.
Changes to working structures: a more “structured structure” enabled by technology.
Professional attitudes.
Confidence in the system and the implementers, suppliers and designers.
Link between empowerment and cultural change.
Clear rationale.
Clear information given to health professionals.
The availability of evidence.
Cost.
Time and convenience: “you just can’t use your pen now”.
Change to working spaces: “you can’t just sit in the corner of an office and do some paperwork. You need the computer.”
Financial cost to the organisation.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 74
Table 5 re-iterates the relationships demonstrated in the results section of this report; how the themes identified were recognisable as part of a process of normalization, or indeed hindrances to such a process.
On considering these themes, we drew two conclusions which relating these seven main themes to one another. These two meta-themes emerged because they were subtly and at times blatantly constant factors within the initial themes. They were less identifiable in the initial analysis of data as they were expressed in many different ways. It was through the exploration of the seven general themes that their presence became apparent.
1. The tension between universal standardisation and individual need.
2. Broader cultural change.
The tension existing between the desire for universal standardisation (for the sake of efficiency, compatibility and communication) and individual specification was a major issue. It was at times blatant, as in the theme of standards and universality, and at other times more subtle, such as when it emerged in relation to ease of use, collaborative working, and security. It was an issue of interactional workability in terms of its impact on the system with which the individual professional worked, relational integration, in terms of the affect on the confidence (either positive or negative) of the professional users, of skill set workability, in terms of the role-blurring potentially created in a standardised system, and of contextual integration as it related to a broader political will to standardise practice.
Broader changes in society and culture which had led to the current policy of e-Health implementation were referred to implicitly by task group members, in the terms described above, such as the desire to standardise, the expectations of patients surrounding technology and the friction between expectation and reality. However, key informants spoke more explicitly of societal and cultural change as a factor in implementation. This was most apparent when referring to cost, rationale and professional attitudes. The theme of culture was most obviously a factor in the contextual integration of an implementation. Changes in cultural expectation, influenced the interactional workability and relational integration of a system, in that the use of technology was all but unavoidable, and accompanied by a managerial expectation that its presence would lead to a more efficient and better service. Furthermore, the use of standardised technology amongst professionals had led to a more general cultural acceptance (albeit in some cases a reluctant acceptance) of a blurring of their roles; hence, the skill set workability of a system was also related to cultural change.
The additional direct coding to the NPM followed by examination of how the data was distributed across the model led to a richer analysis as it immediately showed that the health professionals provided a more complete picture of implementation than reference to the published literature alone permitted. Professionals did not focus so much on the organisational issues such as resource allocation and policy, instead paying greater attention to socio-technical issues such as usability, accountability, confidence, workload and support issues.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 75
Methodological Issues/Limitations
We did experience difficulties in recruitment as discussed in Section 4.3.3. However, our task groups and interviewees covered a broad range of professionals, at different locations across the UK, and with different experiences of e-Health services. By the end of the interviews no new themes appeared to be emerging. Furthermore, a major strength of this work package was our dual approach to analysis which was undertaken rigorously, our strong theoretical underpinning and the discussion of our analyses within a multidisciplinary team. Such an approach to analysis adds robustness and increases confidence in our analyses and findings. Our analyses have helped to identify the main barriers and facilitators to the implementation of e-Health, as well as factors which would facilitate the successful establishment of these implementations and the normalization of an e-Health system.
This work-package allowed us to identify a broader range of implementation issues than the literature alone had provided. Importantly, direct coding to the model also illustrated areas highlighted by professionals as important that did not fit within the NPM, in particular, this included issues such as clinical engagement, user involvement, technical issues, and evaluation and appraisal of the benefits, if any, of e-Health systems. This therefore highlighted aspects of implementation, which the NPM does not consider, but that are still clearly relevant. This was less clear through use of thematic analysis alone. Thus our dual approach to analysis added value.
Recommendations stemming from these analyses are offered below.
4.8 Conclusion: Recommendations for Implementation.
1. The chances of a successful implementation will be improved by the establishment of an ongoing three-way dialogue between designers, implementers and professional users.
Such engagement is essential to maximise the interactional workability of new e-Health systems, exemplified by “ease of use” issues - the ability of an e-Health system to help the user in accomplishing defined tasks. Furthermore such interactions are needed to ensure that professionals can have confidence in new systems and also so that new systems are designed to be flexible and meet the workflow or “relational integration” needs of professionals and their organisations. Only once a system starts to be operationalised, will some difficulties in use be identified, hence the need for ongoing interaction between designers, implementers and professional users.
2) It is essential to communicate a clear rationale for implementation of any e-Health service, in terms of both its professional and patient benefits, in order to promote uptake and utilisation.
New e-Health services are more likely to be successfully implemented if system users, in this case the health professionals, have confidence that the effort required to get the system into everyday usage is balanced by the likely benefits not only to themselves but to their patients. Potential
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 76
benefits need to be perceived to have relevance from both a national and local perspective.
3) It is necessary to establish a balance between individual need and standardisation for the purpose of compatibility.
If arrived at consensually, this would lead to a broader, more enthusiastic uptake of a system by professionals.
4) Education, training and ongoing support is crucial.
Importantly, such training is not solely about how to work the system, but also about increasing appreciation of anticipated benefits and limitations of the system as well as how to use any system to greatest advantage. Such support also needed to be long term rather than one off.
5) The safety/reliability of any new e-Health system must be clear.
For example, a back-up plan should exist if a system fails or “locks a user out” in a medical emergency. Health care is unique in that a system failure can literally have “life or death” consequences unlike so many other sectors where new technologies have been introduced. Health professionals are particularly cognisant of medico-legal issues and the “risk” element of their work and are therefore understandably intolerant of system failure. Professionals implementing any new technology must first and foremost be convinced that it is a “safe” thing to do. Furthermore, health data is considered particularly personal and sensitive and therefore data security is seen as an even higher priority than in many other sectors.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 77
5 Work package three (WP3) Development and Validation of the Technology Adoption Readiness Scale (TARS)
5.1 Background
Approaches to understanding innovation implementation have begun to focus on the concept of ‘readiness.’ Although, much of this work has centred on telehealth, we believe it is still has important implications when considering implementation issues in the broader field of e-Health. In the context of tele-health, readiness has been defined as ‘the degree to which a community is prepared to participate and succeed in tele-Health, and is the cognitive precursor to behavioural resistance to, or support for, change’ (Jennett et al. 2003a). A number of models have been developed (Jennett et al. 2005a; Jennett et al., 2003a; Jennett et al. 2003b; Oliver & Demiris, 2004; Overhage et al. 2005; Snyder-Halpern, 2002) that identify both different stages of readiness and different levels of assessment of readiness. In terms of stages, Jennett et al. (Jennett et al., 2003a) specify four types: core; engagement; structural; and non-readiness. Campbell et al. (Campbell et al. 2001) classify readiness into ‘fertile soil’, ‘partly fertile soil’, and ‘barren soil’ and suggest strategies for implementation that are stage appropriate. Levels of assessment of readiness that have been studied include patient, public/community, practitioner, and organisational levels. The majority of these studies focus on practitioner (Hebert et al. 2002, Paquin, & Iversen, 2002; Jennett et al., 2003a; Oliver & Demiris, 2004; Snyder-Halpern, 2002) and/or organisational readiness (Jennett et al., 2003a; Oliver & Demiris, 2004; Overhage et al., 2005; Snyder-Halpern, 2002; Hebert et al., 2002; Lehman et al. 2002), while practitioner readiness has been assessed mostly in relation to tele-Health in rural contexts. Summarising this literature, Jennett et al. (Jennett et al., 2005) identified three themes common to these models: an appreciation of the practice context; strong leadership; and a perceived need to improve practice. All of this research was conducted either in Canada or the United States of America, and that which focused on tele-Health was conducted in rural contexts where the needs of rural providers have been emphasised. Jennett and colleagues emphasise a need for readiness tools to be developed and utilised in the context of tele-health (Jennett et al., 2005).
The models referred to above identify a range of factors that relate to readiness to use information technology in healthcare (including tele-Health). The studies which are potentially most relevant to the present study have all focused on tele-Health or telemedicine (rather than technology innovation in general), and have included the perspectives of health professionals (Jennett et al. 2005, Hebet et al. 2002 and Campbell et al. 2001). However, these models, whilst rich in content as they have been developed through thematic analysis of qualitative data, have not been extended to the development of quantitative tools for assessment of staff perspectives. The research on organisational readiness has led to the development of a small number of quantitative instruments for assessing readiness. This includes the ‘Organisational Readiness for Change Model’ (ORC) (Lehman et al., 2002); the ‘Organizational Information
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 78
Technology/systems Innovation Model’ (OITIM) (Snyder-Halpern, 2002); and a questionnaire (Oliver & Demiris, 2004) who drew on both the ORC and the OITIM. However, with their primary focus on organisational issues, these instruments were not suitable for assessing health professionals’ perceptions of the impact of new technology on collaborative working. The latter approach is the objective of this work-package, thus reflecting the underlying propositions of the NPM as outlined in Section 2.
5.2 Aims/Objectives
To develop a structured, predictive instrument to test the contextual readiness of a health care setting for uptake and routine use of a specific e-Health system. The notion of ‘contextual readiness’ refers to a state of readiness for practice that is based on the interaction between individual and organisational factors affecting use of the system, rather than on individuals’ intentions to use a system. It is intended that the instrument developed in this project could be used to identify staff perceptions of factors related to the collaborative work required for the normalisation of particular e-Health systems, as set out by the Normalisation Process Model (outlined in Section 2 and Appendix 1).
5.2.1 Structure of the Work Package Report
This work package undertook the development and testing of a quantitative instrument to assess the readiness of a health care setting for the uptake of e-Health technologies by health professionals in their work. There were three phases of this work package, which are detailed in separate sections:
1 Instrument development processes that drew on the NPM, existing literature and a survey of experts in the field;
2 Development of a generic instrument of factors for rating relative importance and the testing of this with a sample of health professionals; and
3 Development of a specific instrument to elicit ratings of specific e-Health systems and testing of this in two different NHS contexts of e-Health use.
5.3 Phase 1: Item development & expert survey
5.3.1 Aim of phase 1
Phase 1 aimed to develop an initial set of items for inclusion in TARS and to assess the perceived relative importance of these items using a sample of experts knowledgeable about e-Health implementation.
5.3.2 Methods
We used two methods to achieve the aim: (i) construction of items representing factors (barriers and facilitators) affecting the normalisation of e-Health; and (ii) validation of the items using an online survey of experts.
5.3.3 Item construction and piloting
Item construction began by generating potential questions reflecting the four constructs of the NPM . (TF, CM). This involved translating the
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 79
constructs into plain language statements, each of which having a single and comprehensible meaning. For example, a direct translation of part of the CI construct in the NPM might be the statement ‘…… the extent to which organizational effort is allocated to an e-Health system in proportion to the work that the system is intended to do.’ Apart from the complexity of the language, the construct is multi-dimensional, so it was necessary to express it in more simply worded statements, such as ‘sufficient organisational effort has gone into supporting the system’ and ‘the rewards of using the system outweigh the effort’. This process of translation was necessary to devise theory derived statements for rating as stand-alone questionnaire items.
The theory derived statements were mapped (TF, CM, GA) against the thematic findings of the scoping review in WP1. This confirmed that the NPM derived statements reflected adequate coverage of key factors affecting the implementation of e-Health as identified in the review and no additional items were required. This process resulted in 23 items for rating. The draft item set was then circulated amongst the research team for critical review and comment, and as a result, amendments were made to develop a final set of 27 rating items to be included in the online survey.
The item set was pilot tested as a live link by members of the project advisory group (n = 5 of 10 invited). This resulted in several refinements to the questionnaire, including technical features concerning its delivery, content of text descriptions accompanying questions, and the revision of two factor items that were unclear to testers. .
5.3.4 Online survey of experts
The purpose of this stage was to collect expert views about the relative importance of the factors being considered for inclusion in TARS
Sample.
This consisted of experts on e-Health implementation issues - defined as the authors of published reviews of e-Health identified and included in the scoping review in WP1.
Recruitment.
We used a database containing details of the authors of the WP1 scoping review papers, especially email addresses. This resulted in a database of 203 potential respondents. It soon became apparent that a larger pool of potential respondents was required to achieve an adequate response rate (see Figure 4 below). A second database of authors was compiled through a bibliographic database search (ISI Web of Science and Medline) for papers published in the last five years that contained the word ‘review’ in the title and specified at least one of four keywords: telemedicine; tele-Health; telecare or e-Health. Records were scanned manually to ensure inclusion was based on relevance to the objective of the survey (i.e. a review paper, in the topic area, and addressing implementation issues). This process resulted in an Endnote database of 126 references, which was reduced to 116 potential respondents after removing duplicates with the initial sample database. Email addresses could be located for 105 authors in this second database, resulting in a total pool of 308 potential participants.
The approach to recruitment differed between the two databases. Database 1 authors were emailed an invitation to take part in the study, which contained a personalised link for completing the survey (to enable response tracking for targeting reminders). This led to many unused survey links due
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 80
to invalid email addresses, so Database 2 authors were first sent a personalised email inviting them to ‘opt in’. To these we sent a personalised link to the survey, as we had done for Database 1 authors.
Reminders
For Database 1 authors, targeted reminders (i.e. to non-responders) were issued twice, at approximately 10 day intervals. For Database 2 authors, those who had not responded two weeks after receiving a link were sent a reminder, specifying a closing date for the survey that was one week later.
Data collection
Participants were asked to rate the importance of each item to the routine use of e-Health, using a scale in which 0=not at all important; 1=some importance; 2=moderate importance; 3=very important; 4=extremely important; with the option of choosing 'don't know'.
The survey also invited respondents to suggest further factors that they felt were not covered in the questionnaire, using a free-text box. Online data was captured automatically from the website, in excel format, and converted to formatting for analysis with SPSS.
Analysis
Analysis aimed to assess the relative importance of items, and possible redundancy between items within the set. Quantitative and qualitative data analysis was undertaken. Quantitative analysis was descriptive, examining frequency distributions, means and standard deviations (means were calculated to exclude ‘don’t know’ responses) and inspection of correlation matrices (based on Spearman Rank Order Correlation co-efficient analysis). Although the data collected are categorical and not normally distributed, means were used to give an indication of relative importance for decision-making purposes (explained in Phase 2).
Qualitative analysis was conducted on free-text comments about participants’ perceptions of ‘omissions’ in the coverage of the items in the survey. Using thematic analysis, these comments were analysed in relation to (i) the constructs of the NPM, and (ii) the questions that were already included in the survey.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 81
5.4 Results
Figure 4 shows the pattern of response to the survey of experts conducted in Phase 1:
Figure 4. Response to survey of experts
SAMPLE 1
SAMPLE 2
203 emailed 105 emailed
26 Undelivered
177 presumed received
75 presumed received
30 Undelivered
32 agreed in principle & sent link (43% of
delivered links)
8 Explicitly Declined
Completed = 42
(24% of
delivered links)
Completed = 2
(Sampling
phase
unknown)1
Completed = 19 (59% of those who
agreed; 25% of
those receiving
initial invitation)
Total completed = 63 (24% of those receiving invitations)
5.4.1 Description of sample
The characteristics of the sample, in terms of location of residence, research background, sex, and distribution of own work across the four domains of e-Health are presented in Appendix 11. This data shows that location of residence is relatively consistent with country of origin of review papers in the field. It also reveals that the authors had diverse backgrounds and broad expertise in e-Health research.
1 This was due to a temporary technical glitch on the survey site whereby the participants’ email address was not captured and thus their sample origin couldn’t be determined.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 82
5.4.2 Data analysis/results
Quantitative data.
Descriptive data for the set of 27 factor rating questions are presented in Appendix 12. The questions are presented in descending order from the highest mean rating values to the lowest. Across rating items, the frequencies reveal that, although skewed towards the high importance end of the scale, there was some spread of scores across the full range of response options. Although data is not normally distributed and thus not ideally treated as continuous data, the magnitude of mean ratings on individual items indicate the experts’ perceptions of the relative importance of these factors to the routine use of e-Health. For example, item 3 ‘impact of the system on existing ways of working’ received the highest rating, followed closely by ‘ease of using the system’. The contextual integration (CI) items (items 1-4) were all rated in the top 7 positions (rating from highest to lowest). The bottom positions (lowest ratings) appeared to be dominated by relational integration (RI) items, though this may in part be due to the relatively higher number of items representing the construct as well as the more detailed nature of the items. Interactional workability (IW) and skill-set workability (SW) items tended to fall in the middle to top end of the ratings table.
Correlations between items were low to moderate, indicating little redundancy. Correlations of r>0.5 were considered when deciding whether to exclude or combine items in TARS in Phase 2. Correlations exceeding r=0.05 tended to occur within (rather than outside) sets of items representing the NPM constructs of contextual integration (CI), skill-set workability (SW), relational integration (RI) and interactional workability (IW) with the exception of some of the RI items (eg ‘availability of users’ knowledge of clinical effectiveness of the system’ and ‘availability of technical expertise’) correlating moderately with items in the IW set2 . Although subject to sample size limitations and lack of statistical power, this pattern of relationships provided some preliminary support for the face validity of the items with reference to the constructs of the NPM, as items within construct sets tended to correlate more highly with each other than with items outside their own construct set.
Qualitative data.
Appendix 13 presents conceptual analysis of factors underlying the free-text comments made by (n=31) participants. In general, ‘additional’ factors stated by respondents were usually more specific factor statements than the questions included in the survey and as such were considered to be represented. The exception was for the construct of ‘contextual integration’, as the existing items in the survey did not appear to adequately capture the breadth and specification of factors suggested by participants that could be classified as relating to this construct. This information was used in Phase 2 to create additional items for inclusion in TARS.
5.4.3 Key messages from Phase 1
Phase 1 represented a key stage in the initial development and validation of the factors to be included in TARS. The results (i) confirmed the importance of items within the set that we had developed, as perceived by experts within the field of e-Health; (ii) suggested that these items were
2 Selected correlations are reported in Phase 2 where they were used for decision-making about items.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 83
conceptually valid with respect to the NPM; and (iii) led to the identification of other factors to include in the TARS that were perceived, by experts in the field, to be missing from the existing set.
5.5 Phase 2: Development of TARS
5.5.1 Aim of phase 2
The second phase aimed to:
(i) refine the set of factor items relevant for inclusion in TARS based on the results from Phase 1 (TARS Generic); and
(ii) test the generic TARS items (and the mode of delivery) with health care professionals.
5.6 Developing TARS Generic
5.6.1 Methods
Data collected in Phase 1 was used to make decisions about excluding or combining existing factor items. This process began by examining each question in terms of (i) the mean rating of importance for that item, and (ii) any correlations between the item and other items in the set (Appendix14). Mean ratings along with matrices of inter-item correlations (correlations of r>0.5) were explored in making such decisions. Items that were highly correlated with other items were either discarded or re-written into a single item, particularly where importance ratings were relatively low. This process reduced the 27 items to 21, which were circulated amongst the research team for comment.
This peer review resulted in further revisions to the item set including: (i) three new items to reflect further development of the NPM since the study commenced (Q.29-31); (ii) refining the wording of items to improve clarity (e.g. Q. 16, ‘obtainability’ of new skills became ‘learnability’); and (iii) distinction of items into separate components (e.g. ‘time spent with patients versus quality of interaction’). Some of these changes reflected data emerging from the perspectives of professionals participating in WP2 at the time, such as emphasis on patient benefit and issues of liability (thus the liability item was retained rather than omitted for TARS Generic).
In addition, the qualitative analysis in Phase 1 indicated some omissions from the factor list that participants had rated, particularly with respect to CI items. Six new items (Q.5-10 in Appendix 15) were added to represent contextual integration factors which the expert survey panel considered to be missing from the existing set of items. TARS Generic is presented as Appendix 16.
5.6.2 Results
These processes resulted in a final set of 31 generic TARS items. As a final check on the appropriateness and coverage of the 31 items, the full set was mapped against the major themes emerging from preliminary analysis of the qualitative data collection undertaken as part of WP2 (see Appendix 15, which also displays TARS Specific items as developed in Phase 3). This shows a good level of representation of key factors across the two work
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 84
packages. It is particularly noteworthy that the 10 questions added to TARS in the final stage represented themes that were emerging from WP2, which could have otherwise been considered significant omissions from the Generic TARS.
5.6.3 Testing TARS Generic with health care professionals
The second stage of Phase 2 development work involved testing the generic TARS items with health care professionals and testing the proposed mode of delivering the questionnaire. Before developing the TARS further, it was important to test the importance ratings approach conducted with the expert sample in Phase 1, with a sample of NHS staff including health professionals and administrative staff. This was not about assessing “readiness” to adopt new e-Health technologies but to ensure that the items selected for the generic TARS were appropriate to users of e-Health systems.
Sample
A study site was chosen which met these criteria: (i) use of the full range of e-Health technologies that were of interest to this study; (ii) use of such technologies for a sufficient time to provide users with experience of implementation; and (iii) a potentially large pool of participants for the survey. The sample chosen for testing TARS Generic was thus a regional NHS Hospitals Trust which met these criteria.
Recruitment
Participants were recruited through the key contact at the study site. Invitation to participate in the survey was made by emailing an invitation to the study that included a link to the survey website. All staff on the distribution list for the Trust were emailed by the site contact.
Reminders
Issuing of reminders depended on the site contact. A request for a reminder was made 10 days after the initial email invitation. Although the site contact agreed and stated that staff would be encouraged to respond, despite several follow-up emails, this reminder was not issued.
Data Collection
The survey was conducted online using a commercial survey site (surveymonkey.com). The survey itself contained the TARS Generic items, along with background and demographic questions developed in conjunction with input from the site contact to ensure the relevance of the demographic questions. In addition, the site contact suggested an additional TARS item for inclusion, and advised on the framing of introductory text for the survey and for the invitation email.
As for the expert survey in Phase 1, participants were asked to rate each item on importance using a five-point response scale. Participants were asked to indicate which category of e-Health (from the four domains specified in this study) they were thinking of when answering the questions, or whether they intended to respond about e-Health more generally.
Online data was captured automatically from the website, in excel format, and converted to formatting for analysis with the SPSS statistical package.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 85
Analysis
Descriptive analysis (of frequency distributions, means and standard deviations) was undertaken. Correlations (based on Spearman Rank Order Correlation co-efficient analysis) were explored for relationships amongst the TARS items, but not reported due to sampling limitations.
5.6.4 Results TARS Generic
Response rate and sample characteristics.
The response was disappointingly low. Overall, we achieved 51 responses. Sample characteristics are presented in Table 6.
Table 6 Characteristics of TARS Generic sample
Age groups: % (n)
<25 0 (0)
25-34 18 (9)
35-44 26 (13)
45-54 33 (17)
55+ 23 (12)
Working role:
Administrative 41 (21)
Allied health professional 28 (14)
Nursing 24 (12)
Hospital based consultant 8 (4)
Sex
Male 35 (18)
Female 65 (33)
TARS items
Means, standard deviations and response frequencies for the 32 TARS items used in the TARS Generic survey are presented in Appendix 17, ordered by highest to lowest mean rating. The distribution of the responses indicates ceiling effects, in that most items have been rated by most participants as highly important. Interpretation of the meaning of this data is limited by the low response rate. However, the pattern of ranking of importance appears to differ from that of the survey of experts, in that issues of workability (rather than integration) appear higher in the rankings made by professionals. This reflected a similar distinction apparent between the key findings of WP1 (Scoping review) that also reflected emphasis on Contextual Integration issues compared with what the health professionals participating in interviews and workshops were reporting from WP2.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 86
5.6.5 Key messages from Phase 2
Phase 2 developed and tested (with a sample of health professionals) a generic version of TARS. This was a necessary step towards ensuring that TARS would have comprehensive coverage of items relevant to end users of e-Health. Questions could then be developed for the rating of particular e-Health systems by health professionals (in Phase 3), rather than by experts as examined in Phase 1. Although limited by the poor response, the data confirmed that the factor items included in TARS Generic were considered to be of sufficient importance as factors affecting the routine use of e-Health for inclusion at sites where a specific e-Health system was to be assessed. The TARS Generic survey also served as a pilot for online data collection. The poor response indicated that although the online survey is potentially a useful and efficient means of collecting quantitative data, a much more targeted approach with specific user groups and more effective means of encouraging response rates is necessary to use this method successfully.
5.7 Phase 3: Testing and Validation of TARS Specific in study sites
5.7.1 Aim of Phase 3
This section reports the third and final phase of the work package, which involved developing and testing site-specific versions of the TARS (TARS Specific) through data collection at two study sites. In contrast to Phase 2, the aim of this Phase was to test the utility of TARS as framed specifically in relation to a particular e-Health system, amongst specific samples of health professional users of the system.
5.7.2 Methods
Site selection
Phase 3 undertook data collection in two of the NHS contexts sampled in WP2. These sites were chosen as (i) specific e-Health systems were in use by health professionals, that could be the subject of assessment using TARS, (ii) the two sites differed in terms of levels of ‘normalisation’ of e-Health technology thus enabling comparisons of the utility of the TARS instrument being developed; and (iii) access to the study sites had been agreed as part of the project.
At Site 1, use of the e-Health system (community nurses using PDA technology) was relatively new, and provided an opportunity to use TARS in a context where e-Health was still in the experimental stages for some users. At Site 2, the entire organisation is based on e-Health systems – so staff could be expected to have a relatively high experience of e-Health systems.
Development of TARS Specific for study sites
The TARS Generic items from Phase 2 were changed into specific questions. To minimise possible ceiling effects as evident in Phase 2 data collection, TARS Specific used a 7 point response scale representing level of agreement. The two specific versions differed in terms of introductory and explanatory text, and categorisation of work roles for the demographic items, as appropriate for the two sites being sampled. In addition, Q.10
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 87
from the full set, about commissioning of services, was irrelevant to site 2, so was omitted. As for TARS Generic, the survey included an item assessing ‘comfort with using computer based technology’. For TARS Specific, two additional questions were included to assess: (i) participants’ perceptions about whether the system was not at all, partly, or completely in routine use; and (ii) their perceptions about the likelihood of it becoming routine (on a 5 point scale). The two TARS Specific surveys are included as Appendices 18 and 19.
Recruitment
As for Phase 2, recruitment of staff into the survey was facilitated by key site contacts who negotiated the necessary permissions and approvals within their organisations to email the participation invitation (containing ethics approved participant information sheet and link to survey) to staff email lists. On approval, site contacts emailed the invitation to staff members within their organisations.
Reminders
As before, it was not possible to send reminders to non-responders as we did not have their email addresses (as we did not have ethical approval). It was thus necessary for site contacts to issue reminders upon request. At both Phase 3 sites, requests for reminders to staff were requested at one week intervals, however in practice these intervals were longer (10 – 14 days). At Phase 3 sites, two reminders were issued following the original invitation, which increased response rates.
Data collection
As in Phase 2, the two surveys for Phase 3 were administered using a commercial online survey service provider (www.surveymonkey.com). Online data was captured automatically from the website in Excel format, and converted for analysis with SPSS.
Analysis
Descriptive analysis (of frequency distributions, means and standard deviations) was undertaken. Correlations (based on Spearman Rank Order Correlation co-efficient analysis) were explored for relationships amongst the TARS items in each sample. Sub-group analyses were conducted using Cross-tab analysis with Pearson’s Chi Square statistic for comparisons of TARS items ratings according to (i) level of perceived integration of the e-Health system into routine practice (both sites); and (ii) professional grouping (Site 2 only). For cross-tab analysis at Site 1, responses to the TARS items were dichotomised into groups indicating non-agreement (responding 0 strongly disagree -3 neutral midpoint) and those responding with various levels of agreement (rating 4-6). At Site 2 (with a larger sample size and different spread of responses), TARS item responses were trichotomised as follows: Disagreement (0-2); neutral or some agreement (3 or 4); and moderate to strong agreement (5 or 6).
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 88
5.7.3 Results
Response rates and demographics
At Site 1, 46/243 nurses completed the survey (19% response rate). At Site 2, 231/1351 (17% response rate) completed the survey sufficiently for inclusion in the analysis. Sample characteristics for both sites are presented in Appendix 20.
TARS items
Means, standard deviations and response frequencies are presented in Appendix 21 for Site 1, and Appendix 22 for Site 2. Correlation matrices (based on inter-item correlations using Spearman’s Rank Order (RHO) analysis) were inspected, and for both sites, key findings based on correlations exceeding r=.06 are summarised in Appendix 23.
In relation to the frequency tables, the high number of ‘don’t know’ responses selected by participants particularly when rating contextual integration items such as ‘the e-Health system is adequately resourced financially’ (111/231 participants at Site 2) and ‘government policy initiatives are supportive of this e-Health system’ (106/229 Site 2 participants) is noteworthy. The data indicates that, in undertaking assessment of perceptions using TARS, closer consideration must be made concerning who particular questions are relevant for.
In terms of correlations between items, Appendix 23 shows that the items included in the TARS to assess perceptions relating to Interactional Workability appeared to relate quite strongly to each other, and consistently across study samples. In both sites, similar relationships amongst Skill-set Workability items were evident. Comparing these two sites on inter-item relationships within the Relational Integration item set, Site 1 generated a higher number of high correlations compared with Site 2, but included the key correlations evident at Site 2. In terms of Contextual Integration, again there appeared to be a higher number of high correlations within the data collected at Site 1. All three of the NPM items (q.28 ‘coherence’, q.29 ‘cognitive participation’ and q.30 ‘reflexive monitoring’) related moderately highly with each other. In each sample, some of these items related highly with other items in the full item set, though patterns differed between samples. Appendix 23 also indicates that some items within TARS were either consistently highly correlated or poorly correlated with other items across NPM construct categories. In both sites, for example, the item about efficient use of time seemed to relate highly across constructs (rather than just within RI), and the item about co-operation with others did not relate well with any items across the full item set.
Sub-group analyses
Cross-tab analyses are presented in Appendix 24 for Site 1, Appendix 25 and Appendix 26 for Site 2. For Site 1, due to the relatively small sample, frequencies from within the group (rather than percentages) are reported.
For Site 1, Appendix 26 reveals significant differences between groups perceiving e-Health as ‘partly routine’ compared with ‘completely routine’ in response to 12 out of the 30 items. For the majority of these items, the pattern of relationship is such that those who perceived the e-Health system to be completely a routine part of their work were more likely to agree than not agree with the statements about the system, or to show a higher
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 89
proportion within the group responding with agreement (ie overall, they indicated more positive responses). Here, the strongest significant differences occurred on two of the CI items ‘this organization has a culture that is supportive of change’ and ‘this e-Health system fits in with the priorities and challenges of our organization’, along with the coherence item ‘the staff who work here have a shared understanding of what the system is for and how it is to be used’.
For Site 2, analysis of differences between professional groups (Appendix 25) shows that compared with call handlers, nursing and health related staff were more likely to perceive the e-Health system as making them feel autonomous in their work; and were more likely to agree that they understood their own liability for practice. Health professionals were less likely than call handlers to feel the skills required by the systems they used were easily learned; or to agree that using the e-Health system involved spending the right amount of time with patients. Such differences were in directions that would be expected, based on their working roles.
At Site 2, analysis of TARS responses between participants perceiving e-Health to have become ‘completely’ routine in their work (n=174), with those indicating it had either not at all or only partly become routine (n=37) revealed significant differences on nine items (significant differences only are reported). These results (Appendix 26) suggest that compared with those who feel that e-Health has already become ‘completely routine’, those for whom it hasn’t become routine were less likely to agree that sufficient organisational effort has gone into supporting the system; and less likely to show strong agreement (rather than being neutral or some agreement) that e-Health is a different way of working; that the organisational culture is supportive of change; that they understand their own accountability and liability; and that there are ongoing mechanisms for monitoring and appraising how e-Health is used. The group for whom e-Health was not yet a completely routine part of their practice were also more likely to disagree that there is good evidence of clinical effectiveness of the e-Health system, and that there is a shared understanding of what the system is for and how it is to be used. Here, the strongest differences between groups were evident on items relating to liability, accountability and appropriateness of skills.
Key messages from Phase 3
The development and testing of TARS Specific in two practice sites resulted in key findings that contribute to contextual validation of the TARS and its further development.
Representation of NPM constructs
In general, there was a reasonable level of face validity of the TARS items, and support for their representation of the four constructs of the NPM (as suggested by patterns of correlations between items). Further exploration of these items as representing the NPM (in its original and extended form) in further development and testing of the model in data samples of sufficient size for the application of factor analytic techniques, is thus supported.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 90
Relating TARS items to perceptions of level of routinisation
The findings of Phase 3 suggest some evidence of discriminant validity for the items in the TARS survey instrument, in that groups representing different levels of perceived routinisation of the e-Health systems differed in their ratings of some of the items (in the direction that would be predicted by the NPM). This is particularly promising, given that the sites surveyed had already been using e-Health for some time (particularly at Site 2), and that this would have limited the potential for variability in responses concerning whether or not e-Health was considered ‘routine’. Although the data is suggestive only of association rather than causality, this supports the further development and use of TARS in prospective studies where the predictive value of TARS items can be assessed.
Multi-perspective assessment within sites
The findings of Phase 3 suggest that the use of TARS should be sensitive to differences between professional groups. At Site 2, differences between professional groups in the TARS ratings were few, but in directions that would be expected. More relevantly however, the data showed that several items (mainly relating to contextual integration) were unanswerable for significant proportions of participants. This raises questions about how to incorporate different staff groups’ perspectives about a referent e-Health system within an overall assessment of readiness for e-Health.
5.8 Discussion
The primary objective of this work package was to develop an instrument for assessing the readiness of NHS settings for the uptake of e-Health technology, based on the perspective of health professionals and related staff. In developing the TARS instrument, we have drawn on several sources of relevant information concerning factors that affect the take-up of e-Health. In Phase 1, we translated the constructs of the NPM and literature findings into factor items representing issues known to affect the normalisation of e-Health and obtained data on the relative importance of these factors from ‘experts’ in the field. In Phase 2, we developed TARS Generic and surveyed health professionals to elicit importance ratings in order to develop TARS Specific for use in specific e-Health use contacts. Finally, in Phase 3, we translated TARS Generic into TARS Specific and tested it in two NHS sites, which were using different e-Health systems. This section discusses the main findings, strengths and limitations, implications, and recommendations for future work.
5.8.1 Main findings
Turning the NPM into factors
This work package has contributed to the development of the NPM in a number of ways. Firstly, it has successfully achieved the development of a set of quantitative questions that can be used to assess staff perceptions relating to different underlying aspects of the constructs within the NPM, along with pre-testing of single items that may be used to represent quantitative assessment of the constructs of coherence, cognitive participation, and reflexive monitoring as proposed in the extended version of the NPM (NPT) (discussed more fully in Section 7). The data collected in Phase 3 of this work package has confirmed a reasonable level of face
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 91
validity of these items as representing the constructs within the model (discussed below).
Assessment of the relative importance of factors affecting normalisation of e-Health
This work package generated information about the perceived relative importance of a comprehensive set of factors known to affect the normalisation of e-Health. The survey of experts in Phase 1 was particularly instrumental in providing face validation of the factors to be represented in the TARS instrument, as well as identifying what may have been missing so that further items could be developed for inclusion in TARS. The results of Phase 1 however, had the added benefit of providing a form of cross-validation between work packages in this project. The findings here were consistent with those of the literature reviews conducted in WP1, in that the experts participating in the survey also emphasised issues of contextual integration over practice-based issues of workability, compared with WP2 which found that health professionals considered workability issues to be more important.
Relationships between TARS items and constructs
The key findings from Phase 3 (Development and Testing of TARS Specific) offered support for the TARS items as reflecting the constructs of the NPM. From the patterns of correlations between items that were evident in both study sites, the relationships between items within the constructs of Interactional Workability, Relational Integration, Skill-set Workability and Contextual Integration were generally of the magnitude and direction that would be expected on the basis of the NPM. The support for the model in this respect warrants further work on factor analysis of items in studies where sample sizes are sufficient for such analysis.
This work package also provided a preliminary test of the relationship between responses on TARS items and perceptions of whether or not the e-Health systems in the respective study sites had become part of routine practice. Despite the collection of data in study sites in which e-Health had been ongoing for some time, several TARS items did significantly differentiate between groups perceiving different degrees of normalisation of the e-Health system they were rating. Although the data can only be treated as representing associations (rather than causality), it lends support to the assessment of the predictive value of TARS in prospective, longitudinal studies that provide an opportunity to assess staff perceptions of a new technology at the initial stages of its introduction to and use within a workplace.
Production of generic and specific versions of TARS
This work package has produced two versions of TARS: (i) TARS Generic, which is presented in the form of factor statements for the elicitation of importance ratings; and (ii) TARS Specific, which presents questions on which professional users of e-Health can rate a specific system based on their experience and their perceptions of the context in which they work. TARS Generic was developed as part of a process towards designing a useable instrument for assessing staff perceptions about a particular e-Health system they are using, rather than as an instrument for use in its own right.
However, the process of developing and using both generic and specific versions of TARS has raised both challenges and opportunities. It has been
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 92
intended that TARS Specific could be used to assess staff perceptions of a particular e-Health system, based on some experience of or familiarity with the system. Even with this objective in mind, for use in other clinical contexts, it is likely that the specific framing of the questions within TARS Specific will need to be adapted to appropriately reflect the stage of implementation/use of the e-Health system. For example, if the questions are to target perceptions in relation to a technology not yet used, then they will need to be framed (and interpreted) as the assessment of expectations of the technology. Another possible use of TARS Specific may be to compare perceptions of a new technology with an existing technology in used – in which case questions would need to be framed as including a comparison. Even in contexts where TARS Specific is framed appropriately for use in its current form, the inclusion or omission of certain questions will need to be considered on the basis of relevance to specific staff groups that are the intended recipients of the survey. These issues present challenges for further validation of TARS Specific as ‘an instrument’, but offers a range of opportunities for practical use in assessing staff perceptions of factors that this study has shown to be important for the normalisation of e-Health.
TARS Generic has not been developed with the intention of applied use in the kinds of ways suggested above, however it too may have practice and research-based uses. For example, assessments of staff perceptions of the relative importance of the factors included in TARS Generic may be useful in certain situations, such as collecting information about issues of importance during planning stages for e-Health implementations, and for comparisons of these assessments between different professional groups within an organisation.
5.8.2 Methodological issues
This work package has undertaken considerable instrument development and testing activity with respect to the assessment of NHS staff members’ perceptions of factors related to the normalisation of e-Health technologies in practice. However, several methodological limitations must be considered.
Response rates
In terms of scale development, response rates and sample sizes at survey sites were insufficient to permit the use of statistical scaling methods that would be necessary for formally testing the statistical properties of TARS as a research scale. Indeed, it was anticipated that statistical testing of TARS at this level would not be achievable within the scope of this particular study, and that this would be required in subsequent research using TARS.
Survey approach
The results of this study show that, although online survey research has the potential to achieve the collection of survey data from significant numbers of participants efficiently, several conditions are necessary to achieve this. The role of key facilitators at survey sites is essential, particularly when they are responsible for identifying appropriate staff groups and actively encouraging staff participation. Although all site contacts who facilitated this work package were helpful and encouraging, in the context of their busy working practices, the frequency and timing of reminders to non-participants were compromised by lack of control over this by the research team.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 93
Site selection
The survey sites included in this study already had at least some level of adoption of e-Health technology. Although not an objective of this study, the sites included did not allow for prospective analysis of the predictive utility of either the NPM constructs, or the TARS items in particular in relation to the likelihood of normalisation of e-Health. Further research using TARS and the NPM should therefore be undertaken in longitudinal and prospective research studies conducted within practice contexts where a new technology is about to be introduced.
5.8.3 Comparison of findings with existing literature
The key findings of this work package are generally consistent with the empirical literature (reviewed as part of WP1) in terms of providing support for the importance of a comprehensive range of factors and issues known to affect the uptake and routine use of e-Health. In terms of the existing literature on technology readiness that has been conducted in relation to tele-Health (as one particular kind of e-Health within the taxonomy developed for this study), the themes emergent from that literature are generally compatible with those represented in the items contained within TARS. However, as such studies have not yet tested the importance or relevance of such themes in the form of quantitative survey items, it is difficult (if at all possible) to compare the findings of this study against existing research in this field.
In terms of the theoretical literature, the concept of normalisation as framed by the NPM can be interpreted as referring to both a ‘state’ (in that something is ‘normalised’) and to a process that is dynamic and ongoing. This does raise the question of what ‘readiness’ actually means in the context of technology adoption, and in particular, if measuring ‘the degree to which a community is prepared to participate and succeed in tele-Health…’ (Jennett et al 2003a) is the objective, then timing of assessment becomes a crucial question. In this study, we explored the concept of readiness as proposed by Jennett (Jennett et al. 2003a) in settings in which e-Health had already been in use by staff. It would be necessary to further explore responses to the items included in TARS, in contexts where ‘readiness’ of this kind can be assessed by using it in studies where e-Health implementations are about to be conducted (rather than already underway). However, the finding that TARS items did differentiate between groups perceiving different levels of normalisation of a technology even in contexts where use of e-Health had been ongoing for sometime suggests that a broader view of ‘readiness’ should be taken. We would propose that such assessments of likelihood of success in achieving routine use of a new technology such as e-Health in a practice context are more appropriately focused on determining ‘receptiveness’ rather than ‘readiness’ as has been the focus on previous literature concerning tele-Health, and that the role of factors affecting staff perceptions and behaviours in relation to e-Health should be considered as dynamic and changing across time.
5.8.4 Recommendations for further development & application of TARS
1. Further development of TARS for practical use should involve:
� Specification of different ways of wording TARS items for different purposes, and development of guidance of its use for different purposes and in different contexts; and
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 94
� Further research and analysis concerning the appropriateness of (i) different items within TARS alongside (ii) alternative assessment approaches, for assessing and combining the perceptions of different professional groups whose work is affected by the introduction of an e-Health technology
2. Further research concerning TARS and the NPM should involve:
� Testing the statistical properties of TARS as a research scale, in contexts where the necessary requirements concerning sample sizes and response rates can be met; and
� Testing the predictive utility of TARS in relation to normalisation of e-Health technology, by using TARS in longitudinal prospective studies in contexts where assessment of perceptions can be undertaken prior to the introduction of an e-Health system.
5.9 WP3 Outcomes
This work package has achieved key outcomes in relation to understanding the implementation and integration of e-Health in the NHS:
� An instrument (TARS) for assessing staff perceptions relating to receptiveness of e-Health technology within their working contexts that can be further developed and tested in subsequent research;
� Data concerning the relative importance of a range of factors that contribute to staff perceptions about workability and integration potential, from the perspective of staff members themselves;
� Preliminary quantitative testing of aspects of the NPM that contribute to further development of the model (and the subsequent NPT);
� Recommendations for further development and testing of TARS for use in practice contexts; and
� General guidance for undertaking assessments of staff perceptions about the likely normalisation of e-Health
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 95
6 Work-Package 4 (WP4) The implementation of e-Health systems: perspectives of e-Health implementers on the integration of new technologies into everyday work.
6.1 Background
Work-package 4 (WP4) focused on implementers, aiming to determine their views on barriers and facilitators to implementation of e-Health technologies, and to develop a toolkit to aid future implementations.
Implementers, defined here as any person who has been charged with an e-Health system implementation, have important insights into factors that lead to implementations being successful or otherwise. At the outset of this project, it was our view that their perspective had been relatively less studied than the views of health professionals, and this view was confirmed by the literature review undertaken in WP1. Hence the data collected in WP4 form a unique and valuable addition to the total literature on implementation of e-Health systems. Some of our interviewees were health professionals, but they were selected for interview on the basis of their role as implementers.
This work-package had two components:
Phase 1 - an exploratory phase to collect primary qualitative data on implementers’ perspectives; and Phase 2 where the primary data collected in WP1 (the scoping exercise) and the qualitative data from this work-package were examined and integrated, using the NPM as the underpinning theoretical framework to develop an implementation toolkit (e-HIT)
6.2 Phase 1 Aims and Objectives
The aim of this first phase was to identify, describe and understand core mechanisms in the implementation of e-Health systems, from the perspective of implementers, using the NPM as an analytical framework.
Definitions
The four work packages used common definitions, as presented and discussed in Section 1.3. As in the other work-packages we have classified e-Health interventions into the 4 domains introduced in Section 1.3 Box 1.
Implementers
In this study, an implementer is any person who has been charged with assisting with an e-Health system implementation. Depending on the policy
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 96
level sponsor, implementers may be found at national, regional and/or local levels and may include health service “supremos”, chief executives, clinical directors, senior healthcare managers, ICT staff, health professionals, local NHS managers, staff involved in training, and staff working for private companies contracted to supply, facilitate or support technology implementations. Although our focus was not on health professionals, some health professionals with a lead role in an e-Health implementation were interviewed.
In common with the other work packages the Normalization Process Model (NPM) provided the theoretical framework for WP4 data collection and analysis.
6.3 Phase 1 Methods
Design
A qualitative study, using interviews with a range of implementers drawn from three case studies.
Setting and sample
Identification of case studies
We identified three criteria to guide selection of case studies, with the goal of maximising the transferability of the results by achieving a maximum variability sample. The three criteria were:
1 The technologies in the case studies should include a range of e-Health domains (i.e. management, communication, decision support and information);
2 The case studies should include a range of clinical contexts (i.e. primary, secondary and community care);
3 The case studies should exemplify a range of sponsors of the implementation, as the first version of the NPM identified sponsorship as an important variable.
Finally, we limited our case studies to sites where the implementation occurred after 2004 but before 2006 to ensure that the implementation was both recent enough to remain alive in respondents’ memories, and old enough for it to be apparent whether the system implemented was becoming normalized into routine care or not. A fully normalized intervention is one that is taken for granted and has become completely embedded into routine practices.
Having determined these criteria for selecting three case studies, we undertook a three stage data collection exercise to identify appropriate case studies.
a) We held a brainstorming session with the entire EH94 project group to generate a list of potential e-Health initiatives and a list of people to interview to learn more about each initiative;
b) We searched the Department of Health, Connecting for Health and other suitable websites for information about national and local e-Health initiatives;
c) We interviewed key national informants for their views on which e-Health initiatives should be sampled and why.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 97
Data derived from this three-stage process were combined and discussed at a steering group meeting of all EH94 co-investigators. The decision was made in two stages: first we agreed on the technologies we would study, and secondly we agreed on a geographical location for each technology. Thus each of the final case studies was described in terms of a technology and a location.
Identification of interviewees within each case study
Within each case study, we identified key informants. Once again, we adopted a purposive sampling strategy to obtain maximum variety. We aimed to include implementers from three levels:
1. Senior Department of Health or Connecting for Health staff with responsibility for a number of projects in multiple organisations;
2. Senior staff from within the Trust or Health Board of the case study with lead responsibility for implementing either this particular system or a group of e-Health initiatives (e.g. Chief Executive Officers, Departmental Leads or senior staff from the Local Service Provider (LSP);
3. Middle management, including IT leads, training leads, and others with day-to-day responsibility for the implementation under study.
Data Collection
We collected data through face-to-face or telephone interviews with staff identified through our purposive sampling framework. Semi-structured interviews were used to determine not only “what happened”, but also the implementers’ perceptions of “why it happened”. The content of the interview was informed by the normalization process model, and included.
� A description of the e-Health implementation process the interviewee was involved in.
� The context of the implementation process e.g. in which healthcare sector/s was the technology located? Who was the policy level sponsor of the initiative? Which staff groups were expected to use the new technology?
� Interviewee perceptions of those factors which had promoted, inhibited, or had little impact on, the integration of new technologies into every day work.
� Interviewee perceptions of the degree of normalization achieved (the extent to which it has become embedded in routine clinical practice), and the extent to which this varied between different groups of users.
� The extent to which the various professional groups targeted by the e-Health implementation reported changes to the stability and order of the clinical encounter (interactional workability) and implementers’ awareness of the nature of these changes e.g. what mechanisms existed to report the impact of the intervention on doctor-patient interactions?, what changes were reported within the clinical encounter?, and implementers’ views on the reasons for these changes.
� The extent to which the professional groups targeted by the implementation process reported changes in trust in professional knowledge and practice within networks (relational
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 98
integration) e.g. what was the effect of the initiative on professional confidence in clinical practice? How did implementers gain an awareness of these impacts? Why did they think this had happened?
� Identification of the impact of the e-Health implementation on the division of labour within the healthcare organization e.g. how were tasks relating to the e-Health initiative assigned to and agreed by particular staff groupings (skill set workability)? What was the impact upon the performance (competency, quality, autonomy) of those staff in subsequently carrying out this work?
� The extent to which new procedures and systems were used to integrate the e-Health service into the work of the organisation and the impact of these on existing patterns of activity (contextual integration) e.g. What, if any resource reallocations or system adaptations took place? How did these affect service delivery?
The interview schedule for Case Study 1 is shown in Appendix 27. The interviewer also kept contemporaneous field notes which provided additional information.
Data analysis
All interviews were tape-recorded and transcribed verbatim. Interviews were coded according to the main constructs of the NPM. Additionally we searched the data for evidence of the degree to which each system studied had normalized, and for data which could not be coded using the NPM.
Initial data were coded by the interviewer (Jo Burns, JB) and the lead investigator on this Work Package (Elizabeth Murray, EM). The coding framework devised was then tested and refined at a 2 day multi-disciplinary Data Analysis Clinic (FM, CM, TF, GA, KOD, EM, JB). The revised coding framework was re-applied to the previously coded interviews and all subsequent interviews. After the change of research staff toward the end of Case Study 2 EM coded all interviews. At the end of this process, we undertook a final quality check as CM independently coded all data.
6.4 Results
6.4.1 Identification of Case Studies and Interviewees.
Identification of case studies
Ten national figures in e-Health and Connecting for Health (CfH) were contacted for their advice. Five agreed to be interviewed, and provided information on the e-Health initiatives they considered to be of national importance, together with information on the relative progress made by a range of Trusts and Clusters across England.
The three case studies finally selected were:
1 Choose and Book (C & B) in an English early adopter site made up of one hospital and the lead PCT providing referrals to that hospital - Case Study 1 (CS 1);
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 99
2 Picture Archiving and Communication System (PACS) in one acute hospital trust. The trust included several hospitals at different sites, and was located in England – Case Study 2 (CS 2);
3 Clinical Nursing Information System (CNIS) designed for District Nurses working across 10 Community Health Care Partnerships within the largest Health Board in Scotland – Case Study 3 (CS 3).
The salient features of the three Case Studies are presented in Appendix 28. We achieved good coverage across our three criteria, with CS 1 involving primary and secondary care as well as the primary-secondary care interface, CS 2 located in secondary care, and CS 3 located in the community. CS 1 and 2 both covered the e-Health domains of communication and management, while CS 3 covered information and decision support. Finally, in CS 1, Choose and Book had been implemented prior to the establishment of CfH and had strong local sponsorship, CS 2 was a CfH priority, and CS 3 was in Scotland and out with CfH responsibility.
The relationship between the case studies selected and the sampling criteria are shown in Table 7.
Table 7 Relationship of Case Studies to Selection Criteria.
Context e-Health Domain
Management Communication Decision Support
Information
Primary Care C & B C &B C & B
Secondary Care C & B
PACS
C & B
PACS
Community Care CNIS CNIS CNIS
Primary/secondary care interface
C & B
Our third criterion was the level of policy sponsor: C & B and PACS both had a national level policy sponsor (CfH), while the CNIS had a local level policy sponsor.
Identification of interviewees
Within each case study, interviewees were selected according to the criteria described above. Ten interviews were carried out in Case Study 1, 5 in Case Study 2 and 8 in Case Study 3. We had pre-set a target of ten interviews per case study, but in both CS 2 and CS 3 saturation was achieved relatively early and, as no new data were being generated in subsequent interviews, we decided to halt the interviewing process. Table 8 presents details of the roles of the interviewees which included regional leads for the Cluster (CS 2) or Local Service Provider (CS 1), Chief Executives for the Trust or Health Board for all three Case Studies, Clinical and or IT leads, and a range of middle management with “on the ground” responsibilities. It can be seen that the intended sampling framework was achieved.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 100
Table 8 Roles of interviewees
Case Study Choose and Book
PACS CNIS
Regional Level Lead for Local Service Provider
Regional Implementation Director for Cluster
Chief Executive CEO of Trust CEO of Trust Managing Director of provider company; General Manager of Health Board
Senior Management
Clinical Lead for Hospital Trust
Clinical Lead for Hospital Trust
IT Manager Health Board; Clinical Services Manager
Middle Management or “on the ground”
GP and clinical lead in PCT; Consultant; Practice Manager; Project Manager for Hospital Trust; Outpatient Manager; Primary Care Director for Hospital Trust
Radiology Manager; IT Manager
Lead Project Nurse; IT training manager Health Board; Senior Nurses x 2
6.4.2 Findings from case studies
Case Study 1: Choose and Book
Choose and Book (C & B) is a national electronic service that provides patients with the opportunity to choose which hospital their GP refers them to for a particular problem, and to book the time and date of their first appointment. Full normalization of Choose and Book would require normalization both in primary care / general practice, and in the hospitals referred to. At the time of our study, Choose and Book had been variably implemented across England, and our study site was an early adopter.
Degree of normalization
It was clear that Choose and Book had normalized to quite different extents in primary and secondary care. It was very patchy in primary care, with the Clinical Lead of the hospital estimating that about 30% of total referrals were electronic with a minority of practices generating most of the electronic referrals, whereas in the hospital, almost all clinical specialties had implemented Choose and Book, and it was well integrated into everyday work in most.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 101
‘I think that 30% of our referrals are electronic. You will find that they are very skewed towards certain practices.’ (CS1 Clinical Lead)
‘I think it [integration] varies a little bit for specialties, specialties such as my own – it is now the majority referral methodology and so it is completely integrated…overall we yes happy with it in the hospital – it is working for us.’ (CS1 Clinical Lead).
‘We have got it in as mainstream business processes. My consultants are used to it ... it is becoming a way of life here…’ (CSI Chief Executive)
‘completely embedded in standard operational workings’ (CS1 Project Manager)
Contextual Integration
The hospital studied was very committed to Choose and Book as a means of improving their market share of referrals, at a time when the number of referrals from primary to secondary care was falling. The hospital could only survive financially if it could maintain a steady flow of inward referrals, and was in the challenging position of having a number of competing providers within a mile or two, including at least two teaching hospitals. Choose and Book became a central part of this hospital’s business plan, and was seen within the hospital as a very important tool to maintain inward referrals. This view had permeated throughout the senior management of the hospital, who invested considerable effort in making Choose and Book work. This translated into considerable managerial and financial support aimed at overcoming any and all difficulties encountered. In NPM terms, Choose and Book had very high levels of contextual integration from a hospital perspective.
In contrast, Choose and Book had relatively low contextual integration in general practice. At the point this study was undertaken there were no financial incentives for GPs to use Choose and Book, although there were some general exhortations to “offer choice” to patients at the point of referral (low contextual integration). As there was little requirement to use Choose and Book, relatively little effort was invested in making it work, and problems with using it went largely unaddressed. GP practices that adopted it hoped it would streamline the referral process, and reduce the amount of time spent chasing appointments in secondary care on behalf of patients. However, this potential advantage was offset by the amount of administrative time taken sorting out problems caused by Choose and Book.
‘I wanted to make it so easy to book an appointment in this hospital that people would start to use this hospital for booking’ (CS1 Hospital Chief Executive).
‘The potential competitive advantage that direct booking would have over my competitors in encouraging that small percentage of GP practices who are promiscuous referrers to refer into my hospital … use the business leverage …’ (CS1 Hospital Chief Executive)
‘the reason we went for it initially because we felt there would be real advantages to it and it would hopefully streamline the process of referring patients to hospital’ (CS1 GP Practice Manager)
‘… overall our secretary would say she’s got fewer people phoning saying I saw the doctor three weeks ago and I thought he’d made a referral but I still haven’t heard anything, I haven’t had an appointment yet….’ (CS1 GP Practice Manager)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 102
‘If they [the patient] then have a problem, it’s invariably the secretary they come back to. So she’s had to deal with all of those.’ (CS1 GP Practice Manager).
Interactional Workability
For GPs, the defining feature of Choose and Book was the slowness of the web-based programme. Even GPs who wanted to use it found that it took 3 – 4 minutes to make a referral with Choose and Book, which had a severe negative impact on consultations with patients. GPs perceived themselves as under constant time pressure, and resented this additional task being added to consultations, particularly as consultations which resulted in referral tended to be complex. In NPM terms, Choose and Book had very poor interactional workability, both in terms of congruence (GPs did not think that time spent on booking an appointment was a good use of consultation time), and in terms of disposal (GPs found that using Choose and Book delayed the outcome of the consultation). This negative impact on the consultation and poor interactional workability dominated GP reactions to Choose and Book.
‘Right you are saying within my 10 minute slot and you have said Choose and Book will take a couple of minutes – it doesn’t – what, even two and a half years on it takes at least four and is not even working properly today. So it took me 10 minutes to do one this morning. So you really think that I am going to stop to help people exercise that choice…?’ (CS1 GP Early Adopter and IT enthusiast)
‘And again for people to say that a Choose and Book takes two minutes – it just doesn’t – it is two minutes if you are really lucky, if the wind is in the right direction on the right day. Even if those 2 minutes but still 20% of the common consultation … 20% that wasn’t there before.’ (CS1 GP Early Adopter and IT enthusiast).
Choose and Book had relatively little impact on interactional workability for consultants, except when patients had been booked into inappropriate clinics, which tended to lead to poor consultations (negative impact on IW).
Relational Integration
Choose and Book had some negative impact on relational integration (trust and confidence in different professional groups) for both hospital specialists and general practitioners. Consultants were used to receiving referral letters, reading them, and allocating the referred patient to an appropriate clinic, often also allocating a degree of urgency (e.g. urgent, soon or routine). As Choose and Book referred patients directly into specified clinics, Consultants lost the opportunity to direct referrals appropriately, and had to rely on GPs making the right referral decisions. In turn, GPs were dependent on the information provided by Choose and Book, which was often inadequate to guide appropriate choice of a specific clinic. Choose and Book also mediated against referral to a specific consultant, and encouraged GPs to refer to the clinic with the shortest waiting time. This had a small negative impact on GPs trust and confidence in the specialist that the patient was referred to – GPs felt they were referring to a discipline rather than to a known, named, individual specialist. Equally specialists felt that they were no longer developing relationships with individual GPs.
‘… so that we now refer to a generic gastroenterologist or a generic chest physician …’ (CS1 GP Early Adopter and IT enthusiast)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 103
‘ Sometimes a GP hasn’t had so much personal knowledge. I know a GP who’d say Dr P will sort this, he’s a good fellow’ (CS1 Hospital Consultant)..
‘one of the frustrations was and continues to be that it almost pulls referrals into a speciality so actually instead of writing a referral letter to a consultant its more likely through Choose and Book that you end up in an amorphous .. neurology or something…’ (CS1 Primary Care Director for Hospital Trust).
Skill set Workability
Skill set workability was also unsatisfactory for Choose and Book in both environments. GPs felt that booking an appointment for a patient was a clerical task, which should not be taken on by highly trained clinicians. In the hospital setting, implementing C & B required considerable investment in training for outpatient clerks and medical secretaries.
Summary
To sum up, Choose and Book had normalized to very different levels in the hospital setting and in primary care. Our analysis suggests that the relatively high degree of normalization within the hospital was due to the very high contextual integration, with Choose and Book seen as an essential part of a financial survival strategy which had to be made to work. This very high contextual integration overcame any problems that might have arisen due to difficulties with interactional workability, relational integration or skill set workability. In primary care, there was no such overwhelming financial or other imperative. This meant that the GPs perceptions of a negative impact on interactional workability, and low skill set workability led to relatively low levels of normalization in this environment.
6.4.3 Case Study 2: Picture Archiving and Communication System
The Picture Archiving and Communication System (PACS) is a system for digitizing images, such as X-rays, scans or photographs. The digitized images can be stored on-line, and can be accessed simultaneously by several clinicians in different locations. We studied PACS in one hospital trust, in the Region where PACS implementation was most mature. The trust was located across several sites which were widely geographically separated. This trust had simultaneously implemented PACS and a Reporting Information System (RIS). The focus of this work package was PACS, but for many of our informants, implementation of PACS was inextricably linked with implementing the RIS. In order to maintain the focus on PACS, we only present data pertaining to PACS.
Degree of normalization
The PACS implementation was widely seen as successful, and PACS appeared to have almost completely normalized in this study site.
‘It’s fully implemented now. We have had it, we’ve had it just over two years now PACS so you won’t see any images now. I think at all, I think we’re all completely filmless, yes.’ (CS2 IT Training Manager).
‘it’s just taken for granted that you come in and you use PACS and that’s how your images are that’s it…. Just normal practice now.’ (CS2 IT Training Manager).
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 104
‘I think, um, it is so into the routine that we couldn’t take it out… It would be impossible to step back.’ (CS2 Radiology Manager).
‘It [PACS] is firmly embedded. If you walk around this hospital two years on, the clinical team wouldn’t even stop to think about what’s they’re doing. It just, there’s not other way for them to get to look at the x-ray image. … No, no it’s truly embedded...’ (CS2 Hospital Chief Executive).
‘Oh, superbly well. I think we could not function without it any longer. Literally its like that, no one, no one would go back.’ (CS2 Consultant Radiologist).
Less successful had been the simultaneous implementation of a Reporting Information System (RIS), which had been dogged by software problems, particularly with the voice recognition component.
Contextual Integration
As with Choose and Book, the degree of contextual integration of PACS appeared to have been a major factor leading to normalization. PACS was a national flagship initiative, with both policy makers and senior management seeing PACS as an excellent way of improving efficiency within radiology services and meeting demanding waiting list targets.
PACS also has a high contextual integration at a local level. It was seen as a way of obtaining clinician “buy-in” to IT adoption, and the Chief Executive Officer (CEO) of the Trust was determined to make this project successful. The CEO set up an implementation project board, which included a strong clinical lead who was very committed to the project, and the finance director. This project board was able to ensure adequate resources were allocated to the implementation - time, staff expertise, and funding.
‘the main driver was that hard copy film technology was beginning the cause more and more problems… [the films] were never in the right place at the right time. Well never is too strong a word, but I think there were times when we were running up to about 20% lost films. And what I mean by ‘lost films’ is just not being in the right place at the right time. So if you were in clinic … [and the films] were on the ward, but they weren’t in the right place to be used, so they were lost films.’ (CS2 Radiology Manager).
‘… it [the main driver to the implementation] was to improve efficiency, was probably the first and primary thing of information travelling between doctors.’ (CS2 IT Training Manager).
‘Well, nationally it [the main driver to the implementation] was to capture new technology which allowed images to be stored electronically rather than in brown envelopes and moved around at huge human costs. The secondary aim, which I don’t think has been fulfilled, was the easy sharing of information between clinicians on different sites. So that was the national one. But the motivation for me was I was looking for a way of capturing my doctors to get them to understand the power of IT.’ (CS2 Hospital Chief Executive).
‘And he [the CEO] was also, he was also very aware that diagnostic services were going to become very important in the business plan of the NHS….So
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 105
he was, I think he was aware of what PACS could do for the organization.’ (CS2 Radiology Manager).
Interactional Workability
Clinicians, both radiologists and non-radiologists, found that PACS improved encounters with patients. Consultations were no longer disrupted by lost or missing X-rays. Images were available on-line wherever the patient was seen – which was a substantial advantage in this trust which covered several sites. Images could be more easily shared with patients.
‘I would say the biggest advantage is in having images available all the time to everyone. So where, if you were to come into A & E and you’ve broken an arm and you have to be referred to the orthopaedic surgeons, there is no backwards and forwards of one piece of film following you around or not as the case may be. The fact that you have a picture that any doctor can see, the orthopaedic surgeon can see, it can be in the theatre if you get up there in 10 minutes time.’ (CS2 Radiology Manager).
‘One of the things that clinicians have been more willing to do, is to actually show the patient the images on the computer screen.’ (CS2 Radiology Manager).
‘…it [impact of PACS on the clinical encounter] is very positive. They can view images in the theatre more readily, they can view images in the outpatient clinic more readily, on the ward more readily, they can share the images with patients and relatives, they [clinicians] can have their queries answered by real-time telephone consultation. You know if you’ve got the patient there at the time and you want to tee them up with a particular programme of investigation or a particular treatment programme, and a decision around the image is critical to that …it’s a one stop shop and you deal with it all at the same time and off the patient goes down that particular pathway.’ (CS2 Consultant Radiologist).
Relational integration
PACS had a positive impact on relational integration. Non-radiologists could look at an image on-line, and if they had questions about the image or report, could contact the radiologist by telephone. Both clinicians could look at the same image simultaneously, despite being separated by distance, and this improved trust and confidence in reporting on the image and on subsequent clinical decisions.
‘I believe that, I can’t quantify it but I firmly believe it’s improved patient safety because if you’re not losing x-rays, you’ve always got the image and therefore the clinician then can look at he image and help make their diagnosis. If you haven’t got the image because we’ve misplaced it then there’s a patient safety issue there. You’re not exposing patients unnecessarily to extra x-rays.’ (CS2 Hospital Chief Executive).
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 106
‘you have now got clinicians, talking to each other via video conferencing, discussing cases, looking at the same image and I would argue that makes a better quality of conversation for the sake of the patient.’ (CS2 Hospital Chief Executive)
‘We have probably put into PACS a billion plus images, we have lost 3. Now I am disappointed we lost three! But within the clinical team they became a strong talking point – ‘Huh PACS lost an image!’. Whereas before – ‘oh yeah, we’ve lost an image – send the patient back to x-ray. So the actual confidence in the efficiency of the department to produce what is asked for and for it to be available, is just there now.’ (CS2 Radiology Manager).
‘And I think particularly with the interaction between say one of the clinicians and one of the radiologists, that’s improved because the consultant outside knows that the consultant radiologist inside has access to those images.’ (CS2 Radiology Manager).
‘ two doctors in different places to actually view those images at the same time from a clinical point of view …’ (CS2 IT Training Manager).
Skill set workability
There were relatively few problems with PACS in terms of skill set workability; however the accompanying RIS engendered a number of problems. Radiologists had to take on tasks that had previously been done by administrative staff, in terms of registering the patient on the system. Also some radiologists had difficulties with the voice recognition software for reporting images, and preferred to stick with the previous system of dictating to Dictaphone for subsequent typing up by the medical secretaries.
Summary
PACS had fully normalized in this study site. This appears to be due to high levels of contextual integration, interactional workability and relational integration. PACS clearly enabled the organization to meet its goals more efficiently than the old system. There was very strong leadership within the organization (including both managerial and clinical leadership). PACS had a very positive impact on interactional workability, as it clearly benefited patients and patient care. Finally, relational integration was improved, as communication and trust between radiologists and clinicians was made easier and more transparent. What problems still remained focused on the RIS, and occurred in the area of skill set workability.
6.4.4 Case Study 3: Clinical Nurse Information System
The Clinical Nurse Information System (CNIS) consisted of hand-held Personal Digital Assistant devices (iPAQs) which were wireless enabled. District Nurses could use them to record clinical assessment information while out in the community, and download the information to the central server once back at base. The system also included some minimal decision support, in the form of standardized assessment tools for ten common nursing problems, with associated algorithms for care. We studied the CNIS in one Community Health Care Partnership in Scotland. The implementation of the CNIS had started some 3 – 5 years prior to the interviews, and was initially designed as a pilot. It had been intended to act as a way of sharing
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 107
information between community nursing and social services, but this potential had yet to be realized, with social services not able to access information gathered by community nurses.
Degree of Normalization
The CNIS was at best only partially normalized in this district nursing community, and provides a good example of the difference between adoption and normalisation. Although some 80% of the District Nurses were using it, some teams were still maintaining dual systems (the new electronic record system and the old paper-based one) and it was evident that not all nurses felt comfortable using it, with the hand-held devices still seen as new or strange.
‘It’s been slow in, in places.’ (CS3 Senior Nurse A).
‘In our area 80% of the nursing staff are using it. That’s district nurses, and that’s all grades of district nurses.’ (CS3 Senior Nurse A).
‘that little bit different with the iPAQs - it’s got that sexiness to it.’ (CS3 Health Board Services Manager).
Contextual Integration
There were both positive and negative contextual integration components to the CNIS. On the positive side, the Scottish Executive were keen to modernize, and saw electronic record keeping as essential to this, so they made funds available for this initiative. Many of the nurses were frustrated with the existing paper record system, which often required them to drive long distances simply to obtain a patient record. Senior managers were keen to encourage IT use amongst district nurses, and saw the CNIS as a good way of doing this. On the other hand, there appears to have been weak leadership, with the dedicated implementation group being disbanded after the first year. Training was under-funded and under-resourced, with no planning for on-going or “top-up” training, despite an acknowledgement that the target workforce had very low IT literacy.
‘I think there’s probably a few things that prompted it, one being that nurses felt they needed an updating on the paper documentation that they used, and also it was coming into the 21st century, and we really felt that we wanted to move ahead and have some modern electronic system and a method of gathering patient information. One other external driver was the need to be able to share Single Shared Assessment information with social work departments, and we wanted to be able to do that electronically.’ (CS3 Lead Project Nurse)
‘a frustration of the nurses when they had to see a patient who was not residing in the area that they were based in and they had to drive to another area in order to take a copy of the records. That was one of the big drivers for nurses at that time. But one of the main drivers for nurses was actually the impracticalities of if you needed information about someone whose condition had deteriorated, perhaps on a Friday afternoon, you then
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 108
had to write a different set of documentation and drive it to the place that the patient needed to be seen, otherwise there was no way of getting the information to them. So one of the big things was to actually have a system that talked to nurses across the area.’ (CS3 Clinical Services Manager).
‘There's always been, it’s always been a wish for the business to have; electronic data capture and the catalyst for it was a mandate from the Scottish Executive that our staff should be recording a single shared assessment electronically. That was the driver for it, that was the catalyst that gave us some initial funding to pilot things, so wanting to get a pilot thing going. So, it had always been a wish, and this was the, this was the excuse that gave us some money to do it… A longstanding wish to have, from the Government aspect, to have electronic health registers for community nursing as opposed to bits of paper and typewriters and all the rest of it. This [funding for the pilot] was through the Scottish Executive Modernising Government fund. One of their mandates was that health and social work should be able to record single shared assessments electronically.’ (CS3 Health Board Services Manager).
‘it would introduce them [the district nurses] to – as individuals, as professionals – to this world of electronic record-keeping and information sharing, which they just simply had no experience of.’ (CS3 Director Community Health and Care Partnership).
‘the steering group that was set up for community nursing IT system hasn’t met, therefore there’s not a defined lead in this. There is a project manager, but we don’t really see him very much.’ (CS3 Senior Nurse A).
‘I was on the original Implementation Group and then it kind of went into abeyance... So we have raised it with our Heads of Service to say that we really feel that there needs to be an Implementation Group to drive this forward. And we need to have some support for the nurses in order to get this one, otherwise it won’t work. And I think the feeling is that we’ve never got this far in [name of Health Board] with an IT system for community nurses, that we want it to work, you know, as best as it can. But it won’t unless we stop and take recognition of where we are, and see what we can do to support the staff.’ (CS3 Senior Nurse B)
Interactional Workability
One of the positive features of the CNIS was its use of small hand held devices (iPAQs). These were relatively robust, cheap, and portable. This meant nurses felt comfortable carrying them when visiting patients, and were reassured that the device could be replaced if necessary. This was in contrast to previous initiatives, which had involved laptops. Laptops were too heavy for nurses to carry around, and made them feel vulnerable to attack. As the nurses were able to carry the iPAQs around, they were more likely to use them during, or immediately after, patient visits, which improved their efficiency.
‘You’ve seen how streamlined they are quite you know petite. You can put them in your pocket. Well the devices that we used in the past were sort of
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 109
bulbous and very big and the nurses just thought well I can’t be bothered carrying those sort of things around and especially in the environment and the area that they are.’ (CS3 IT trainer).
‘Well, that was the key thing. They wanted to use and record data at the point of patient contact as a domiciliary workforce, that was the only way we could do it. We couldn’t afford to get them laptops. They didn’t want laptops, because it was like an extra, big thing. They were really about some of the areas they go to, being seen as the weekly District Nurse to people having drug habits and things - it’s an expensive piece of equipment for that type of thing.’ (CS3 Health Board Services Manager)
Relational Integration
Our data contained no information that suggested that the CNIS had any impact on relational integration (either positive or negative).
Skill set Workability
Skill set workability was the major factor impeding normalization of the CNIS. The nurses started from a very low level of IT literacy, and many were alarmed that this would jeopardize their future employment. Trainers had to spend a great deal of time on one-to-one training, and reassurance that nurses were not going to be sacked if they failed to learn how to use the CNIS.
‘It’s basically nurses who don’t even have a computer in their own homes and they haven’t actually come across this sort of technology and they’re having to face it at work and sometimes you get that sort of nervous reaction that they maybe might feel a bit inadequate in the sense that that oh this is really daunting. I’ve never used a computer system before. Will this mean I’ll be out of a job?’ (CS3 IT Trainer).
‘We do hold their hand quite a lot … we’ll go out to their health centre where they’ve got their caseloads and they’ve got basically computers that they actually use. So we’ll go out and train them and we’ll actually sit through, maybe putting on a patient register and we’ll go through the whole process with them so they get the training in terms of the functionality and then we’ll sit there with them doing one or two new patients that they’re having to register onto the system.’ (CS3 IT Trainer).
‘probably a hindrance, not all nurses are IT literate; you know, there IT is somewhat limited. And that’s quite frightening for staff when we’re saying to them, you know, you’re not the right…. (laughing) for the future. So we’ve had to address one or two IT issues.’ (CS3 Senior Nurse B).
Summary
The CNIS had achieved relatively low levels of normalization, despite data collection occurring some 3 – 5 years after the start of implementation. Factors contributing to this low level of normalization included mixed contextual integration, with a strong political will to introduce electronic record keeping, but poor levels of implementation planning and execution within the organization. A major hurdle was poor skill set workability, with
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 110
many nurses needing more substantial training than was provided. The positive impact on interactional workability was fairly slight, and possibly not sufficient to overcome the negative skill set workability and poor local contextual integration.
6.4.5 Summary of overall findings
The Normalization Process Model was found to be a suitable analytical framework with good explanatory power in these contrasting case studies (Table 9).
Table 9 Summary of Main Findings
NPM Construct CS 1 = Choose and Book
CS 2 = PACS CS 3 = CNIS
Interactional Workability (IW)
Poor, especially in GP Good Medium to good
Relational Integration (RI)
Poor to neutral Good Medium to poor
Skill Set Workability (SSW)
Poor for GPs, Neutral for Hospital
Medium to Good
Medium to poor
Contextual Integration (CI)
Neutral for GPs, very good for Hospital
Good Medium to poor
Choose and Book had normalized to very different extents in secondary and primary care. The overwhelming importance of Choose and Book to the hospital’s financial survival, in terms of maximizing the proportion of new outpatient referrals, combined with strong managerial and clinical leadership and a well planned, well resourced implementation strategy had resulted in relatively high levels of normalization within the hospital. In contrast, the negative impact on interactional workability in primary care, caused by the slowness of the technology, and the negative impact on skill set workability, with GPs resenting being asked to take on work that was essentially clerical in nature had led to low levels of normalization in primary care.
PACS had become almost fully normalized in our study site. This appeared to be due the positive impact that PACS had on three of the four constructs of the NPM, with no negative impact on the fourth. PACS had high contextual integration, in that it helped the study site achieve national targets on waiting lists for diagnostic services. It increased the site’s efficiency, reducing the number of lost x-rays, and hence the need for repeat imaging. These efficiency gains had been predicted by senior management, who had invested heavily, both in the system itself, and in re-structuring workflow to fit in with the new system. There was very strong managerial and clinical leadership, with adequate resources allocated to the implementation. Interactional workability was also high, with all concerned able to see (and experience in consultations) the benefits to patients. Relational integration was good, as images could be shared between
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 111
radiologists and clinicians, allowing for good interdisciplinary discussions, and streamlining patient care. There were some problems with skill set workability, particularly with the associated RIS and voice recognition software which had limited some of the benefit realization.
The CNIS had, at best, only partially normalized. There appeared to be low contextual integration, with no clear vision as to the benefits of CNIS for the organization or workforce, and weak, unsustained leadership around the implementation. The most positive aspect of the CNIS was its interactional workability, with light portable hand held devices which the nurses found useable and time saving. Skill set workability was an issue, with insufficient attention paid to on-going training after an initial good start.
6.5 Discussion and conclusions
Main results
This work package successfully identified three case studies which between them spanned community, primary and secondary care, as well as covering the main e-Health domains of communication, management, decision support and information. Views of those responsible for implementing these e-Health initiatives were successfully obtained, and the Normalisation Process Model (NPM) provided a good explanatory model for the relative normalization of the three e-Health initiatives.
Relationship with existing literature
Our work is highly congruent with existing literature. The problems with implementation of Choose and Book have been well documented, with just 63 referrals made using Choose and Book in the first year (Nolan and Whitfield) and Primary Care Trusts (PCTs) only half way to the Choose and Book target in 2007 (Bell 2007), with considerable discontent about Choose and Book voiced by senior managers in hospital trusts (Hendy et al. 2007). A questionnaire study of 500 GPs published in 2006 found that nearly two-thirds of respondents were not in favour of Choose and Book, citing time constraints and the inflexibility of the system (Pothier et al. 2006), reflecting our finding that poor interactional workability had impeded normalisation of Choose and Book in primary care.
In contrast, the literature on the Picture Archiving and Communication System (PACS) suggests that this has been widely adopted internationally (van de Wr et al 2008). Where system reorganisation has accompanied PACS implementation, there have been marked improvements in workflow (van de Wr et al 2006), with improvements in reporting times and productivity, and fewer requests for repeat x-rays (Collin et al 2008). An early interview study of users of PACS in one hospital in 1999 reported user preference for PACS over traditional films, because of improved ability to share images between clinicians (relational integration), faster reporting times (contextual integration), and potential benefit for patients (interactional workability).
The literature on the use of hand-held computer devices supports our findings in Case Study 3 that their lightness and portability are significant positive features, as were their potential to improve patient safety by
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 112
ensuring patient records were available at the point of care (Lu et al 2005). Looking specifically at the use of Portable Digital Assistants (PDA) by nurses, Lu found that contextual integration factors, such as lack of integration with hospital information systems or lack of support at work were perceived as major barriers by nurse users (Lu et al 2006). Overall, nurses are lower users of PDAs than doctors (Garritty and Ek 2006).
Methodological issues
The case study approach worked well, and identifying “implementers” was straightforward. “Implementers” did provide, as hypothesized, useful data from a perspective that differed to clinicians. These implementers tended to have a whole systems approach. The method for identifying the case studies appears to have worked, in that the three studies covered a range of health care contexts and e-Health domains. The interview schedule had been well piloted.
We had initially intended to undertake ten interviews per case study. We did achieve this number in Case Study 1, but in both Case Studies 2 and 3 it became apparent that we had sampled to redundancy after relatively few interviews. In contrast, we do not think that we sampled to redundancy in Case Study 1, as there were other perspectives that could perhaps have been usefully gained. As this was our first case study, we were not aware that we would not need to undertake so many interviews for the subsequent studies, and stopped doing interviews after ten to ensure the overall work package was completed in time and within budget.
An additional potential weakness was the limitation of Case Study 2 to PACS. Given that PACS had been implemented with the new Reporting Information System, it might have been better to expand Case Study 2 to include both technologies, particularly as they were inextricably linked in the minds of many of our informants.
However, a major strength of this work package was the analysis which was undertaken rigorously, with repeated double coding, and discussion within a multidisciplinary team, a process known to aid reflexivity and rigour in qualitative research (Barry et al. 1999).
6.5.1 Discussion and Conclusions
Our findings have major implications for policy, practice and research. This work package, like the preceding ones, highlights the value of the NPM as a theoretical framework for assessing the likely normalization of a new e-Health initiative. E-Health initiatives that score highly on all four constructs (contextual integration, interactional workability, relational integration and skill set workability), are highly likely to normalize. Low scores on any one construct should alert policy makers and senior managers to potential difficulties in this area, which need careful consideration during and after implementation. Low scores across all four constructs suggest that the initiative has relatively low likelihood of normalizing successfully, and that some rethink may be needed.
Contextual integration also depends on the organisation. The importance of strong leadership combined with a carefully planned, adequately resourced and well executed implementation strategy is clear.
It is worth noting that normalisation does not necessarily imply standardisation or the uniform deployment of e-Health technologies, and that this is important when any evaluation of their use-in-practice is made
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 113
in the future. As in the CNIS case study 3 it might well be possible for elements rather than a complete system to be normalised.
6.5.1.1 Recommendations
1. We strongly advise senior staff planning an e-Health initiative to consider the following triad prior to implementation:
� The overall context, including national priorities and strategies, as well as local priorities, leadership and resources.
� The nature of the e-Health initiative, its likely impact on clinical interactions and inter-professional relationships, and its user-friendliness.
� The impact of the implementation on the workforce, including workload, training needs, and alteration in responsibilities.
2. Future research should be commissioned to:
� Examine the potential of the NPM as a predictive tool, in the context of e-Health initiatives.
We have used the data from this work package to generate an e-Health implementation toolkit (e-HIT). This was designed to assist senior managerial staff with planning and implementing future e-Health initiatives, and is discussed in detail below.
6.6 Phase 2 Development and Formative Evaluation of the e-Health Implementation Toolkit.
6.6.1 Introduction and Background.
One of the initial planned deliverables from this Work Package was the development and formative evaluation of a Model Implementation Process (MIP). During the development phase it became apparent that, as far as implementation of e-Health initiatives is concerned, one size definitely does not fit all. With this in mind, it seemed more sensible and valuable to concentrate on developing an e-Health Implementation Toolkit (e-HIT), rather than a model implementation process. In view of our emerging research findings it became clear that such an instrument would prove useful as a sensitizing tool for senior managers or other staff who are considering, planning or undergoing an e-Health implementation.
6.6.2 Aims and objectives
The aim of phase 2 of this work package was to apply the understanding gained from this work package and the other work packages in EH94 to produce an e-Health Implementation Toolkit (e-HIT).
The e-Hit was designed to be used at multiple points during the implementation process, including whether or not to embark on the implementation, planning the process, monitoring progress, and reflecting on solutions to unanticipated problems. It was therefore envisaged as a tool that could be used widely.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 114
6.6.3 Methods
The e-HIT was developed by Elizabeth Murray in collaboration with a commercial IT consultancy company “Rocket Science” with support from Carl May and Frances Mair. EM wrote the content, while Rocket Science designed the presentation and navigation, and CM and FM provided feedback throughout the development process.
Development of the content of the e-HIT.
The content of the e-HIT was derived from the data obtained in all four of the work packages making up the EH 94 programme, but with particular emphasis on the results of the scoping and systematic reviews undertaken in WP1 and the results of interviews with implementers undertaken in WP4. The Normalisation Process Model provided the theoretical framework, and the concepts within the NPM are well represented in the e-HIT.
The content of the e-HIT was derived by considering the main themes identified in the scoping review and the themes identified in interviews with implementers. These themes were then synthesised with each other to create a database of items which had theoretical and empirical validity. The theoretical validity came from the NPM, while the literature review and WP4 provided empirical support.
To facilitate comprehension and use of these items, they were grouped into three major components: the context (organizational factors, national and local policies, other drivers); the intervention (user friendliness, fitness for purpose, flexibility, impact on clinical practice, effectiveness and cost-effectiveness); and the workforce (impact on workload and workflow, training, power relationships between different professional groups, responsibility and accountability). The items were reviewed and reduced in number to yield a manageable toolkit. Each item was operationalised by statements, which were anchored by extreme negative and extreme positive positions. These statements were reviewed by the co-investigators and the RocketScience team.
Once the main theoretical and empirical content had been determined, the rest of the content was written with a view to making the e-HIT comprehensible and accessible. An introduction included information on what the e-HIT was, who should use it, when it should be used, and how it should be used. There was a section on how it was derived, and detailed instructions for first time users. This section could be easily bypassed, with return users encouraged to go straight to the toolkit.
The main body of the toolkit contained the content described above. The final section consisted of case studies (the three case studies described within Phase 1 of WP4), to give users an example of how they might complete the toolkit, and how the information gained could predict normalisation of an initiative.
Development of the presentation of the e-HIT.
We were fortunate to work with RocketScience, an independent software and consultancy company who have done a lot of work with e-Health and the NHS. They already had a template for a toolkit, which they adapted for the e-HIT. In the main content section, users were asked to provide a score between 0 and 10 for each statement, with a box for text comments to justify the score allocated.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 115
These scores then automatically generated a report for the user, summarising the data entered, and providing a visual display. We emphasised that the toolkit should be used as a sensitising tool, to help implementers consider, address and thereby forestall potential difficulties.
Formative evaluation of the e-HIT.
A two stage formative evaluation of the e-HIT was undertaken, using on-line task groups. For the first stage of the formative evaluation, the draft e-HIT was circulated to the EH94 Advisory Group (6 senior academics and managers with extensive experience of e-Health and the NHS) and the EH94 steering group. These experts were asked to complete the e-HIT for an e-Health initiative they had personal experience of and, on the basis of this experience, to comment critically on the e-HIT. Specifically, respondents were asked whether they thought the e-HIT would be useful to senior managers considering, planning or undertaking an e-Health implementation, what would make the e-HIT more useful, what were the positive features of the e-HIT, and what features needed modifying. Respondents were asked to suggest modifications which would improve the e-HIT, and for other general or specific comments not covered by the questions listed.
There was a good response from both the Advisory Group and Steering Group, with all members contributing comments and discussion. The overall response was overwhelmingly positive, with comments including “fantastic” and “excellent piece of work”. Specifically, respondents thought that the non-prescriptive approach and emphasis on the e-HIT as a sensitizing tool would appeal to senior managers. Respondents thought that the overall layout was clear, the language easily comprehensible, and that the main areas of importance were well covered. They liked the sliding scales, space for explanatory text, and instant feedback.
Areas of concern were the mechanisms for dissemination, with several respondents referring to “toolkit fatigue”, with subsequent need for active steps to bring the toolkit to potential users, rather than leaving them to find it. Respondents also felt that the toolkit would be improved if it was fully web-enabled, allowing users to store their responses on the web for others to view, as this would allow build up of shared knowledge, expertise and experience across the NHS.
There were also specific suggestions for improvement about the navigation, layout and wording of individual components of the e-HIT, with requests for more information to help with completing some of the questions, and clarification of individual questions.
Representatives from Rocket Science and Elizabeth Murray met to discuss these comments, and agreed on the following changes:
� A redesign of the Introductory section, allowing experienced users to bypass this completely;
� Improved explanation of the uses of the report section;
� Provision of explanatory mouseovers to assist with completion of individual questions;
� A more streamlined overall lay-out.
Resources were not available for making the toolkit entirely web-enabled, and we agreed to seek subsequent funding for this. Similarly, the issue of dissemination needs further discussion with the Service and Delivery Organisation.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 116
Once these changes had been implemented, the second stage of the formative evaluation of the e-HIT was undertaken. This involved circulating the revised toolkit to all participants in Work Package 4, i.e. all twenty-two of the implementers who had been interviewed for one of the three case studies (Choose and Book, Picture Archiving and Communication System, and the Clinical Nurse Information System). At this stage participants were thanked for their initial contribution to the study, and asked to comment on the e-HIT as one of the products of the study. They were asked to comment on the likely usefulness of the e-HIT, suggestions for improving its usefulness, whether it adequately reflected their experience of a specific e-Health implementation, and any other comments or suggestions.
This second formative evaluation yielded only minor suggestions, which were easily incorporated into the final version.
6.7 Results
The e-HIT is freely available for downloading from the following url:
http://www.ucl.ac.uk/pcph/research/
A colour print-out is also attached as an appendix 29.
6.8 Discussion and Conclusions.
The e-Health Implementation Toolkit combines a strong theoretical underpinning (the NPM) with empirical data derived from a literature review and primary data collection. It has been favourably formatively evaluated. For senior managers planning to use the toolkit we would recommend that representatives from the various different professional groups affected by any given e-Health initiative complete the tool, and discuss their relative inputs, to help generate a whole systems viewpoint. We believe that the e-HIT would prove most valuable when used in this way. The effort that the research team, together with our external advisory group, invested into understanding the ways in which e-HIT would be made most useful, was in itself an indication of the research team’s understanding of how best to normalise e-HIT and has informed the development of our recommendations regarding its further development and dissemination.
6.8.1.1 Recommendations:
1 We suggest the SDO host the e-HIT on their website, and disseminate it to senior managers involved in planning e-Health implementations.
2 Future research should be commissioned to build on this work, with a view to:
� determining whether the e-HIT is a useful tool for senior managers;
� extending the tool, by making it fully web-enabled, to allow live data capture, and storage of new data on additional case studies.
� extending it to non e-Health technological initiatives.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 117
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 118
7 Taking Analysis To A Higher Level
7.1 Background
Throughout this project we have used the NPM as our conceptual framework. Each work-package has used the NPM to inform data collection and our approach to analysis. However, as indicated in Chapter 2, in parallel with this project the NPM was undergoing further development, and this led to a middle range theory of implementation and integration – Normalization Process Theory, or NPT (May and Finch 2009). Qualitative analysis in the systematic review (WP1) and task groups (WP2) revealed that between one third and one quarter of coded items did not ‘fit’ the NPM. This data was relevant to the topic of study, and required further analysis. We took this as an opportunity to employ the extended and refined NPT. We recoded qualitative data in work-packages 1 and 2, and added NPT oriented questions to structured instruments developed for work-package 3, (as data collection design in WP4 was based on the NPM, reanalysis using the NPT was not felt to be useful). This section of the report outlines the use of the ‘next generation’ conceptual model, as we employed it to develop and extend our analysis.
The NPT retains the key assumptions and constructs as presented in the NPM, but extends these in a number of ways. The constructs of interactional workability (IW), relational integration (RI), skill set workability (SW) and contextual integration (CI) are retained as the four key domains of work that represent a construct of ‘collective action’, but three other higher level constructs have been added to extend the utility of the model as a theory that can be applied to achieve a better understanding of the processes of normalisation not only in relation to the kinds of technologies that have been of interest to this project, but to complex interventions more generally. The NPT thus describes how the work of implementation is operationalized through four generative social mechanisms (coherence; cognitive participation; collective action; reflexive monitoring). These four mechanisms are defined as follows:
� Coherence refers to the sense-making work that people do individually and collectively when they are faced with the problem of operationalising some new way of working.
NPT proposes that: embedding a complex intervention depends on work that defines and organizes it as a cognitive and behavioural ensemble. Embedding work is shaped by factors that promote or inhibit actors’ making sense of it. The production and reproduction of a complex intervention requires that actors collectively invest meaning in it. This is the work that people do in preparing to engage with a new technology or complex intervention. In relation to e-Health, coherence refers to a range of ‘sense-making’ activities, such as, for example, agreeing collectively the purpose of an e-Health system and ensuring that it fits with organisational and practice objectives; and identifying and valuing the perceived benefits of working with an e-Health system (both collectively, and as individuals implicated in the use of the system).
� Cognitive participation is the work that people do to engage with a new technology or complex intervention. The NPT proposes that: embedding of a complex intervention is
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 119
dependent on work that defines and organizes its participants. Embedding work is shaped by factors that promote or inhibit actors’ participation. The production and reproduction of a complex intervention requires that actors collectively invest commitment in it. In relation to e-Health, for example, cognitive participation relates to issues such as organisational strategies for engaging different categories of workers in the design and organisation of e-Health systems in order to enable them to use the system effectively in working collaboratively; and facilitating processes that encourage individuals to commit themselves to the work required for embedding the new system into practice.
� Collective Action describes the work that people do to enact a new technology or complex intervention. This encompasses the four constructs of the NPM, namely interactional workability, relational integration, skill set workability and contextual integration. In relation to e-Health, this refers to the ongoing work of embedding itself, and how it gets done in practice, and this report has already detailed the kinds of work that promote and facilitate collective action with respect to these four domains of work.
� Reflexive monitoring relates to the informal and formal work that people do to appraise the effects of the new technologies’ complex interventions that they have enacted. NPT proposes that: embedding of a complex intervention is dependent on work that defines and organizes the everyday understanding of it in practice. Embedding work is shaped by factors that promote or inhibit individual and collective appraisal of the complex intervention. The production and reproduction of a complex intervention requires that actors collectively invest in its understanding. In relation to e-Health, reflexive monitoring refers to the processes by which organisations and individuals are able to reflect on and understand the effectiveness and utility of an e-Health system as experience and knowledge of the system is gained over time. This includes not only being able to make ongoing appraisals about whether the benefits of using the e-Health system are being realised and are in proportion to the effort required, but also being able to use that understanding to make changes and improvements to the design and organisation of the e-Health system in order to achieve the objectives of using it.
7.2 Aims/Objectives
The aim of this part of our project was to:
1 Use NPT to inform analysis of qualitative data gathered from our systematic literature review (stage 2 of WP1) and our task groups and interviews from WP 2.
2 Assess the potential contribution of NPT to our understanding of implementation and integration issues.
It is important to note that we did not seek to formally test the theory in WP1 and 2, since these were re-analyses of already collected data. In WP3 we sought indicative information about value if NPT through a small number of additional items. This work should be regarded as theory refining research, not theory testing.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 120
7.3 Methods
Qualitative analyses had already been conducted in which all included papers in WP1 (systematic review) and interview and task group transcripts (in WP2) had been subjected to attribution analysis using NPM to define a coding framework. These methods have been described in detail elsewhere in the report. In this additional work, these same attributions were recoded using a coding framework defined by NPT. FM and CRM recoded this data independently of each other. There was a high degree of congruence between the two coders (NPT is easier to use) and disagreements were rare. When these occurred they were resolved by discussion. If any areas of disagreement remained then a third reader (TF or EM) served as arbiter.
In work-package 3, following the original protocol, the TARS questions reflected mainly the four domains of work specified in the NPM (IW, RI, SW, CI). Prior to administering the Phase 3 surveys however, single question items were written and included to reflect the three new constructs of the NPT as follows:
Coherence: The staff who work here have a shared understanding of what the system is for and how it is to be used
Cognitive participation: The staff here are committed to making the system work
Reflexive monitoring: There are ongoing mechanisms for monitoring and appraising how this e-Health system is used
The survey results relating to these items were analysed descriptively as part of the full item set, as specified in Section 5 that reports WP3.
7.4 Results
WP 1 Analysis Using NPT
Recoding of the attributions from the systematic literature review undertaken in WP1 resulted in the following distribution of codes (Table 8).
Reassuringly, it is immediately clear that the bulk of attributions (67%) fall within the Collective Action category of the NPT, which encompasses all four constructs of the NPM. This analysis therefore demonstrates that the bulk of implementation issues relate to the NPM constructs of interactional workability, relational integration, skill set workability and contextual integration. Table 10 and Figure 5 show that the rest of the attributions are fairly evenly spread across the other three components of the NPT, namely, coherence, cognitive participation and reflexive monitoring. A small number of attributions remain outside of the scope of the NPT, (36/411 coded statements, less than 9% of the total).
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 121
Table 10 Number of Attributive Statements From WP1 Within Each Category of the NPT
Coherence Cognitive
Participation
Collective
Action
Reflexive Monitoring
Outside NPT
1. Broens 2007 3 2 17 3 1
2 Chaudrhy 2006 0 1 18 2 0
3 Hebert 2006. 3 0 5 2 1
4 Hilty 2002 2 5 14 2 1
5 Jennett 2004 7 3 8 7 1
6 Jennett 2005 2 2 5 2 1
7 Johnson 2001 3 5 17 0 2
8 Kawamoto 2005 1 1 9 1 0
9 Kukafka 2003 1 3 13 2 2
10 Leatt 2006 6 6 15 9 5
11 Lu 2005 2 1 29 4 4
12 Ohinmaa 2006 4 3 10 3 2
13 Papshev 2001 3 3 15 1 2
14 Peleg 2006 0 7 28 7 8
15 Shekelle 2006 1 1 10 0 3
16 Studer 2005 1 4 13 1 0
17 Vreeman 2006 0 1 19 6 1
18 Yarborough 2007 3 2 16 3 2
19 Yusuf 2007 4 3 14 0 0
Totals 46 53 275 55 36
The results in Table 10 are visually displayed in the radar plot seen in Figure 5. This radar plot is provided for illustrative purposes only and graphically displays the distribution of coded statements in relation to the constructs of the NPT, namely, coherence, cognitive participation, collective action and reflexive monitoring. The shaded zone indicates where the bulk of coding was located, within the collective action domain. This latter domain contains all of the constructs of the NPM and therefore it can be seen that the bulk of implementation activity is addressed by the original NPM constructs.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 122
Figure 5. Distribution of Attributed Statements From the Literature on Implementation
CO = Coherence; CP = Cognitive Participation; RM = Reflexive Monitoring; CA= Collective Action
Illustrative examples of what types of attributive statements from the literature fell into each category of the NPT are now presented.
7.4.1 Coherence
Coherence
Coherence could be taken to describe participants’ understandings of the broad policy mechanisms or actions that contributed to a rationale for promoting the uptake and implementation of e-Health interventions as this extract from Ohinmaa 2006 demonstrates:
‘Directed, systematic government policy aiming to increase investment in technology can enhance networking and collaboration within the healthcare system’ (Ohinmaa 2006)
Or it could be used to relate to the work needing to be undertaken with staff intended as implementers and users of the new e-Health service, to identify and explore beliefs about the system to be implemented as the following extract shows:
Distribution of Atributions Across NPT
Series1 0
50 100 150 200 250 300
CO
CP
CA
RM
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 123
‘Prior to implementation it may be beneficial to assess expectations and concerns regarding EMR system implementation’ (Studer 2005)
Thus coherence was concerned with preparatory work undertaken either locally or to facilitate understanding of the purpose and benefit of the e-Health service in support of its implementation but also addressed the issue of engaging with stakeholders regarding their expectations, concerns or requirements.
Cognitive Participation
Cognitive participation addressed different means of encouraging engagement with and enrolment of health professionals in the implementation of e-Health services and how such new services could be seen to become legitimised or not, by health professionals.
Hilty 2002 proposes ways to encourage health professional participation which included:
‘Incentives for each of the parties involved’ (Hilty 2002)
Cognitive participation, therefore, included actions that could potentially serve as drivers to participation in the implementation of new e-Health services. This could refer to a whole range of incentives, although obviously it is financial incentives that are referred to in this instance.
However, there were other types of issues that fell into the cognitive participation category. These included a range of actions that helped legitimise participation in the e-Health implementation process and promote it as a worthwhile activity. One aspect mentioned on a number of occasions was that of local “champions” as this extract from Johnson 2001 shows:
‘IT advocacy. In concert with increasing awareness of IT, we need to establish a network of IT advocates who can facilitate the movement of more apprehensive colleagues with the help of other groups.’ (Johnson 2001)
Such champions were seen as having the ability to promote utilisation of new e-Health services by more reticent colleagues. However, rather like all types of “peer pressure” this could be a double edged sword as Peleg suggests:
‘lack of improved performance could be due to any number of factors, such as lack of support among colleagues’ (Peleg 2006)
Thus health professionals, by being enthusiastic advocates, could legitimise and promote enrolment and commitment of colleagues or alternatively by
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 124
negative actions could jeopardise the commitment of staff needed to make the system work and thus impede implementation.
Collective Action
Collective action refers to the work undertaken to operationalise a new e-Health service. This encompasses all the constructs of the NPM previously described, interactional workability, relational integration, skill set workability and contextual integration. Examples of collective action attributions have been provided in great detail within the workpackages and will therefore not be repeated here.
Reflexive Monitoring
Reflexive monitoring concerns how individuals or groups decide whether an e-Health intervention is worthwhile or not. For the most part this has to do with issues of evaluation and monitoring and how this is used to influence utilisation and future e-Health implementations as the following comments suggests:
‘Comprehensive cost effectiveness studies are essential in developing future financing structures’ (Broens 2007)
‘the complexity of medical practices and the high cost of implementing CDSSs make evaluation of CDSSs both a challenge and a necessity.’ (Peleg 2006)
However, evaluation was also promoted as necessary to ensure safety concerns were addressed as the following comment highlights:
‘computer systems have the potential to introduce errors’ (Vreeman et al. 2006)
Addressing such concerns through evaluation could of course, either alleviate them, or verify them, necessitating amendments to the e-Health service being implemented.
Issues Outside the NPT
There remained some issues that did not fit within the NPT. Although this related to only a minority of attributions it is important to consider what still lies outside the theory. Issues of a strictly technical nature or general attitudinal issues remain outside the NPT. Examples of technical issues were:
‘Customisation: various functions or programs can be added to support different medical specialities.’ (Lu et al. 2005)
and
‘product quality’ (Papshev 2001 )
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 125
Indeed, some statements, such as “product quality” as provided above are so generic and vague, without accompanying contextual data, that it is uncertain whether the concept actually lies outside the model or whether it is merely its generic nature that renders it unable to be coded.
Similarly, if attitudes were simply general comments such as:
‘providers resistance to change’ (Papshev 2001)
Then these too fell outside the theory. The NPT, therefore, focuses on the work that people do in the same way as the NPM. Thus general comments about culture, technology quality and attitudes are not included. However, all the work which individuals undertake to implement and integrate e-Health services does fit well within the extended NPT.
7.5 WP2 Analysis Using NPT
Normally, qualitative analysis excludes even simple quantitative descriptions of the contents of interview data. There are good reasons for this, this data is heterogeneous and is produced in non-replicable encounters. However there is merit in describing this body of data quantitatively here, because it shows first that NPT provides a framework that appears to account for more of the data collected. This is important if attribution analysis is used. If the theory does not account for the attributions, then it is an insufficient resource for interpretation. Full details of the count are presented in Table 11: 712 attributions were coded. Of these, 69% referred to aspects of collective action. This is to be expected since respondents were discussing enacting e-Health implementation processes Also, as before, only a minority, 19/712 (3%) of comments did not fit within the NPT.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 126
Table 11 Number of Attributive Statements From WP2 Within Each Category of the NPT
Coherence Cognitive Participation
Collective Action
Reflexive Monitoring OUTSIDE
Task group 1 11 0 57 2 0
Task group 2 5 3 59 5 0
Task group 3 8 3 26 6 1
Task group 4 5 3 39 11 3
Task group 5 14 9 55 15 3
Interview 1 3 4 13 2 0
Interview 2 5 1 17 2 3
Interview 3 0 2 17 2 0
Interview 4 5 1 18 1 0
Interview 5 11 0 42 4 2
Interview 6 1 2 13 0 0
Interview 7 2 4 7 3 1
Interview 8 1 1 10 1 1
Interview 9 2 3 16 3 0
Interview 10 4 1 14 0 1
Interview 11 1 2 19 2 1
Interview 12 2 1 15 0 4
Interview 13 2 1 20 2 0
Interview 14 0 1 15 2 0
Interview 15 4 4 4 1 0
Interview 16 2 1 16 1 0
Totals 88 47 492 66 19
Figure 6 illustrates this data via a radar plot. This figure provides the data from Table 10 in a graphical form and again is used for illustrative purposes so that it is clear to the reader that the bulk of codes (the shaded zone) falls within the collective action category, as did the WP1 data, thus showing once again that the NPM captures the vast majority of implementation activity and that the additional constructs of the NPT identify other implementation issues in quite a balanced way.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 127
Figure 6. Distribution of WP2 Attributions across the NPT
Distribution of WP2 attributions across the NPT
0
200
400
600Coherence
Cognitive Participation
Collective Action
Reflexive Monitoring
Coherence, cognitive participation and reflexive monitoring receive between 6% - 12% of the share of the coding. Thus the NPT addressed 97% of the implementation or integration issues raised by health professionals.
Examples of attributive statements from task group or interview participants within each of the categories are provided below.
Coherence
Coherence was sometimes exemplified by this group by descriptions of how beliefs and knowledge about e-Health systems could affect their utilisation as the following comments highlights:
‘That’s really important, the NHS 24 was completely new, and it was available to the public and they didn’t have a clue how to use it, particularly at the start’
(Task group member)
Also the following quotation illustrates how “attitude” could now be classified as part of NPT when it related to issues of engagement and involvement of users:
‘I think… professional attitude to implementation: that might come in terms of the management of change, but also, their involvement in understanding the engagement within the procurement process, I think is quite key. That’s actually quite key ….., if you have that engagement, I would suggest, that, what you get, is, you get confidence of that group of staff and their colleagues, that the people who use them have been involved in choosing them, or designing them, or, you know, specifying them.’ Interviewee 5
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 128
Attitude, in this instance, was not simply a descriptive term but instead was referring to improving individuals’ understandings of an e-Health service (in this case through processes of engagement) and therefore fell within the coherence domain of the NPT
Cognitive Participation
Within this data set cognitive participation included issues about champions as before but highlighted the importance of getting the cooperation and involvement of health professionals and appreciating the important role of patients. One task group member commented:
‘And I’m not sure that the field…test that we’re, even in our own organisation and nationally, gives enough to pull the clinical community on board as well. And I think people now almost see the project to be implemented and deadlines to be fixed, rather than always appreciating that there’s patients at the end of the clinical……..’(Task Group Member)
The importance of involving health professionals and obtaining “buy in” was a recurring theme as this participant notes:
‘So we reviewed the decisions, the symptoms that we were looking for in light of best evidence and we got consensual agreement on what the level of care was, that whole process, the Airth process got support from Quality Improvement Scotland it was then called Clinical Standards Board for Scotland and I and others promoted this with other clinical leaders and management leaders across the NHS, so we got support for the process the algorithms was XXX the clinicians were bought into this so they made them pretty good in terms of information and what was the knowledge if you like’ (Interviewee 1)
Collective Action
Once again collective action referred completely to constructs contained within the NPM. Examples were provided within the WP2 section and are not repeated here.
Reflexive Monitoring
As in the preceding section reflexive monitoring refers to evaluation issues but professionals often commented on the importance of this in terms of contributing to the further development and reconfiguration of new services. The value of such an approach was emphasised by this respondent:
‘Well, that you don’t make steps by a sudden giant leap, that what you do is that you take what you have and you logically improve it bit by bit. If you look at the modern PC, that wasn’t invented overnight. That has taken about 20 years to develop by a series of logical steps from where it first started to where it is now. And of course, this contrasts entirely with the Connecting for Health strategy where what they assumed was you would construct a model of where you wanted to be and then you would spend a
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 129
huge amount of money in buying that. And you would actually get people to build something that didn’t exist. I think there’s good evidence that that approach has been disastrous because they’ve spent huge amounts of money and they haven’t actually achieved very much.’ (Interviewee 7)
In addition, it was clear that when benefits of new e-Health services were visible to health professionals then they were much more likely to embrace them enthusiastically as this comment illustrates:
‘…..but I think the benefits from it are so immediately obvious and large that actually the little bits of pain that go with it people have tried to work around.’ (Task Group Member)
Thus reflexive monitoring was not simply about how individuals or groups evaluated systems but how their perceptions of the utility of systems affected integration and also how iterative development was possible and could improve the quality of the e-Health services in use.
Outside the NPT
There was very little that fell outside the NPT, mostly it related either to specific statements relating to technology or more often to generic attitudinal statements such as the following:
‘I think there was a perception of, buy the technology and it will happen. We had either begged, borrowed or stolen. We had loan of various pieces of equipment and I think there was a misconception that if you buy the equipment, then the problem is solved. I always said beware of the man who is trying to sell you Telemedicine equipment because he probably does not understand what he is doing.’(Interviewee 12)
7.6 WP3 Analysis using NPT
The data collected in the WP3 surveys indicated some significant findings in relation to the NPT constructs that are worthy of mention. At both the Scottish site (regarding use of CNIS) and the call centre, responses on TARS items were compared using cross-tab analysis for differences between sub-groups of participants rating the e-Health system in question as either not/partly routine or as completely routine. At the Scottish site, responses to all three global NPT items (coherence, cognitive participation and reflexive monitoring) differed according to perceptions of routinisation. For all three items, those indicating that use of the system was only ‘partly routine’ were more likely to disagree with the statements (i.e. respond negatively) compared to the group who saw the system as a completely routine part of their work. This difference was particularly strong for the coherence item (c=10.124, p<.001), suggesting a potentially important association between understanding the system and what it is intended for and seeing it as a routine part of working practice.
In the call centre sample, the same cross-tab analysis indicated significant differences for the coherence and reflexive monitoring items, in that those disagreeing with the statements were more likely to perceive the e-Health
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 130
systems they used as only partly rather than completely routine. In the call centre sample, the cognitive participation item failed to differentiate between groups indicating different levels of perceived routinisation – this may not be unexpected given that staff are employed specifically to use the systems in question, and therefore a certain level of commitment may be presumed from the outset. It is interesting however, that despite mandatory use of e-Health systems in this context, there was still a proportion of staff who did not view the systems as a completely routine part of their working practice (n=37 vs n=174 responding ‘completely routine’). Taken together, these results provide support for inclusion of the constructs of coherence, cognitive participation and reflexive monitoring in the NPT.
This project has explored barriers and facilitators to e-Health implementation by means of:
� Systematic review of the literature (WP 1)
� Semi-structured interviews and task groups with health professionals (WP 2)
� Online surveys of the authors of published literature in this sphere and of health professionals at selected case study sites. (WP3)
� Case studies and semi-structured interviews with e-Health implementers (WP 4)
We have used mixed methods in order to produce recommendations and to develop instruments and tools that are ready for validation and testing by e-Health implementers. We have undertaken this work within 4 work-packages which have been described in detail in the preceding sections. Although the co-applicants have been involved in all of the work-packages, each work-package has been quite distinct, although it is worth noting that WP 2 and 3 shared two case study sites and WP 3 and WP 4 shared one, thus allowing different methods to be used at common sites. Additionally, in WP 1 and 2 we have analysed the same data using two different techniques: conventional thematic analysis and framework analysis.
The common thread to all the work-packages has been our use of the NPM as our theoretical underpinning. All the work-packages have used the NPM to guide and inform data collection and to aid analysis and interpretation. This has linked the work-packages from conception to analysis, and has enabled theoretical triangulation. The following section highlights our key findings and what has been gained by addressing this important issue with different methods, using the NPM as the theoretical model.
7.7 The “Added Value” Provided by the NPM and NPT and a multiple work-package approach.
WP 1 had an initial scoping exercise which was used to inform data collection in work-packages 2, 3 and 4. However, the key element of WP1 was our systematic review of the e-Health implementation literature and the particularly novel element was our use of the NPM as a coding framework for this work. Our systematic review highlighted a range of weaknesses in the literature, both in terms of content and methodology. We undertook a conventional thematic analysis of this literature. In this approach, thematic categories were identified in the course of data analysis. This identified a
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 131
list of barriers and facilitators to implementation that included: conditions prior to implementation; costs; the need for and impact of validation and evaluation; professional attitudes; ease of use of systems; security, confidentiality and standards; education and training; technological and communication issues.
Although our review had highlighted the general methodological flaws in the literature and the lack of depth to many of the issues described, the main gaps, if any, in the literature were not immediately apparent. However, coding using the NPM as a coding frame made it immediately clear that the literature over-emphasised organisational issues. This meant that the literature gives insufficient attention to problems of workability and integration for health professionals working with e-Health technologies ‘on the ground’. This matters because of the important role that systematic reviews play in informing policy debates. Figure 2. pg 43 of Section 1.7 illustrated via a radar plot that the distribution of codes across the model showed how the focus of the literature was upon contextual integration issues – how an organisation resources and delivers an e-Health technology.
This lack of attention to the effects of e-Health services on the interaction between users, the technology and their workplace is a major gap in the evidence base and provides a misleading picture of e-Health implementation barriers and facilitators as it clearly does not provide a holistic view of the processes at play. The NPM therefore served as a guide to conceptualising the literature. Our analysis also demonstrated that there were policy relevant issues such as the importance of evaluation that were not covered by the NPM. Further analysis using the extended model, the NPT, allowed the vast majority of these to be coded and their role in the implementation and integration process more clearly conceptualised and reflected in our recommendations which should be of value to policymakers and e-Health implementers.
Importantly, our analysis using the NPT across both WPs 1 and 2 served to demonstrate the robustness of the NPM as a model as it was clear that the NPM addressed the bulk of important implementation issues. The NPT then captured most other issues, and data which did fall outside the NPT may have done so merely because the issues were framed too generally, for example, having insufficient detail to permit categorisation.
WP2, which involved key informant interviews and task groups, generated a vast amount of qualitative data. This data was again analysed thematically, using NPM as a coding framework. Using the two methods had clear benefits. First, use of a thematic coding approach highlighted immediately the emphasis users placed on the issue of benefits in determining whether to utilise any new e-Health service. It was immediately apparent that evidence of benefits, particularly for patients, would be a major driver for e-Health implementation. This was an issue, notable by its absence from the literature. Coding to the NPM showed that health professionals took a much more holistic view of implementation issues (compared with representation of the problem as evident in the literature review) as was illustrated by the distribution of codes across the NPM in Figure 3, section 4.6, which were fairly evenly distributed.
Thus socio-technical issues received greater attention from health professionals with relational integration issues (which refer to confidence in the system) and accountability issues featuring prominently. This reflects
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 132
the attention given to perceived benefits as an issue of importance as highlighted by the thematic analysis. This also helped us conceptualise our key recommendations, leading us to emphasise the need to communicate a clear rationale for the implementation of any e-Health service, in terms of both professional and patient benefits, to health professionals in order to increase confidence that the effort required to get the system into everyday usage will be worthwhile to themselves and their patients.
There is therefore “added value” gained from undertaking an initial emergent analysis (free coding) in addition to the coding using the NPM and NPT as a framework. This is because our findings demonstrate that although the NPM (and to a greater extent the NPT in the higher order analysis) explained most of the key issues, some themes within the data may have been less clearly conveyed. For example, it was the thematic coding that very clearly highlighted the importance of benefits as an issue of importance for health professionals. Free coding therefore ensured that all important issues were identified from the data, which might have appeared less prominent within the constraint of applying a pre-determined framework for the whole analysis.
WP3 used the constructs of the NPM to inform development of the items for use within TARS, but this too was informed by the scoping review conducted in WP1. Questions for TARS were generated initially by drawing on the NPM (and later added to by the NPT), but were also mapped against the key emergent findings of the scoping review to ensure that key issues concerning the implementation of e-Health as represented in the literature were included in the instrument. This again illustrates how the different work-packages were used to inform one another.
Our expert survey of the instrument showed that the items developed were deemed to be important and conceptually valid, thereby, reinforcing our view of the value of the NPM. The surveys of academics and health professionals, undertaken with TARS, showed interesting differences in ratings of importance attributed to statements reflecting factors within the model that are perceived to influence the routinisation of e-Health. These differences resonated with the differences in perspective noted in WP1 and WP2. The preliminary survey of experts (review authors), showed that Contextual Integration issues were accorded primary importance, reflecting similar results as our systematic literature review (WP1), while this emphasis on contextual integration issues was far less apparent in our surveys of health professionals, again reflecting the findings of our interviews with health professionals in WP2, where they were noted to take a much more holistic view of implementation issues. This triangulation of findings gives us added confidence in the robustness of our findings and the key recommendations which stem from them.
WP4 involved qualitative interviews with key informant implementers across three case study sites, one of which was shared with WP3. The content of the interview guide was informed by the NPM and responses coded against its constructs. We also searched the data for evidence of the degree to which each system studied had normalized, and for data which could not be coded using the NPM. This approach allowed us to determine that the level of normalisation of different e-Health services could be explained by the degree to which they had a positive impact on different issues relating to the four constructs of the NPM.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 133
Our findings have important implications for policy makers and e-Health implementers as we have demonstrated that the NPM is a valuable tool for assessing the likely normalization of a new e-Health initiative. E-Health initiatives that score highly on all four constructs (contextual integration, interactional workability, relational integration and skill set workability), are likely to normalize. While low scores on any one construct should alert policy makers and senior managers to potential difficulties in this area, which need careful consideration during and after implementation. Low scores across all four constructs suggest that the initiative has relatively low likelihood of normalizing successfully, and that some rethink may be needed. In our view this represents a fundamental advance in knowledge in this field.
In addition, the findings from all four work-packages, but particularly WP4, have been used to develop an e-Health implementation toolkit (e-HIT) which can serve as a sensitizing tool for senior managers or other staff who are considering, planning or undertaking an e-Health initiative implementation. The e-Hit was derived from data from our work-packages and as these were all informed by the NPM, the concepts of the model are well represented within it. The theoretical validity of this toolkit came from the NPM, while the literature review and WP4 provided empirical support. Thus confirming the added value of our approach and how triangulation of data has added strength to our findings and helped us to produce concrete deliverables that will be of practical value to those involved in e-Health implementation.
7.7.1 Discussion
This further analysis of WP1 and WP 2 was undertaken because: a) we had noted that there were a number of gaps when using the NPM alone which we believed related to issues of some importance and relevance to policymakers and implementers in particular; b) it provided an opportunity to examine how well the NPT addressed these gaps; and c) it helped us to further conceptualise the barriers and facilitators to the implementation and integration of e-Health services. We believed it would be useful to undertake this work with two quite distinct types of data sets, the first being that arising from a systematic literature review, and the second, qualitative task groups and interviews. Such an approach would allow us to examine whether coding to the NPT would produce similar results when used in this way in quite distinct contexts.
Two key findings from these further analyses were:
� It is clear that the NPM effectively covered most important implementation and integration issues. This is shown clearly by the comparison of results of the analyses from the two datasets.
� The extended NPT addresses implementation problems not covered by NPM and offers a greater level of explanatory sophistication.
Recommendations
Additional recommendations stemming from these analyses are:
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 134
1. Engagement with users of e-Health services prior to and throughout the implementation and integration of new services is essential in order to maximise the potential for normalisation – both in terms of facilitating their understanding of the purpose and benefits of the system (coherence), and preparing them for effective participation in using the system (cognitive participation).
Such engagement should include professional and lay users of such systems and is likely to facilitate more effective uptake and utilisation of e-Health services.
2. Ongoing evaluation and feedback of results of the implementation of e-Health systems (reflexive monitoring) is crucial and should be an integrated part of any e-Health implementation process.
Such an approach is necessary in order to allow iterative development of systems to meet user’s needs and to ensure that users can be confident in the benefits and safety of e-Health services.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 135
8 Conclusions/Key Findings
8.1 Overview
This project has explored barriers and facilitators to e-Health implementation by examining the published literature and exploring the perspectives of health professionals and e-Health implementers. Importantly, we have also developed practical tools for use by those tasked with implementing e-Health services, which represent a fundamental advance. This multi-perspective study of e-Health has major policy implications as we have highlighted barriers to e-Health implementation and identified strategies to promote e-Health implementation. Consequently, our findings should be of value to policymakers, e-Health implementers (health professionals/managers/industry) and researchers. In section 8.1.1 onwards we present our key findings and highlight matters that need to be considered when planning the introduction of new e-Health services. There are numerous illustrations within the full body of this report of the issues raised below.
8.1.1 Implications for Policymakers
1. Development of a simple taxonomy for e-Health
E-Health is a broad term with numerous definitions. Pagliari’s review (Pagliari et al 2005), commissioned by SDO, listed countless definitions for e-Health. A major challenge for policymakers tasked with communicating e-Health strategies to users (both lay and professional),is how best to describe and classify e-Health services in an accessible and easily understood manner. A major task in this project, which dealt with the broad field of e-Health, was to develop a classification of e-Health systems that would be easily understood by study participants. We have therefore created and shown that it is a useful, simple four domain taxonomy of e-Health with clear definitions, which can be used by policymakers to facilitate communication with e-Health users. The four domains of e-Health are:
� Management systems
� Communication systems
� Computerised Decision Support systems
� Information Systems.
Full definitions appear in Section 1.3, Box 1
2. Findings from systematic reviews of e-Health implementation are misleading
Our systematic review of published reviews of e-Health implementation shows that this literature is problematic and misleading because:
� The published literature is not robust, as it is methodologically flawed, so conclusions must be viewed with caution.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 136
� It appears that the fundamental assumptions underpinning research in this area are unsatisfactory in that there is undue focus on purely organisational issues and insufficient attention being paid to problems of workability, that is, the work that health professionals, as individuals or teams, must undertake to make these systems function satisfactorily in practice. There is, for example, little attention given to how e-Health systems affect roles and responsibilities and whether they help or hinder health professionals in completing key tasks.
3. At the present time there is little to be gained by commissioning a further systematic review in this field.
Although the literature in this field has methodological limitations, the findings resonate with two other reviews recently conducted by the co-applicants. Based on this fact, and our knowledge of the paucity of high quality primary literature, we believe it is unlikely that a further review, even if robust, would add anything new to our knowledge in this area at the present time. We would therefore not recommend this as a good use of scare research funds.
4. Monitoring and evaluation should be an integral component of new e-Health Services that are being commissioned.
A great deal of money is being invested in e-Health services, as they are seen as having the potential to address many of the challenges of health care delivery in the world today. However, we live in an era of evidence based medicine and professionals expect that there should be evidence of benefit when new services are introduced. Importantly, lessons should be learned from both successful and unsuccessful e-Health implementations and shared with users in order to increase the chances for successful implementation and prevent wasted investment in repetition of initiatives, which experience shows are unlikely to succeed.
Furthermore, technologies are constantly undergoing development and a key theme from our work is the need for flexibility of systems and the ability to further develop and refine e-Health systems over time based on users’ experiences of the advantages and disadvantages of the systems in practice. To permit such iterative development requires evaluation and feedback to be an integral part of e-Health service delivery. This does not necessarily mean full scale research proposals but rather ongoing evaluation that can feed iteratively into the development and improvement of systems. Thus, policymakers must ensure that when commissioning new e-Health services they also contract for ongoing evaluation and an obligation to undertake iterative development of systems, as necessary based on user feedback. This should increase the likelihood that systems will be responsive to users’ needs and ensure knowledge obtained from current e-Health initiatives can be mobilised and utilised to inform future implementations.
5. The e-HIT is a useful tool which could be used by policymakers to inform the planning of new e-Health initiatives.
The e-HIT (http://www.ucl.ac.uk/pcph/research/ehealth/tools.htm) has not been designed as a research tool but as a means to highlight where areas of potential difficulty may exist with regard to specific planned e-Health
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 137
services. It looks like this tool would act as a sensitising tool that could be of practical value. Policymakers could use this instrument to help them identify potential areas of difficulty for any planned implementation which would then enable them to consider strategies to overcome potential barriers in advance of roll out of services.
6. Enhanced engagement with users, both lay and professional is likely to promote uptake and utilisation.
� Formalised mechanisms need to be put in place to ensure the establishment of an ongoing three way dialogue between designers, implementers and professional users
Although engagement exercises are often undertaken, at present these appear to be less effective than anticipated. This is possibly because engagement is concentrated on the wrong individuals, in particular, managers rather than the users of the systems. Engagement needs to be begun prior to roll out of the new technology to ensure that technology being purchased meets users’ specifications and needs to be ongoing to facilitate iterative development of systems. Ideally, the need for such ongoing interaction should feature in contractual arrangements.
� Communicating a clear rationale for implementing e-Health services.
A strong, yet recurring theme from our research was the need to demonstrate likely benefit and communicate a clear rationale for any e-Health service to users, particularly in relation to potential benefits of these systems. It was slightly surprising to the research team that this remains a problem for professionals on the ground. Although efforts have already been made to communicate benefits of e-Health services we believe it is essential that policymakers invest more money and energy into engaging with users, be they lay or professional, in a very specific manner in advance of implementation of e-Health services in specific contexts to increase the likelihood of future uptake and utilisation. It appears that generic engagement efforts need to be supplemented by targeted, specific local engagement efforts. Unless the benefits of any particular planned e-Health service can be seen to balance the effort required locally to make the system work then integration of new services will remain slow and implementation problematic. Where a new technology is seen to have clear benefits, PACS representing an excellent example, then implementation and normalisation into routine service delivery is far more likely.
7. Safety and reliability e-Health systems must be made transparent to users.
The fact that this remains a problem for professionals on the ground is an unexpected finding. Health professionals implementing any new technology must first and foremost be convinced that it is a “safe” thing to do, and one important aspect of this is knowing that “back up” or “over-ride” systems are in place for immediate use if anything goes wrong and that they can have real confidence in the security of the systems that are coming into play. Many people point to the successful implementation of new technologies within other sectors such as banking, where issues such as confidentially are also of importance, and question why it seems to be so
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 138
much more difficult to get new technologies implemented and integrated into the health care arena. However, a key factor that sets health care apart from other services is the fact that whereas in other sectors system failures may be a nuisance and inconvenience, in health care, system failures can literally have “life or death” consequences. Health professionals have been trained to be acutely aware of the “risk” element of their work and are therefore understandably intolerant of system failure. This may be linked to growing awareness of medico-legal issues. However, it is not just the well being of patients that is a major concern but also the safety of patient data which also appears to be a greater problem than in other sectors and may be due to the peculiarly sensitive nature of medical information. If professionals are not confident in the safety and reliability of e-Health systems uptake and utilisation will be resisted. While policymakers are likely to be cognisant of this as an issue, it is clear that this concern is still not being addressed to the satisfaction of health professionals and continues to serve as a barrier to utilisation. It is also noteworthy that there is likely to be variation between types of e-Health systems (management, communication, computerised decision support, information systems), with regard to the extent to which this is viewed as a cause for concern.
8. System compatibility
� There is a Need to Establish a Balance Between Individual Requirements and Standardisation
It is well accepted that standards and interoperability are important when introducing new e-Health systems in order to facilitate communication between sectors and data sharing. However, it is important to note that within the NHS, users often have quite different experiences and backgrounds when it comes to e-Health systems. Some are already quite advanced and experienced users, with well established systems in place, while others are relatively inexperienced. This creates a tension when trying to implement “universal” systems which needs to be recognised and addressed by policymakers. This is most likely to be most effectively addressed by increased engagement with users as outlined in recommendation 6 above.
9. Tensions between national and local policy priorities
� In order to increase the chances for successful implementation of any new e-Health service it is important that local and national policies and priorities are congruent.
It is clear that if national and local priorities match then this will promote successful implementation whereas the converse is true. Policymakers need to acknowledge this problem and ensure efforts are made to minimise conflicting strategies when trying to implement e-Health systems otherwise it is unlikely that an e-Health service will have sufficient local resources and support made available to permit integration into routine service delivery.
8.1.2 Implications for e-Health implementers.
As mentioned in section 8.1.1, within this project we have developed an e-Health implementation toolkit, the e-HIT, which we believe could serve as a useful tool for those planning to implement e-Health services. In particular, we would like to highlight the following issues that merit consideration by e-
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 139
Health implementers and which can be addressed in greater detail through use of the e-HIT:
1. The overall context
This includes;
� National priorities and strategies
� Local priorities
� Leadership
� Resources.
In order to increase the chances for successful implementation of any new e-Health service it is important that local and national policies and priorities are congruent and that there is senior leadership, both managerial and health professional, supporting the initiative. In addition, adequate resources, both financial and personnel, are essential in order to facilitate implementation of e-Health services. For example, without adequate initial and ongoing IT support e-Health initiatives will struggle to become integrated as part of routine service delivery. It is also important that implementation and integration is seen as a long term and ongoing activity rather than a short term activity as implementation of new services will invariably require much more time and effort than initially envisaged.
2. The nature of the e-Health initiative
In particular it is important that consideration is given to:
� User-friendliness
� Effects, if any, on Clinical Interactions
� Effects, if any, on Inter-professional relationships
Our research has demonstrated that the ease of use of systems, and the extent to which systems help professionals to complete tasks efficiently and effectively impacts greatly on implementation. Systems which hinder the smooth and efficient delivery of care and which are deemed unreliable or insecure will not be welcomed and are less likely to be successfully implemented. It is therefore important that contracts with suppliers include a substantial element of ongoing support so that difficulties encountered by users can be addressed as part of the implementation process.
3. The impact of the implementation on the workforce
This includes attention to a broad range of issues that includes:
� Workload
� Training needs
� Alterations in roles and/or responsibilities.
Education, training and ongoing support is crucial. Such training should not be simply focused on how to use the system but also should address limitations of the technology and how to optimise potential benefits of the technology. E-Health implementers need to try to anticipate likely effects on workload and roles and responsibilities and sure these issues are being examined and addressed throughout the implementation process.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 140
8.1.3 Implications for Research
Our work has demonstrated that the evidence base in this area is weak. We therefore highlight the following areas for research.
1. Primary e-Health research to examine implementation processes
In particular there is a pressing need to address the following research questions:
� What are the key steps in any implementation process?
� How can these key steps or processes be classified and what is their relative contribution to implementation?
� How can researchers help e-Health implementers to predict which services will or will not become normalised into everyday practice? .
Our work has demonstrated that the evidence base in the area of e-Health implementation research is unexpectedly weak. Barriers and facilitators to e-Health implementation have been identified but their relative importance remains uncertain. For example, it is unclear whether lack of engagement or champions is outweighed by the presence of adequate funding and congruent local and national policies or vice versa. There is therefore a need for studies that examine the implementation process in a much more detailed and thorough manner.
Crucially, this project has led to the development of robust tools and instruments which can be utilised by researchers, e-Health implementers and policy makers when considering and/or introducing e-Health services. We have developed a technology adoption readiness scale (TARS) as part of WP3 and an e-Health implementation toolkit (e-HIT) as part of WP4. These developments represent a potentially fundamental advance in this area of research as these tools and instruments have strong conceptual underpinnings and have been developed through the rigorous analysis of an extremely broad range of data. We would suggest that further research be commissioned to enable:
2. Testing of the general and predictive utility of e-HIT
It would be useful to conduct longitudinal studies of e-HIT in contexts where assessment of perceptions can be undertaken prior to the introduction of an e-Health system in order to determine the true potential value of e-HIT.
3. The development of different versions of the Technology Adoption Readiness Scale (TARS)
� It would be useful to explore and develop different ways of wording TARS items for different purposes
� To create guidance for its use for different purposes and in different contexts.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 141
4. Testing of the statistical properties of TARS as a research scale
� It will be important to test TARS, in contexts where the necessary requirements concerning sample sizes and response rates can be met.
Although not the focus of this study, our work has clearly contributed significantly to the growing portfolio of studies suggesting the value of the NPM and of the extended NPT as conceptual models. The NPM and NPT have demonstrated in this study that they can be used to aid understanding of the implementation of e-Health services and have explanatory value in terms of clarifying why e-Health services have or have not been successfully implemented. The NPT obviously has implications as a theory beyond the context of e-Health and also further potential specifically in the area of e-Health. We would therefore recommend that further research is needed to:
5. Examine the predictive power of the NPM and NPT
It would be valuable to determine to what extent the NPM and NPT can be proved useful as a means of predicting whether any particular e-Health implementation will or will not be successful.
6. To examine the wider utility of the NPT.
It will be important to explore how useful the NPT is as a theoretical model beyond the field of e-Health. In particular, it will be important to explore what role this theory is able to play in addressing a full range of complex interventions and policies.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 142
References
Al-Qirim NA. 2005. Critical success factors for strategic telemedicine planning in New Zealand. Telemed J E Health. 11(5): 600-7.
Anderson JG, Aydin CE. 1997. Evaluating the impact of health care information systems. Int J Technol Assess Health Care. 13(2): 380-93.
Anderson JG. 2000. Computer-based ambulatory information systems: recent developments. J Ambul Care Manage. 23(2): 53-63.
Anderson JG. 2007. Social, ethical and legal barriers to e-health. Int J Med Inform. 76(5-6): 480-3.
Angood PB. 2001. Telemedicine, the Internet, and World Wide Web: overview, current status, and relevance to surgeons. World J Surg. 25(11): 1449-57.
Anonymous [No authors listed]. 1999. Telemedicine: an overview. Health Devices. 28(3):88-103.
Barry CA, Britten N, Barber N, Bradley C, Stevenson F. 1999. Using reflexivity to optimize teamwork in qualitative research. Qual Health Res. 9(1):26-44.
Becker, H S, Hughes, E C, Geer, B, and Strauss, A.L 1961. Boys in White: Student Culture in Medical School. Chicago: University of Chicago Press.
Bell J. 2007. PCTs still only half way to choose and book target. Health Serv J. 117(6050):9.
Bond GE. 2006. Lessons learned from the implementation of a Web-based nursing intervention. Comput Inform Nurs. 24(2): 66-74.
Broens, T. H. 2007. Determinants of Successful Telemedicine Implementations: a Literature Study. Journal of telemedicine and telecare. 13, 303-309.
Campbell, J. D., Harris, K. D., & Hodge, R. 2001. Introducing telemedicine technology to rural physicians and settings. Journal of Family Practice. 50(5), 419-424.
Carrino JA, Unkel PJ, Miller ID, Bowser CL, Freckleton MW, Johnson TG. 1998. Large-scale PACS implementation. J Digit Imaging. 11(3 Suppl 1): 3-7.
Chaudhry, B. 2006. Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care. Annals of internal medicine. 144, 742-752.
Childs S, Blenkinsopp E, Hall A, Walton G. 2005. Effective e-learning for health professionals and students--barriers and their solutions. A systematic review of the literature--findings from the HeXL project. Health Info Libr J. 22 Suppl 2:20-32.
Collin S, Reeves BC, Hendy J, Fulop N, Hutchings A, Priedane E. 2008. Implementation of computerised physician order entry (CPOE) and picture archiving and communication systems (PACS) in the NHS: quantitative before and after study. BMJ. 337:a939.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 143
Cook DA, Dupras DM. 2004. A practical guide to developing effective web-based learning. J Gen Intern Med. 19(6):698-707.
Copeland M. 2002. e-Community health nursing. J Holist Nurs. 20(2):152-65.
De Lusignan S. 2005. The barriers to clinical coding in general practice: a literature review. Med Inform Internet Med. 30(2):89-97.
Department of Health. 2000. The NHS Plan - A plan for investment a plan for reform. Department of Health.
Department of Health. 2001. Building a Safer NHS for Patients: Implementing an organisation with a memory. Department of Health.
Department of Health. 2002a. Delivering the NHS Plan: The next steps on investment, next steps on reform. Department of Health.
Department of Health. 2002b. Delivering 21st Century IT Support for the NHS: National Strategic Programme. http://www.dh.gov.uk/PublicationsAndStatistics/Publications/. Department of Health.
Department of health 2004, Improving Chronic Disease management. Department of Health, London.
Dinh M, Chu M. 2006. Evolution of health information management and information technology in emergency medicine. Emerg Med Australas. 18(3):289-94.
Dixon -Woods M, Agarwal S, Young B, Jones D, Sutton A. 2004. Integrative approaches to qualitative and quantitative evidence. NHS Health Development Agency, Leeds.
Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. 2005. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 10[1], 45-53.
Elford DR. 1997. Telemedicine in northern Norway. J Telemed Telecare. 3(1):1-22.
Elwyn, G., et al. 2008. Arduous implementation: does the normalisation process model explain why it is so difficult to embed decision support technologies in routine clinical practice. Implement Sci. 31;3:57.
Eng TR. 2001. The e-Health Landscape: A Terrain Map of Emerging Information and Communication Technologies in Health and Health Care. The Robert Wood Johnson Foundation.
EU Commission. 2004. Health – Making Healthcare Better for European Citizens: An Action Plan for a European e-Health Area, EU Communication, COM 356. EU Commission.
EU Commission. 2005. i2010 - A European Information Society for growth and employment. EU Commission.
Falas T, Papadopoulos G, Stafylopatis A. 2003. A review of decision support systems in telecare. J Med Syst. 27(4):347-56.
Garritty C, El EK. 2006. Who's using PDAs? Estimates of PDA use by health care providers: a systematic review of surveys. J Med Internet Res. 8(2):e7.
Gask, L., et al. 2008. Beyond the limits of clinical governance: the case of mental health in primary care. BMC Health Services Research. Vol. 8.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 144
Geibert RC. 2006. Using diffusion of innovation concepts to enhance implementation of an electronic health record to support evidence-based practice. Nurs Adm Q. 30(3):203-10.
Goldberg MA, Dwyer SJ 3rd. 1995. Telemammography: implementation issues. Telemed J. 1(3):215-26.
Grams RR, Moyer EH. 1997. The search for the elusive electronic medical record system--medical liability, the missing factor. J Med Syst. 21(1):1-10.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. 2004. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly. 82:581-629.
Guler NF, Ubeyli ED. 2002. Theory and applications of telemedicine. J Med Syst. 26(3):199-220.
Hanson CW, Marshall BE. 2001. Artificial intelligence applications in the intensive care unit. Crit Care Med. 29(2):427-35.
Hebert, M. A. 2006. Moving Research into Practice: A Decision Framework for Integrating Home Telehealth into Chronic Illness Care. International journal of medical informatics. 75, 786-794.
Hebert, M. A., Paquin, M. J., & Iversen, S. 2002. Predicting success: stakeholder readiness for home telecare diabetic support. Journal of Telemedicine and telecare. 8, 33-36.
Hendy J, Fulop N, Reeves BC, Hutchings A, Collin S. 2007. Implementing the NHS information technology programme: qualitative study of progress in acute trusts. BMJ. 334(7608):1360.
Hilty, D. M. 2002. Telepsychiatry: an Overview for Psychiatrists. CNS-drugs. 16, 527-548.
Houtchens BA, Allen A, Clemmer TP, Lindberg DA, Pedersen S. 1995. Telemedicine protocols and standards: development and implementation. J Med Syst. 19(2):93-119.
Hunt RC. 2002. Emerging communication technologies in emergency medical services. Prehosp Emerg Care. 6(1):131-6.
Hussein R, Engelmann U, Schroeter A, Meinzer HP. 2004. DICOM structured reporting: Part 2. Problems and challenges in implementation for PACS workstations. Radiographics. 24(3):897-909.
Jaatinen PT, Forsstrom J, Loula P. 2002. Teleconsultations: who uses them and how? J Telemed Telecare. 8(6):319-24.
Jadad AR. 2002. Evidence-based decision making and asthma in the internet age: the tools of the trade. Allergy. 57 Suppl 74:15-22.
Jennett P, Watanabe M, Igras E, Premkumar K, Hall W. 1996. Telemedicine and security. Confidentiality, integrity, and availability: a Canadian perspective. Stud Health Technol Inform. 29:286-98.
Jennett PA, Andruchuk K. 2001. Telehealth: 'real life' implementation issues. Comput Methods Programs Biomed. 64(3):169-74
Jennett, P., Jackson, A., Healy, T., Ho, K., Kazanjian, A., Woollard, R., et al. 2003a. A study of a rural community's readiness for tele-Health. Journal of Telemedicine and telecare. 9(5), 259-263.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 145
Jennett, P., Yeo, M., Pauls, M., & Graham, J. 2003b. Organizational readiness for telemedicine: implications for success and failure. Journal of Telemedicine and telecare. 9, S27-S30.
Jennett PA, Scott RE, Affleck Hall L, Hailey D, Ohinmaa A, Anderson C, et al. 2004. Policy implications associated with the socioeconomic and health system impact of telehealth: a case study from Canada. Telemed J E Health. 10(1):77-83.
Jennett, P. A. 2005. Preparing for Success: Readiness Models for Rural Telehealth. Journal of postgraduate medicine. 51, 279-285.
Jerant AF. 1999. Training residents in medical informatics. Fam Med. 31(7):465-72.
Johnson, K. B. 2001. Barriers That Impede the Adoption of Pediatric Information Technology. Archives of pediatrics and adolescent medicine. 155, 1374-1379.
Kaplan B, Shaw NT. 2004. Future directions in evaluation research: people, organizational, and social issues. Methods Inf Med. 43(3):215-31.
Karlsson D, Forsum U. 2004. Medical decision-support systems and the concept of context. Med Inform Internet Med. 29(2):109-18.
Katz AS, Tilkemeier PL. 1997. Multimedia image display: a view to the future. Curr Opin Cardiol. 12(6):566-70.
Kawamoto, K. 2005. Improving Clinical Practice Using Clinical Decision Support Systems: A Systematic Review of Trials to Identify Features Critical to Success. British Medical Journal. 330, 765-768.
Kukafka, R. 2003. Grounding a New Information Technology Implementation Framework in Behavioral Science: a Systematic Analysis of the Literature on IT Use. Journal of biomedical informatics. 36, 218-227.
Kvale, S. 1996. InterViews. London: Sage.
Lapointe L, Rivard S. 2006. Getting physicians to accept new information technology: insights from case studies. CMAJ. 23;174(11):1573-8.
Lawrenson R, Williams T, Farmer R. 1999. Clinical information for research; the use of general practice databases. J Public Health Med. 21(3):299-304.
Leatt, P. 2006. IT Solutions for Patient Safety--Best Practices for Successful Implementation in Healthcare. Healthcare quarterly Toronto Ont. 9, 94-104.
Lehman, W. E. K., Greener, J. M., & Simpson, D. D. 2002. Assessing organizational readiness for change. Journal of Substance Abuse Treatment. 22(4), 197-209.
Lehmann ED. 2004. Computerised decision-support tools in diabetes care: hurdles to implementation. Diabetes Technol Ther. 6(3):422-9.
Liaskos J, Mantas J. 2002. Nursing Information System. Stud Health Technol Inform. 65:258-65.
Link RE, Schulam PG, Kavoussi LR. 2001. Telesurgery. Remote monitoring and assistance during laparoscopy. Urol Clin North Am. 28(1):177-88.
Implementation research: state of the art and future directions
JD Linton. 2002. Technovation. Elsevier.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 146
Lu YC, Xiao Y, Sears A, Jacko JA. 2005. A review and a framework of handheld computer adoption in healthcare. Int J Med Inform. 74(5):409-422.
Lu YC, Xiao Y, Mills ME, Soeken K, Vaidya V. 2006. Top barriers and facilitators to nurses' PDA adoption. AMIA Annu Symp Proc. 1016.
Mackinnon AD, Billington RA, Adam EJ, Dundas DD, Patel U. 2008. Picture archiving and communication systems lead to sustained improvements in reporting times and productivity: results of a 5-year audit. Clin Radiol. 63(7):796-804.
Maglogiannis I. 2004. Design and implementation of a calibrated store and forward imaging system for teledermatology. J Med Syst. 28(5):455-67.
Mair, F. S. 2008. Understanding factors that inhibit or promote the utilization of telecare in chronic lung disease. Chronic Illness. 4, 110-117.
Mairinger T. 2000. Acceptance of telepathology in daily practice. Anal Cell Pathol. 21(3-4):135-40. 40
Mandl KD, Kohane IS, Brandt AM. 1998. Electronic patient-physician communication: problems and promise. Ann Intern Med. 15;129(6):495-500.
Maulden SA. 2003. Information technology, the internet, and the future of neurology. Neurologist. 9(3):149-59.
May, C.; Gask, L.; Atkinson, T.; Ellis, N.; Mair, F.; Esmail, A. 2001a. Resisting and Promoting New Technologies in Clinical Practice: the Case of Telepsychiatry. Social Science & Medicine. 52, 1889-1901.
May, C., Ellis, N. T. 2001b. When protocols fail: technical evaluation, biomedical knowledge, and the social production of 'facts' about a telemedicine clinic. Social Science and Medicine. 53, 989-1002.
May, C.; Mort, M.; Williams, T.; Mair, F.; Gask, L. 2003a. Health Technology Assessment in Its Local Contexts: Studies of Telehealthcare. Social Science & Medicine. 57, 697-710.
May, C.; Harrison, R.; Finch, T.; MacFarlane, A.; Mair, F.; Wallace, P. 2003b. Understanding the Normalization of Telemedicine Services Through Qualitative Evaluation. Journal of the American Medical Informatics Association. 10, 596-604.
May, C.; Allison, G.; Chapple, A.; Chew-Graham, C.; Dixon, C.; Gask, L., et al. 2004. Framing the Doctor-Patient Relationship in Chronic Illness: a Comparative Study of General Practitioners' Accounts. Sociology of Health & Illness. 26, 135-158.
May, C. 2005. Chronic illness and intractability: professional-patient interactions in primary care. Chronic Illness. 1, 15-20.
May, C.; Rapley, T.; Moreira, T.; Finch, T.; Heaven, B. 2006. Technogovernance: Evidence, Subjectivity, and the Clinical Encounter in Primary Care Medicine. Social Science & Medicine. 62, 1022-1030.
May C. 2006 A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res. 6:86:86.
May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M et al. 2007a. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res. 19;7(1):148.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 147
May, C., et al. 2007b. Process evaluation of complex interventions in primary care: understanding trials using the normalization process model. BMC Family Practice. 8.
May, C. 2007. The clinical encounter and the problem of context. Sociology. 41, 29-45.
May, C., Finch, T. 2009. Implementation, embedding, and integration: an outline of Normalization Process Theory. In Press.
May, C., Finch, T. L., Forthcoming. Normalizing Health Technologies. Palgrave, London.
McDonald CJ. 1997. The barriers to electronic medical record systems and how to overcome them. J Am Med Inform Assoc. 4(3):213-21.
Mort, M.; May, C. R.; Williams, T. 2003. Remote Doctors and Absent Patients: Acting at a Distance in Telemedicine? Science Technology & Human Values. 28, 274-295.
Mosley-Williams A, Williams C. 2005. Computer applications in clinical practice. Curr Opin Rheumatol. 17(2):124-8
Mun SK, Elsayed AM, Tohme WG, Wu YC. 1995. Teleradiology/telepathology requirements and implementation. J Med Syst. 19(2):153-64.
NHS Executive. 1998. Information for Health: An information strategy for the modern NHS 1998–2005. September 1998. NHS Executive. http://www.nhsia.nhs.uk/def/pages/info4health/contents.asp.
NHS Executive. 2001. Building the Information Core – Implementing the NHS Plan. NHS Executive
Nolan A, Whitfield L. 2005. Choose and book. By the book. Health Serv J. 115(5947):suppl-6.
Ohinmaa, A. 2006. What Lessons Can Be Learned From Telemedicine Programmes in Other Countries? Journal of telemedicine and telecare. 12, S40-S44.
Oliver, D. R. P., & Demiris, G. 2004. An assessment of the readiness of hospice organizations to accept technological innovation. Journal of Telemedicine and telecare. 10(3), 170-174.
Overhage, J. M., Evans, L., & Marchibroda, J. 2005. Communities' readiness for health information exchange: The national landscape in 2004. Journal of the American Medical Informatics Association. 12(2), 107-112.
Pagliari C, Sloan D, Gregor P, Sullivan F, Detmer D, Kahan JP, et al. 2005. What Is eHealth (4): A Scoping Exercise to Map the Field. J Med Internet Res. 7(1):e9.
Papshev, D. 2001. Electronic Prescribing in Ambulatory Practice: Promises, Pitfalls, and Potential Solutions. The American journal of managed care. 7, 725-736.
Peleg, M. 2006. Decision Support, Knowledge Representation and Management in Medicine. Yearb-Med-Inform.
Pellegrino L, Kobb R. 2005. Skill sets for the home telehealth practitioner: a recipe for success. Telemed J E Health. 11(2):151-6.
Pothier DD, Awad Z, Tierney P. 2006. 'Choose and Book' in ENT: the GP perspective. J Laryngol Otol. 120(3):222-225.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 148
Rogers, E. M. 1995. The diffusion of innovation. New York: Free Press.
Rudowski R. 2003. Telemedicine in the context of different medical specialities. The Polish perspective. Pol J Pathol. 54(3):219-21.
Sable C. 2001. Telecardiology: potential impact on acute care. Crit Care Med. 29(8 Suppl):N159-65
Safran C, Goldberg H. 2000. Electronic patient records and the impact of the Internet. Int J Med Inform. 60(2):77-83.
Shekelle, P. G. 2006. Costs and Benefits of Health Information Technology. Evidence report technology assessment.
Sheng OR, Hu PJ, Au G, Higa K, Wei CP. 1997. Urban teleradiology in Hong Kong. J Telemed Telecare. 3(2):71-7.
Shiffman RN, Liaw Y, Brandt CA, Corb GJ. 1999. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc. 6(2):104-14.
Silverman, D. 1993. Interpreting Qualitative Data. London: Sage.
Snyder-Halpern, R. 2002. Development and pilot testing of an Organizational Information Technology/systems Innovation Readiness Scale (OITIRS). Proceedings of the AMIA 2002 Annual Symposium. 702-706.
Stanberry B. 2000. Telemedicine: barriers and opportunities in the 21st century. J Intern Med. 247(6):615-28.
Strauss, A. 1987. Qualitative analysis for social scientists. Cambridge University Press, Cambridge.
Studer, M. 2005. The Effect of Organizational Factors on the Effectiveness of EMR System Implementation--What Have We Learned? Healthcare-quarterly-Toronto,-Ont. 8, 92-98.
Styra R. 2004. The Internet's impact on the practice of psychiatry. Can J Psychiatry. 49(1):5-11.
Swinglehurst DA. 2005. Information needs of United Kingdom primary care clinicians. Health Info Libr J. 22(3):196-204.
Tierney WM. 2001. Improving clinical decisions and outcomes with information: a review. Int J Med Inform. 62(1):1-9.
Uhlenhopp MB, Fliedner MC, Morris P, Van Boxtel T. 1998. A global perspective on nurses' Internet access and information utilization. Oncol Nurs Forum. 25(10 Suppl):27-32.
US National Institute of Medicine. 2000. To Err to Human: Building a safer health system. Academy Press.
Van de WR, Batenburg R, Versendaal J, Lederman R, Firth L. 2006. A balanced evaluation perspective: picture archiving and communication system impacts on hospital workflow. J Digit Imaging. 19 Suppl 1:10-7.:10-17.
Van de WR, Batenburg R. 2008. A PACS maturity model: A systematic meta-analytic review on maturation and evolvability of PACS in the hospital enterprise. Int J Med Inform.
Vreeman, D. J.; Taggard, S. L.; Rhine, M. D.; Worrell, T. W. 2006. Evidence for Electronic Health Record Systems in Physical Therapy... Including
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 149
Commentary by Zimny NJ With Author Response. Physical Therapy. 86, 434-449.
Wallace S, Wyatt J, Taylor P. 1998. Telemedicine in the NHS for the millennium and beyond. Postgrad Med J. 74(878):721-8.
Wanless D. 2002. Securing Our Future Health: Taking a long-term view re-investment priorities. http://www.hm-treasury.gov.uk/ Consultations_and_Legislation/wanless/consult_wanless_final.cfm
Wanless, D. 2004. Securing Good Health for the Whole Population. February 2004. HM Treasury.
Weinstein RS, Descour MR, Liang C, Bhattacharyya AK, Graham AR, Davis JR, et al. 2001. Telepathology overview: from concept to implementation. Hum Pathol. 32(12):1283-99/
Whitten P, Love B. 2005. Patient and provider satisfaction with the use of telemedicine: overview and rationale for cautious enthusiasm. J Postgrad Med. 51(4):294-300.
WHO. 2003. Information sheet: facts related to chronic diseases and the
WHO global strategy on diet, physical activity and health [pdf 260kb] .
World Health Organisation.
Wilkes, S. 2007. Evaluation of open access hysterosalpingography in the initial management of infertility in primary care. Centre for Primary & Community Care, Vol. Ph.D. University of Sunderland.
Wootton R. 2001. Telemedicine and developing countries--successful implementation will require a shared approach. J Telemed Telecare. 7 Suppl 1:1-6.
Yarbrough, A. K. 2007. Technology Acceptance Among Physicians: A New Take on TAM. Medical-Care-Research-and-Review. 64, 650-672.
Yellowlees PM, Kennedy C. 1997. Telemedicine; here to stay. Med J Aust. 166(5):262-5.
Yellowlees PM. 2005. Successfully developing a telemedicine system. J
Telemed Telecare. 11(7):331-5
Yusof, M. M. 2007. Health Information Systems Adoption: Findings From a Systematic Review. Medinfo. 12, 262-266.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 150
Appendix 1 NPM Definitions
Interactional workability
The operationalization of a complex intervention in the immediate conditions
in which professionals and patients encounter each other [1]
One way of thinking about interactional workability (IW) is thinking about
the work that goes on in an interaction – in this case the interaction
between a health professional and a patient. In other words, IW refers to
the impact of the new intervention or technology on the work done in a
consultation.
There are two parts (or dimensions) to the construct of interactional
workability. The first, “congruence”, is concerned with the interaction itself:
what should legitimately be dealt with in an interaction (e.g. a
consultation), what is the form of the work to be done, what is the role of
each participant in the interaction, how is the work to be completed in the
time and space available, and the formal and informal rules that govern the
verbal and non-verbal conduct of an interaction. The second, called
“disposal” concerns the effects of the interaction. Disposal considers the
goals of an interaction – e.g. forming a diagnosis, agreeing actions such as
investigations, treatment or follow-up, recording information – and how
disagreement about the outcome of the work is minimised, together with
shared beliefs about the meaning and consequence of the work.
Relational Integration
The mediation of knowledge and practice about a complex intervention in
the network of relations in which clinical encounters between professionals
and patients are located [1]
For relational integration (RI), think about the organization of work and
knowledge around the clinical encounter. The key issue here is whether
those involved in the implementation of a new technology or intervention
trust each other and the work that they are doing individually or as a group.
Do they believe that there is valid knowledge and expertise in the network
of actors for the implementation of the new technology?
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 151
There are two parts (or dimensions) to RI. The first is accountability. Is
there agreement between those involved in the clinical encounter about the
forms and validity of knowledge associated with the new work? Does the
new working practice embody what clinicians personally regard as valid
(clinical) knowledge, as appropriate expertise and appropriate sources of
that expertise? [1]
The second is confidence. Is there agreement about sources of authoritative
knowledge and practice? What criteria are used to assess credibility? Does
the new technology or intervention disrupt beliefs about the practical utility
of knowledge that is mediated through the network?
Skill Set Workability
The mechanisms by which knowledge and practice about complex
interventions are distributed and performed in the formal and informal
divisions of labour in health care settings [1]
Skill set workability (SSW) relates to institutional framing of work and
divisions of labour. It is concerned with the skills that are involved in the
implementation of a new technology or intervention. The key questions are
who needs to do what to implement the new technology or intervention and
to what extent is that work compatible with existing professional roles and
identities? SSW is influenced by both formal and informal policies and
practices.
The first dimension of SSW is allocation. This is about the extent to which
the new technology or intervention requires a shift from existing skills and
the organization of those skills within and among professional groups. Are
the skills needed for a new technology compatible with the current skill base
and division of labour or does the new technology require change and
negotiation around who should do what to complete the work involved?
The second dimension of SSW is Performance. Does the work around a new
technology push boundaries between professional groups and their skill sets
and/or does it affect the levels of autonomy that professional are used to
and require for their work.
Contextual Integration
The capacity of an organisation to understand and agree the allocation of
control and infra-structure resources to implementing a complex
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 152
intervention, and to negotiating its integration into existing patterns of
activity[1].
Contextual integration (CI) refers to the work that needs to be done at the
level of the organisation in order for the complex intervention to become
normalised. There are at least three components that need considering: the
organisational commitment to deploying and sustaining a new intervention;
the capacity of the organisation to do this; and the value that accrues (or is
expected to accrue) to the organisation as a result of deployment.
An organisation may be committed to implementing a new technology or
way of working because this has been mandated from on high – for
example, many of the Connecting for Health targets have been imposed on
trusts, and whether they like it or not, they have to implement them, and
reach pre-determined targets in terms of implementation and use.
Alternatively, an organisation may have reached a local decision, and want
to implement a new way of working or new technology because they think it
will help them reach locally agreed targets or goals. Sometimes there will
be an overlap – a centrally mandated technology may help meet a locally
determined goal. But if there is no commitment on the part of the
organisation to implementation, it is highly unlikely that the innovation will
be implemented across the organisation, even if a few individual enthusiasts
adopt it.
Whatever the level of commitment to the innovation, implementation will
only be possible if the organisation has the capacity to implement. For
example, a hospital trust may decide to implement electronic prescribing,
but be unable to, as there are no computers on the wards, no budget to
purchase computers, and even if the budget were available, no space to put
the computers. Further examples of capacity would include the availability
of IT support to install and maintain the computers, and either a workforce
already trained in the use of the software, or the ability of the organisation
to ensure that the workforce receives training.
A third component of contextual integration is value that accrues to the
organisation as a result of deploying (and sustaining) an innovation. This
value may be financial, in terms of enhanced efficiency or increased
revenue; motivational – for example by increasing status; or around
delivery – for example by improving service delivery, performance or
adherence of staff to agreed operating procedures or protocols.
There are two domains to CI: execution and realisation. Execution refers to
the ownership of control over the resources and agents required to
implement a complex intervention, and includes three components. These
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 153
are: resourcing, power; and evaluation. Resourcing refers to the ability of
the organisation to direct adequate resources to an implementation – the
less an innovation disrupts existing flows of resources (which may be time
or money), the easier it will be to normalise. Power is about the ability of
organisations, or sub-groups within an organisation, to control resources.
Evaluation refers to the ability of the organisation (or sub-groups within an
organisation) to evaluate the work that has been done (or not done).
Realisation also has three components: risk, action, and value. Risk is
about the disruption to current working patterns and systems. Action is
about how decisions are implemented, while value is about perceptions of
the worth of the work that is required relative to the value of the innovation.
Reference List
1. May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M et al.:
Understanding the implementation of complex interventions in health care:
the normalization process model. BMC Health Serv Res 2007, 19;7: 148.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 154
Appendix 2 List of papers included in WP1, Phase 1 scoping review
Authors Year Country of origin
Type of health care system
Type of review Details of methods and results (if applicable)
Key themes Domain(s)
Al-Qirim
NA
2005 New
Zealand
Public. Review of the
health strategy of
the New Zealand
government, with
reference to
appropriate
literature.
None stated,
but draws on
a very
specific and
small body
of literature.
Cost, co-
operation,
security,
confidentiality
and leadership.
Communication
systems.
Anderson
JG, Aydin
CE
1997 United
States.
American
(insurance
based).
Narrative review of
literature and
systems.
None
specified.
Barriers may
be reduced
through
strategic
planning,
evaluation,
designated
authority and
leadership,
making
changes
incrementally
(one step at a
time), allowing
time to adjust,
identifying
benefits for all
concerned,
finding
sponsorship,
training,
managed
expectations,
and a focus on
communicatio
n.
Information
systems.
Anderson
JG
2000 United
States.
American
(insurance
based).
Narrative review. Notes it is
not an
exhaustive
review;
Acceptance of
new systems,
data security,
need for
Information
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 155
review;
articles
selected to
convey the
breath of
computer-
based
ambulance
information
systems.
need for
standards,
cost.
Anderson
JG.
2006 United
States
(also
applies to
Europe,
Canada,
Australia
and New
Zealand).
Mixed. Narrative. review. Literature
analysis and
survey data
from
primary care
physicians
on adoption
of
information
technologies
is reviewed.
Barriers may
be removed
through better
funding,
government
incentives,
standardization
of systems,
improved
security and
legislation.
Generic.
Angood PB 2001 United
States.
Mixed, but
chiefly
focused on
American
(insurance
based).
Narrative review
on the use of
telemedicine,
illustrated with a
case-study of its
use on Mount
Everest.
None
specified.
Barriers:
licensing and
reimbursement
issues,
liability, mixed
quality of
services and
technical
issues.
Facilitator: It
can be used on
Mount
Everest!
Communication
systems.
Bond GE 2006 United
States.
American
(insurance
based).
Review the
development and
implementation of
a web-based
diabetes
information system.
None
specified.
Description
of process.
Centred on
training needs.
Information
systems.
Carrino JA, 1998 United American Review of PACS None Design of Communication
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 156
Unkel PJ,
Miller ID ,
Bowser CL,
Freckleton
MW,
Johnson TG
States. (insurance
based).
implementation in
the US Department
of Defense and
applicability to
public health
service
implementation.
specified. system (ease of
use),
promotion of
system and
education.
systems.
Childs S,
Blenkinsopp
E, Hall A,
Walton G
2005 United
Kingdom.
Public
(NHS).
Systematic
literature review.
Systematic
data-base
search and
articles
screened by
3
researchers
with strict
inclusion
criteria. 57
of an
original 161.
Costs, poor
design,
inadequate
technology,
lack of skills,
need for face-
to-face
interaction,
time, intensity,
computer
anxiety
(general
resistance).
Information
systems.
Cook DA,
Dupras DM
2004 United
States.
American
(insurance
based).
Narrative review. None
specified.
Preparatory
needs analysis,
securing
commitment
from all
concerned,
appropriate
design, ease of
use,
evaluation,
piloting,
ongoing
problem-
solving
(continuous
implementatio
n/
integration).
Information
systems.
Copeland M 2002 United
States.
Not specific
(focused on
the internet
Theoretical article
including a
narrative literature
Online
search of
university
Barriers:
perceived ease
of use and
Information
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 157
as a global
phenomenon
).
review. library using
ProQuest,
excluding
none-
gendered
articles.
gendered
patterns of
communicatio
n.
de Lusignan
S
2005 United
Kingdom.
Public
(NHS).
Systematic
literature review.
Medline and
departmental
website
searches.
Clinicians’
attitudes, skill
with
computers,
technical
problems and
organisational
factors.
Management
systems.
Dinh M,
Chu M.
2006 Australia
(but
covers
English
language
studies
from
across the
globe).
Mixed. Systematic
literature review.
Medline,
references,
organisation
al websites.
Facilitators:
good
leadership,
governance
and
collaboration.
Generic.
Elford DR 1997 Norway Norwegian
(public).
Review of
telemedicine in
northern Norway.
None
specified.
Facilitators
include good
state funding,
the support of
opinion
leaders,
enthusiasm for
new
technologies
and lack of
legal
restriction.
Communication
systems.
Falas T,
Papadopoul
os G,
Stafylopatis
A
2003 Cyprus. Mixed. Narrative review of
computerised
decision support
systems.
None
specified.
Privacy,
security, cost,
efficiency of
systems,
managing
information.
Computerised
decision support
systems.
Garritty C,
El Emam K.
2006 Canada
(but
covers
English
Mixed. Systematic review
of PDA (personal
digitalised
assistants) usuage
Systematic
database
search with
strict
Facilitators:
effective
research and
evaluation,
Generic.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 158
language
studies
from
across the
globe)..
surveys. exclusion
criteria. 23
articles
selected out
of 1775
found.
professionals
growing
accustomed to
technology
quickly.
Geibert RC. 2006 United
States.
American
(insurance
based).
Narrative review of
diffusion of
innovation research
relating to HER
(electronic health
records).
None
specified.
Barriers: too
much
information.
Facilitators:
good
leadership.
Management
systems.
Goldberg
MA, Dwyer
SJ 3rd
1995 United
States.
American
(insurance
based).
Review of
telemammography
systems and
components.
None
specified.
Technological
challenges;
systems which
are simple
enough to use.
Communication
systems.
Grams RR,
Moyer EH
1997 United
States.
American
(insurance
based).
Review of legal
issues in the use of
electronic medical
records.
None
specified.
The changing
nature of
medical
liability.
Management
systems.
Guler NF,
Ubeyli ED
2002 Turkey. None-
specific.
Describes
telemedicine
in an array
of different
countries.
Narrative review of
systems.
None
specified.
Fundamentally
, lack of
standardization
as a global
impediment.
Communication
systems.
Hanson CW,
Marshall
BE.
2001. United
States.
None
specific.
Systematic review
of the use of
artificial
intelligence in
intensive care.
Medline and
bilbliograhp
y search.
Facilitators: AI
is well-suited
to the intensive
care
environment,
reducing time
and cost,
improving
patient
welfare.
Generic.
Houtchens
BA, Allen
A, Clemmer
TP,
Lindberg
DA,
Pedersen S
1995 United
States.
American
(insurance
based).
Narrative review of
policy and
telemedicine
systems.
None
specified.
The need for
both
standardization
and flexibility.
Communication
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 159
Hunt RC 2002 United
States.
American
(insurance-
based).
Review of
communication
technology in
emergency
services.
Analysis of
an
automated
crash-
notification
service and
an
“enhanced
9-1-1
service”.
Efficient
system-design,
funding,
technological
short-comings,
concerns
regarding
privacy, staff
training.
Communication
systems.
Hussein R,
Engelmann
U, Schroeter
A, Meinzer
HP
2004 Germany. Not
specified
(presented
as generally
applicable).
Review of system
implementation/spe
cific system
(PACS).
None
specified.
Software
compatibility
with existing
systems and
the need for
standardization
.
Communication
systems.
Jaatinen PT,
Forsstrom J,
Loula P
2002 Finland. Mixed. Systematic
literature review.
Medline
search for
studies from
between
1966 and
2002. 128
articles
selected.
Appropriatene
ss of systems
for different
disciplines,
time factors.
Communication
systems.
Jadad AR 2002 Canada. Canadian
(insurance
based).
Review of
evidence-based
decision making
tools for asthma.
None stated
for the
assembly of
the paper,
but reviews
research
methods.
Describes
perceived
good
research
practice, but
is not self-
reflexive.
Barriers:
compatibility
with existing
systems.
Facilitators:
effective
technology.
General: the
need for
further
research.
Computerised
decision support.
Jennett P,
Watanabe
M, Igras E,
Premkumar
K, Hall W
1996 Canada Canadian
(insurance
based).
Review of systems
(security measures
for telemedicine.)
None
specified.
Chiefly
privacy and
confidentiality;
relates to
legality and
ownership of
Communication
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 160
the systems.
Jennett PA,
Andruchuk
K
2001 Canada. Canadian
(insurance
based).
Reviews
observations made
by Canadian
authorities as they
have implemented
tele-Healthe-Health
systems.
Not
comprehensi
vely stated.
Environmental
readiness
(policy issues
of privacy,
confidentiality,
security,
reimbursement
, standards and
licensing).
Communication
systems.
Jennett PA,
Scott RE,
Affleck Hall
L, Hailey D,
Ohinmaa A,
Anderson C,
Thomas R,
Young B,
Lorenzetti D
2004 Canada. Canadian
(insurance
based).
Systematic
literature review.
Search of
electronic
databases
(not named)
and hand
searches. 57
articles
found.
Need for
readiness for
tele-Healthe-
Health to be
measured prior
to
implementatio
n.
Communication
systems.
Jerant AF 1999 United
States.
Mixed. Systematic
literature review.
Medline
search 1966-
1999. 154
articles.
Training,
needs
assessment
(planning),
expert
recommendati
ons,
organizational/
faculty
support, user-
centred
approach, easy
access to
computers,
assessment.
Generic.
Johnson KB 2001 United
States.
American
(insurance
based).
Narrative review. Medline,
Google and
Norternlight
search for
articles on
barriers and
facilitators
to clinical IT
adoption.
Barriers:
“current
national health
environment”,
financial and
legal risks, and
lack of
knowledge and
training.
Generic.
Kaplan B,
Shaw NT
2004 United
States and
Canada.
Mixed. Narrative review. None
specified.
Potent
research
crucial to the
Generic.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 161
development
and sustaining
of e-health
systems.
Karlsson D,
Forsum U
2004 Sweden
(covers
research
from an
array of
countries).
Mixed. Narrative review of
empirical and
theoretical studies.
No
information
on how the
studies were
chosen.
Decision-
support
systems are
used
differently by
professionals
and patients.
This may
strain the
professional–
patient
relationship.
Computerised
decision support.
Katz AS,
Tilkemeier
PL
1997 United
States.
American
(insurance
based).
Review of systems. None
specified.
Broader socio-
cultural factors
(the advent of
the digital age)
make the use
of digital
technologies
an
inevitability.
Management
systems.
Lawrenson
R, Williams
T, Farmer R
1999 United
Kingdom.
Public
(NHS).
Review of general
practice databases
for research;
systems review.
Analysis of
two database
systems:
VAMP
Medical and
Meditel.
Barriers: Cost
of access to
databases, size
and structure
of the
databases
(difficulty
extracting
readable
information.)
Management
systems.
Lehmann
ED
2004 United
Kingdom.
None
specific,
refers to
both British
and
American
contexts.
Review of a
decision support
system for
Diabetes.
None
specified.
Need for
research and
evaluation.
Barrier of
potentially
undermining
the role of the
physician.
Computerised
decision support.
Liaskos J,
Mantas J
2002 United
States.
None
specific.
Review of systems
and strategies.
None
specified.
Prior planning
and subsequent
evaluation
Information
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 162
important for
successful and
sustainable
implementatio
n.
Link RE,
Schulam
PG,
Kavoussi LR
2001 United
States.
American
(insurance
based).
Review of systems. None
specified.
Technological
barriers: time
and resources
taken up by
them,
reliability and
security; legal
barriers:
liability,
licensing,
privacy and
confidentiality.
Communication
systems.
Maglogianni
s I
2004 Greece. Greek
(public).
Review of systems. None
specified.
Doctors
acceptance the
most important
factor in
implementatio
n.
Communication
systems.
Mairinger T 2000 Austria. Mixed. Systematic
literature review
compared with
findings of 3
questionnaires (full
results of
questionnaires
published
elsewhere).
Medline
search. 283
articles (but
a
bibliography
of 24).
Communicatio
n and influence
(essentially
education),
cost,
knowledge
barriers
(training),
feasibility
(appropriatene
ss of system),
legality,
difficulties in
telediagnosis,
time-
consuming
without
reimbursement
.
Communication
systems.
Mandl KD,
Kohane IS,
Brandt AM
1998 United
States.
Mixed, but
mainly
American
(insurance-
based).
Narrative Review
of current
arguments on
electronic patient-
physician
No
information
as to how
the cited
articles were
Covers how
inappropriate
use, security
and
confidentiality
Communication
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 163
communication. found or
chosen.
issues can
effect
electronic
patient-
physician
communicatio
n.
Maulden SA 2003 United
States.
American
(insurance
based).
Narrative review of
the impact of
information
technology on the
practice of
neurology.
Uses
hypothetical
scenarios to
illustrate the
use and
impact of
information
technology
on the
practice of
neurology.
Security,
legality,
management of
large data-
bases (internet
searching etc),
quality of
information on
the Internet.
Generic.
McDonald
CJ
1997 United
States.
American
(insurance-
based).
Review of
electronic medical
records systems.
Narrative
based
around
systems used
in US
hospitals.
No
systematic
method.
Covers issues
of
overabundance
of knowledge
and
appropriate
levels of
knowledge for
electronic
medical
records.
Management
systems.
Mosley-
Williams A,
Williams C
2005 United
States.
Mixed. Review of systems,
policy and trends.
Medline
search
9/2003-
9/2004.
Systems can
save time and
money.
Professional
acceptance and
training key
factor.
Generic.
Mun SK,
Elsayed
AM, Tohme
WG, Wu
YC
1995 United
States and
Korea.
Insurance
based.
Review of common
components of
telemedicine
systems.
Description
of
components
with case
studies of
projects
(though
these
projects are
summarized
rather than
Technology
and cost.
Communication
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 164
researched).
[No authors
listed]
1999 United
States.
American
(insurance
based).
Review of
telemedicine
systems.
None
specified.
Barriers: lack
of
infrastructure,
unclear needs.
Facilitator:
adequate
training.
Communication
systems.
Papshev D,
Peterson
AM.
2001 United
States.
Mixed. Systematic review
of electronic
prescribing.
MEDLINE
search.
Barriers: lack
investment
capital,
segmentation
of the
healthcare
market, lack of
technology
standardization
, providers’
resistance to
change, and
regulatory
indecisiveness.
Management
systems.
Pellegrino L,
Kobb R
2005 United
States.
American
(insurance
based).
Narrative review. None
specified.
Education and
training,
integration into
regular
routine.
Communication
systems.
Rudowski R 2003 Poland. Public. Narrative review of
telemedicine and
policy in Poland.
None
specified.
Need for
standardization
in systems use
and policy.
Communication
systems.
Sable C 2001 United
States.
American
(insurance
based).
Review of systems. None
specified.
Software
design
limitations (not
specific
enough
software), cost,
whose fault
(designers or
users) that the
system has
problems, it
may “decrease
Communication
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 165
bedside
presence”,
legal issues,
physicians
discouraged by
limited
reimbursement
.
Sheng OR,
Hu PJ, Au
G, Higa K,
Wei CP
1997 Hong
Kong.
Public. Review of four
major teleradiology
services in Hong
Kong; review of
systems.
None
specified.
System
compatibility
with existing
structures, or
imposed as
part of a
broader
structural
change. Need
for education
and training.
Communication
systems.
Shiffman
RN, Liaw Y,
Brandt CA,
Corb GJ
1999 United
States.
American
(insurance
based).
Systematic
literature review.
Medline and
Cinahl
search for
articles
published
between
1992 and
1998. 25
articles.
Tedious data-
entry
requirements
lead to
disaffection
with the
system.
Computerised
decision support
systems.
Stanberry B 2000 United
Kingdom.
European
(mixed).
Review of systems. None
specified.
Reliability of
software,
responsibility
and
accountability.
Communication
systems.
Styra R 2004 Canada. Mixed. Systematic
literature review.
Medline,
Altavista
and Google
searching.
Availability of
information for
psychiatrists
and patients,
clinical
guidelines,
clinical trial
information,
continuing
education and
training, e-mail
and related
factors of
delayed
Information
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 166
responses and
confidentiality
(changing
physician-
patient
relationship).
Swinglehurst
DA
2005 United
Kingdom.
Public
(NHS).
Narrative review
with a case study.
None
specified,
but implies
use of
Cochrane
database.
The need for
evidence, lack
of time,
attitudes of
clinicians and
organization,
patients’
expectations.
Computerised
decision support/
information
systems.
Tierney WM 2001 United
States.
American
(insurance-
based).
Narrative review of
computerised
decision support.
None
specified.
Facilitators:
Need for clear
guidelines in
the use
computerised
decision
support.
Computerised
decision support
systems.
Uhlenhopp
MB,
Fliedner
MC, Morris
P, Van
Boxtel T
1998 United
States,
United
Kingdom,
Netherlan
ds (Global
study).
Mixed. Narrative literature
review with
anecdotal material.
None
specified.
Internet as a
global
phenomenon;
cultural
differences in
nurses’
reception and
use.
Information
systems.
Vreeman
DJ, Taggard
SL, Rhine
MD,
Worrell TW
2006 United
Kingdom.
Public
(NHS).
Narrative review. Medline
search.
Studies
screened for
inclusion by
3 authors. 18
articles of an
original
2,010.
Facilitators:
end-user
participation,
good training,
workflow
analysis, data
standardization
. Barriers:
“challenges
with behaviour
modification”,
poor
equipment,
lack of
training.
Management
systems.
Wallace S,
Wyatt J,
1998 United
Kingdom.
Public
(NHS).
Narrative review. None
specified.
Professional
attitudes,
Communication
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 167
Taylor P. safety, cost,
impact on
professional/
patient
relations.
Weinstein
RS, Descour
MR, Liang
C,
Bhattachary
ya AK,
Graham AR,
Davis JR,
Scott KM,
Richter L,
Krupinski
EA, Szymus
J, Kayser K,
Dunn BE
2001 United
States,
Poland,
Germany.
Mixed. Review of systems. None
specified.
Influence of
external socio-
political
factors;
importance of
professional
education.
Communication
systems.
Whitten P,
Love B
2005 United
States.
American
(insurance
based).
Narrative review. None
specified.
User
satisfaction,
safety, cost.
Communication
systems.
Wootton, R 2001 Australia Mixed
(telemedicin
e in the
developing
world)..
Systematic review. Unspecified
literature
search.
Lack of
research,
evaluation,
long-term
funding and
quality control.
Communication
systems.
Yellowlees
PM,
Kennedy C
1997 Australia. Australian. Describes the
technology and
experience of
implementation in
Australia.
None
specified.
Confidentiality
and the need
for further
research.
Communication
systems.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 168
Appendix 3 WP1 Papers included and excluded at full paper stage
Papers included and excluded at full paper stage
Reference Include?
Reason for exclusion.
Y/N Not a review*
Not on e-Health
Not on implementation
Anon 1999 (Telemedicine an overview)
N � (d)
Anon 2006
N �
Al-Qirim 2007
N �
Al-Qirim 2003
N � (c)
Al-Qirim 2005
N � (d)
Alberdi 2005
N � (a)
Anderson 1997
N � (e)
Anderson 2007
N � (d)
Ash 2003
N �
Ash 2005
N � (e)
Avrin 2003 N �
Bagayoko 2006
N �
Bates 2002
N � (d,
e)
Beltrame 2001
N � (d)
Bensink 2006
N �
Berner 2005
N � (d)
Bick 1999
N � (d)
Bodenheimer 2003
N � (d)
Bowles 1997
N � (d)
Brender 2006 N � (a)
Broens 2007
YES
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 169
Burton 2004
N � (d)
Carrino 1998
N �
Reference Include?
Reason for exclusion.
Y/N Not a review* Not on e-Health
Not on implementation
Chamorro 2001
N � (d)
Chaudhry 2006
YES
Clemmer 2004
N � (d)
Cohen 2005 N �
Crowe 2001
N �
Davies 2007
N �
De Backer 2004
N �
De-Lusignan 2005
N �
Eadie 2003
N �
Eger 2001 N �
England 2000 N � (d)
Fenton 2006
N � (d)
Finch 2003
N � (a)
Fujimoto 2000
N � (a)
Furness 2001
N � (d)
Gater 2004
N � (d)
Geibert 2006
N � (d)
Georgiou 2007
N �
Gordon 2007
N � (c)
Guler 2002
N � (d)
Hailey 2001
N �
Hailey 2002
N �
Hakansson 2000
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 170
N �
Handler 2004
N
� (d)
Hartzema 2007
N
� (d)
Hebert 2006 YES
Reference Include?
Reason for exclusion.
Y/N Not a review* Not on e-Health
Not on implementation
Helm 2004
N � (d)
Hill 2002
N � (d) �
Hilty 2002
YES
Hilz 2000
N � (d)
Huis 2006
�
Hussein 2004
N �
Jaspers 2006
N � (d)
Jennet 2004
YES
Jennett 2005
YES
Johnson C 2006
N �
Johnson K 2001
YES
Jones 2006
N � (d)
Kanthraj 2007
N �
Kaplan 2001 (#2193)
N �
Kaplan 2001 (#2194)
N � about
evaluation methods not implementation per se.
Kaplan 2004
N � (c) �
Kaufman 2006
N � (c)
Kawamoto 2005
YES
Kluge 2007
N � (d) �
Koch 2006
N �
Koshy 2005
N � (d)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 171
Kuhn 2001
N � (b)
Kuhn 2006
N � (b)
Kukafka 2003
Yes
Reference Include?
Reason for exclusion.
Y/N Not a review* Not on e-Health
Not on implementation
Kukafka 2007
N � (d)
Kuperman 2007
N � (d)
Latifi 2005
N � (d)
Law 2003
N � (d)
Leatt 2006
Yes
Lee 2005 N Abstract only
Lehmann 2004
N � (d)
Lehmann 2006
N � (d)
Liaskos 2002
N � (d)
Loane 2002
N �
Lorenzi 1997
N � (c)
Lorenzi 2000
N � (c)
Lu 2005
Yes
MacFarlane 2006
N �3
Mair 2000
N �
Major 2005
N �
Matusitz 2007
N � (d)
Maulden 2003
N � (d)
Middleton 2005
N � (d)
Miller 2005
N � (d)
Miller 2001
N �
3 Not a review, but doesn’t meet review exclusion criteria a-e: it is a review of services (not literature) plus primary qualitative study
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 172
Miller 2007
N � (e)
Mony 2007
N � (d)
Mosley-Williams 2005
N �
Reference Include?
Reason for exclusion.
Y/N Not a review* Not on e-Health
Not on implementation
Mun 1999
N � (d,e)
Nagykaldi 2007
N � (a)
Nazi 2003
N � (b)
Nies 2006
N �
Norum 2007
N � (b) �
Noss
N � (d)
O’Meara 2007
N � (d)
Ohinmaa 2006
YES
Ozdas 2007
N � (d)
Pagliari 2004
N � 4
Papshev 2001
YES
Payne 2000
N � (d)
Peleg 2006
YES
Pellegrino L 2005
N
� (d)
Poissant L 2005
N �
Previte J 2006
N � (d)
Puskar KR 2004
N � (d)
Puskin DS 1995 N �
Rahimi B 2007
N �
Ruckdaschel 2006
N � (d)
Rudowski R 2003 N � (d)
4 This paper reports a programme evaluation, not a literature review
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 173
Saathoff 2005
N �
Sable S 2001
N �
Reference Include?
Reason for exclusion.
Y/N Not a review* Not on e-Health
Not on implementation
Shekelle 2006
YES
Sheng 1997
N � (b)
Shiffman 1999
N �
Smith A 2002
N �
Sonnenberg 2006
N �
Souther E 2001
N � (d)
Staggers N 2003
N � (d)
Studer 2005
YES
Subramanian S 2007
N � (d)
Swinglehurst D 2005
N � (d) � �
Tang P 2006
N � (e)
Tidd 1999
N �
Tierney W 2001
N � (d)
Upperman J 2005
N � (d)
Van Ginneken A 2002
N � (d)
Vincent DJ 2005
N �
Vreeman D 2006
YES
Wallace S 1998
N � (d)
Walz M 2000
N � (d)
Weiner MG 2000
N � (d)
Whitten P 2005
N � (d) �
Whitten 2007
No �
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 174
Xue Y 2007
N �
Yarbrough A 2007
YES
Yellowlees 1997 N
� (d)
Reference Include?
Reason for exclusion.
Y/N Not a review* Not on e-Health
Not on implementation
Yusof 2007
YES
Table Legend/Definition
A review paper provides an analytic account of the research literature
related to a specific topic or closely related set of topics. It is intended to
contribute to knowledge by answering a research question.
Review papers include:
(i) systematic review: where relevant literature has been identified by
means of structured search of bibliographic and other databases; where
transparent methodological criteria are used to exclude papers that do not
meet an explicit methodological benchmark, and which presents rigorous
conclusions about outcomes;
(ii) narrative review: where relevant literature has been purposively
sampled from a field of research; where theoretical or topical criteria are
used to include papers on the grounds of type, relevance, and perceived
significance; with the aim of summarising, discussing, and critiquing
conclusions;
(iii) qualitative metasyntheses or meta-ethnographies (where relevant
literature has been identified by means of a structured search of
bibliographic and other databases, where transparent methods had been
used to draw together theoretical products, with the aim of elaborating and
extending theory .
Review papers exclude:
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 175
(a) Secondary analyses (including qualitative metasyntheses or meta-
ethnographies) of existing data-sets for the purposes of presenting
cumulative outcomes from personal research programmes,
(b) Secondary analyses (including qualitative metasyntheses or meta-
ethnographies) of existing data-sets for the purposes of presenting
integrative outcomes from different research programmes,
(c) Discussions of literature included in contributions to theory-building or
critique,
(d) Summaries of literature for the purposes of information or
commentary.
(e) Editorial discussions that argue the case for a field of research or a
course of action
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 176
Appendix 4 WP1 Details of included reviews
Details of included reviews
Reference ID Authors Country of
Origin
E-Health
domain
Publisher and date
of publication
Aim of review Methods of review Databases
searched
Inclusion and exclusion criteria Number of
included
papers
Broens
2007
Broens THF,
Huis in’t Veld
RMHA,
Vollenbroek-
Hutten MMR,
Hermens HJ,
van Halteren
AT,
Nieuwenhuis
LJM
Netherlands Communication Journal of
Telemedicine and
Telecare 2007
To identify determinants
of implementation of
telemedicine
interventions
Qualitative review Papers presented
at Telemed
Conference 2004
in London
Any paper presented at this
conference
45
Chaudrhy
2006
Chaudrhy
B,Wang J, Wu
S, Maglione
M, Mojica W,
Roth E,
Morton SC,
Shekelle PG.
USA All 4 Annals of Internal
Medicine
2006
To systematically review
evidence on the effect of
health information
technology on quality,
efficiency and costs of
health care
Systematic review MEDLINE (1995 –
2004)
Cochrane Central
Register of
Controlled Trials,
Cochrane
Database of
Descriptive and comparative
studies and systematic reviews of
health information technology
257
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 177
Abstracts of
Reviews of
Effects,
Periodical
Abstracts
Database
“studies identified
by experts”
Hebert
2006
Hebert MA,
Korabek B,
Scott RE.
Canada Communication
systems.
International Journal
of Medical
Informatics 2006C
To examine research
evidence supporting the
effectiveness of tele-
Healthe-Health and to
develop a decision
framework to
demonstrate an
approach for decision
makers and practitioners
to transfer home tele-
Healthe-Health research
into practice
Narrative review Not explicit Not explicit 3 (these 3
were
themselves
systematic
reviews)
Hilty
2002
Hilty DM, Luo
JS, Morache
C, Marcelo
DA, Nesbitt
TS
USA Communication
systems;
Management
systems; CDSS
CNS Drugs 2002 To discuss the
advantages and
disadvantages of
telepsychiatry for clinical
and educational
purposes and also to
address the practical
Systematic review Searched
following
databases
between 1965 and
june 2001 using
the MEDLINE<
Embase, Science
Inclusion described, exclusion
criteria not explicit
Not provided
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 178
considerations about
delivery of patient care,
pt and provider
satisfaction, effects on
communication and
interpersonal behaviour
and costs. Aims to
outline principles for
using telepsychiatry
Citation index,
social sciences
citation index, and
telemedicine
information
exchange
databases.
Jennett
2004
Jennett PA,
Scott,RE, Hall
LA, Hailey D,
Ohinmaa A,
Anderson C,
Thomas R,
Young B,
Lorenzetti D.
Canada Communication
(tele-Healthe-
Health)
Telemed J and e-
Healthe-Health 2004
Identify policy strategies
for successful
telemedicine
implementation
Comprehensive search
of peer-reviewed and
grey literature 1980-
2002 (including
‘electronic databases,
hand searches of
journals and conference
proceedings, and
communication with
consultants in the field’
Not explicit Not explicit ’57 Policy
Sources’
Jennett
2005
Jennett PA,
Gagnon MP,
Brandstadt HK
Canada Communication
systems (tele-
Healthe-Health)
Journal of
Postgraduate
Medicine
To review and critique
peer-reviewed studies
that have focused on
assessing tele-Healthe-
Health readiness for
rural and remote health
Systematic review Papers between
1996 & 2005:
Medline, TIE,
Pubmed, AMED,
Google, Web of
Science, CINAHL,
EMBASE, AARP
Ageline, Cochrane
Central,
Papers about tele-Healthe-Health
‘readiness’, judged for inclusion
on the basis of theoretical and
methodology quality
Did not
specify
actual
numbers of
papers
included, but
referred to
four models
of readiness
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 179
PyschInfo, ERIC &
ProQuest
that were
critiqued
(cites 8
papers in
relation to
these
models).
Johnson
2001
Johnson KB
USA Not specified,
probably all 4.
Archives of Pediatric
and Adolescent
Medicine 2001
Identify barriers to
adoption of IT by health
professionals
Search using limited set
of search terms, follow
up of ‘relevant
references’ search on
Google and
Northernlight.com
Medline only
References included if they
discussed barriers to adoption
Unclear.
Paper has
63 relevant
references.
Kawamot
o 2005
Kawamoto K,
Houlihan CA,
Balas EA,
Lobach DF
USA CDSS BMJ 2005 To identify features of
CDSS’s critical for
improving clinical
practice
Systematic review Searched Medline,
CINAHL and the
Cochrane
Controlled Trials
Register up to
2003 and
searched
reference lists of
included studies
and relevant
reviews.
Inclusion criteria were studies that
evaluated the ability of decision
support systems to improve
clinical practice. Incl any RCT.
Excluded those with less than 7
units of randomisation per study
arm, non English, mandatory
compliance with CDSS, lack
Kukafka
2003
Kukafka, R;
Johnson, SB;
USA ALL 4 Journal of
Biomedical
To systematically review the
literature on implementation
Systematic review Medline (1999-
present); Science
Papers had to report empirical
research study. Hit at least one
142 papers
reduced to
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 180
Linfante, A;
Allegrante, JP.
Informatics 2003 of IT in health care, to
develop an integrative
framework drawing together
diverse theoretical
perspectives
Direct (1998-
present);
HealthStar (1998-
present); and ABI
Inform (1999-
present)
search term from each key
category (Health; Information;
behave/org.)
57 after
abstract
review;
reduced to
24 after
reading full
paper
Leatt
2006
Leatt P; Shea
C; Studer M;
Wang V.
USA Management
systems (EHRs;
CPOE;
Electronic
Medication
Administration
Records)
Electronic
HealthCare
To review the literature on
the facilitators and barriers
to successful
implementation of EMR,
CPOE & MAR
Literature review
(unspecified)
Not specified Not specified Not
specified
Lu 2005 Lu, Y-C; Xiao,
Y; Sears, A;
Jacko, JA.
USA ALL 4 International Journal
of Medical
Informatics
To review the literature on
issues related to adoption of
PDAs in health care and
barriers to PDA adoption
Systematic review Year span: 1998-
2004. Medline;
National Library of
Medicine’s
searchable
database of peer
reviewed
publications;
published
proceedings of
HIMSS
(Healthcare
Information &
Management
PDAs & mobile computing
devices, and use by all health
professionals
95 reviewed,
out of 200
identified.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 181
Systems Society;
plus internet
searches
Ohinmaa
2006
Ohinmaa A Canada Communication
systems
Journal of
Telemedicine and
Telecare 2006
To assess telemedicine
projects outside the USA
and provide examples of
promising results that could
be disseminated to other
countries.
Narrative Review Articles from the
last 3 yrs of the
“Successes and
Failures in Tele-
Healthe-Health
Conference and a
special issue of
the International
Journal of
Circumpolar
health in 2004
Included articles that showed a
scientific basis for successful
telemedicine conducted outside
the USA with exception of Alaska.
Excluded programmes from
developing countries that were
seen to difficult to implenment in
the US health system. Articles
discussing non medical
applications eg education were
excluded. Review focused on
applications benefiting significant
segments of the health care
population rather than those
restricted to a targeted population
or geographical area.
Not stated
Papshev
2001
Papshev D,
Peterson
A.M.
USA e-prescribing American Journal of
Managed Care
2001
To examine advantages of
and obstacles to electronic
prescribing in the
ambulatory care
environment
Narrative review MEDLINE,
International
Pharmaceutical
Abstracts
January 1980 –
September 2000
Articles, symposia proceedings
and organisational positions
statements published in the US
on electronic prescribing and
automation in health care
Not stated
(35
references
provided).
Peleg Peleg M, Tu
S.
USA Decision
support
International Medical
Informatics
To find trends in Clinical
Decision Support Systems
Narrative review PubMed “past 5
years”.
“Decision Support Systems” in
title
Not stated
(63
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 182
2006
Association
Yearbook
2006
(CDSS) that were developed
over the last few decades
and give some indication of
future directions in
developing successful,
usable CDSS.
references
provided).
Shekelle
2006
Shekelle P,
Morton SC,
Keeler EB,
Wang JK,
Chaudhry
BI, Wu S,
Mojica WA,
Maglione M,
Roth EA,
Rolon C,
Valentine D,
Shanman R,
Newberry
SJ.
USA Management,
Decision
support
Agency for
Healthcare
Research and
Quality
2006
To assess the evidence
base regarding benefits and
costs of health information
technology (HIT) systems,
particularly those providing
pediatric care.
Systematic review MEDLINE,
Cochrane
Controlled Clinical
Trials Register,
Database of
Reviews of
Effectiveness,
Health Affairs,
and “several
reports prepared
by private
industry”.
1995 - 2003
Systematic reviews, meta-
analyses, hypothesis-testing or
predictive analyses on costs,
benefits and barriers to
implementing HIT.
256 studies
(156 on
decision
support, 84
on the EMR,
and 30 on
CPOE).
124 in
ambulatory
care, 82 in
in-patient
settings.
Studer 2005 Studer M USA Management Electronic
Healthcare 2005
To systematically review
studies assessing the effect
of organizational factors on
the effectiveness of EMR
system implementation
Systematic review MEDLINE 1900 –
2005, and
reference sections
of included
studies.
English language; available at the
University Library;
On electronic medical records;
Assessing the effect of
23 papers.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 183
management support, financial
resource availability,
implementation climate and / or
implementation policies and
practices on the effectiveness of
EMR implementation.
No quality or study design filter.
Vreeman
2006
Vreeman DJ, Taggard SL, Rhine MD, Worrell TW.
USA Management
systems
Physical Therapy
2006
To investigate the role of
EHRs in physical therapist
practice. Aim of review was
to identify, review and
summarize the benefits,
barriers, and key factors for
success in implementing
EHRs in physical therapist
practice settings.
Systematic review Medline (1966 to
week 4 of October
2004); the
cumulative index
of nursing and
allied health
literature (1982 to
week 4 of October
2004); Ovid’s All
evidence based
Medicine reviews
(Cochrane
database of
systematic
reviews, American
College of
Physicians Journal
Club, Database of
Abstracts of
Reviews and
Effects, and
Studies were included if they met all of the following criteria: 1) an EHR was the intervention of interest in the study; 2) the HER contained the IOM (institute of medicine) core functionality of health information and 2 or more of the other core functionalities; 3) the study described a primary use of the HER; 4)physical therapists were study participants, and 5) the article reported outcomes that indicated benefits or barriers to system implementation. Studies were excluded if they described only physiological monitoring systems, communication technology for telemedicine applications, or only secondary uses of EHR.
18 articles
included…H
owever
authors of 6
of these
articles
reported on
the initial
implementati
on and
ongoing
analysis of
the same
HER so
these
articles were
aggregated
and
analysed as
1 unit giving
an analysis
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 184
Cochrane Cnetral
Registrar of
Controlled Trials
(third quarter
2004). Also
conference
proceedings from
the American
Medical
Informatics
Association
Annual
Symposium
(1998-2004)
based on a
total of 13
studies.
Yarborough
2007
Yarborough AK, Smith TB.
USA Management,
communication
and information
systems.
Med Care Res rev
2007
To Review the literature on
physician technology
acceptance in order to
improve understanding of
barriers to physician
adoption of new
technologies.
Narrative review. PubMed and ABI
Inform/Complete
databases
between Jan 1996
and Nov 2006.
English only articles in peer
reviewed journals that were about
technology acceptance research
pertinent to health care contexts.
Excluding those not directly
pertaining to : 1) physician IT, 2)
physician barriers to new
technology; or 3) the TAM.Also
studies using specific types of
qualtitative methodologies were
excluded in an effort to focus on
studies using more rigourous
methodologies. Thus case
studies that were purely
18 articles.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 185
descriptive and limited to less
than two sites were excluded as
were review articles that only
summarised findings. Qualitative
studies of multiple organisations
or multiple groups of technology
users that used an inductive
approach were included as were
meta-analyses that used
statistical methods to provide a
review of prior research findings.
Yusuf 2007 Yusof MM, Stergioulas L, Zugic J
Malaysia Management,
communication,
CDSS
MEDINFO 2007 To review the literature on
health information systems
adoption in clinical practice
Systematic review Using a number of
database from
medical,
informatics and
engineering field.
Between 1985 and
2005
Inclusions all computer based info
systems that involve human
interaction used in health care
settings. Those used for training /
education excluded. English only.
Looking at evaluation studies in
clinical studies selecting case
studies only and excluding
experiment and surveys.
55
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 186
Appendix 5 DATA EXTRACTED FOR THEMATIC ANALYSIS - SYSTEMATIC REVIEW
Conditions
Prior To
implementatio
n
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of Use
of System
Security/Confidenialit
y and standards
Education
and Training
Technological
Issues
Communicatio
n Issues
Organisational
Issues
Other
1 Broen
s
2007
�
Involvement of
patients and
professionals
in the
requirements
analysis and
design process
crucial in order
to fit
telemedicine
into their daily
work. Pg 305
para 3
Also related to
legislation and
�
Costs
related to
investments,
maintenanc
e and
operational
costs. Pg
305 para 6
Lack of
financing
structures
for ongoing
work.pg 305
para 6
�
Evidence
based medicine
is regarded as
a requirement
pg 305 para 4
. “ alternative
designs
needed to
evaluate
efficacy and to
convince
professionals,
policymakers
and insurance
companies
� Technology acceptance influenced by prof attitudes. Pg 305 para 3
Stimulating
role of
leading
champions
305 para 5
Technology
acceptance
influenced
by patient
�
Usability of
system is a
major factor
for
success304
para 7
“Professional
s should be
able to
access the
right patient
information at
the right time”
305 para 3
�
Standardisation
Related to
interoperability pg 306
para 3 and to ensure
quality and uniform
practice.
Security 306 para 4
patient physical safety
and pt info security.
For acceptance of
telemedicine
implementations
adequate security
�
Need for
support on
how to
install and
sustain the
system and
how to deal
with errors.
Training at
all levels
Pg 304
para5+6
Creating
familiarity
�
Technology quality
technical problems
a major barrier to
implementation….e
g non-connecting or
malfunctioning
devices. Pg 305
para 2
Eg lack of
working
protocols for
use of new
technologies
and ability to
accommodate
chgs in
collaboration
and team
roles, rights
and
responsibilities
pg 306 para 1
Telemedicine
might require
changes in
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 187
policy pg 306
para 2
about
implementation
.
Page 305 para
6
Comprehensive
cost
effectiveness
studes are
essential in
developing
future financing
structures
pg 305 para 6
attitudes
Pg 305
para 3
305 para 3
mechanisms have to
be taken into account.
These security
mechanisms should
support the crucial
trust relation between
health-care providers
and patients. P306,
para 4.
A need for secure
information transfer
and authorization
mechanisms. P 306,
interoperability
between telemedicine
applications is
important to support
the current trend of
transmural work
practices and is not
guaranteed without
globally accepted
standards
with the
intervention
(p 305, para
2).
Availability
of support to
users (p304,
para 3,
support);
supporting
staff and
doctors
should be
able to
operate the
devices and
should have
flexible
access to
services
offered by
the
telemedicin
e system. P
304,para 5.
changes in
collaboration
and (team)
roles, rights
and
responsibilities
. P306, para 1
Novel working
practices
introduced by
telemedicine
do not always
fit with existing
traditional
working
protocols in
health care (p
306, para 1).
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 188
a need for
training
users how
to use these
novel types
of systems.
Such
training is
needed at
all levels pg
304 para 4
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 189
Conditions Prior
To
implementation
Cost Need and
importance of
validation and
evaluation
Attitude
s
Ease
of Use
of
Syste
m
Security/Confidenialit
y and standards
Education and
Training
Technologica
l Issues
Communicatio
n Issues
Organisationa
l Issues
Other
2 Chaudrhy
2006
However, the
method used by
the benchmark
leaders to get to
this point – the
incremental
development
over many years
of an internally
designed
system led by
academic
research
champions – is
unlikely to be an
option for most
institutions
contemplating
implementation
of health
information
technology….th
One of the
chief barriers
to adoption of
health
information
technology is
the
misalignment
of incentives
for its use.
Specifying
policies to
address this
barrier is
hindered by
the lack of cost
data. Pg 749
2nd col 1st
para.
Effects on
Two critical
questions that
remain are: 1)
what will be the
benefits of
these initiatives
(health
information
technology)
Third a high
priority must be
the
development of
uniform
standards for
the reporting of
research on
implementation
of health
information
Finally, additional
work is needed on
inter-operability. pg
749 col 2 para 2
the study by
Koppel and
colleagues
included
detailed
contextual
information
relating to
human factors.
One health
record study
reported
physician
classroom
training time of
16 hrs before
implementation
. Another order
entry study
reported that
nurses
received 16
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 190
technology….th
e effectiveness
of these
technologies in
the practice
settings where
most health care
is delivered
remains less
clear. ….how
these tools are
used and the
context in which
they are
implemented are
critical pg 748
col 2 para 2
costs –data on
costs were
more limited
than the
evidence on
quality and
efficiency.
Because these
systems were
built,
implemented,
and evaluated
incrementally
over time, and
in some cases
were
supported by
research
grants, it is
unlikely that
total
development
and
implementatio
n costs could
be calculated
accurately and
in full detail.
747 1st
technology,
similar to the
CONSORT and
QUORUM
statements. pg
749 col 2 para
2
Published
evidence of the
information
needed to
make informed
decisions about
acquiring and
implementing
health
information
technology in
community
settings is
nearly non
existent. For
example,
potentially
important
evidence
related to initial
received 16
hours of
training, clerical
staff received 8
hours, and
physicians
received 2 to 4
hours. Pg 748
col 1 para 3
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 191
column, para
3.
these 9 studies
infrequently
reported or
measured data
on costs and
contextual
factors.. Two
reported
information on
costs. Neither
described the
total initial
costs of
purchasing or
implementing
the system
being
evaluated.
Data on
contextual
factors such
as
reimbursement
mix, degree of
capitation, and
capital costs,
effect on
provider
productivity,
resources
required for
staff training
(such as time
and skills), and
workflow
redesign is
difficult to
locate in the
peer reviewed
literature. Also
lacking are key
data on
financial
context, such
as degree of
capitation,
which has been
suggested by a
model to be an
important factor
in defining the
business case
for electronic
health record
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 192
barriers
encountered
during
implementatio
n were scant;
only 2 studies
included such
information. Pg
748 col 1 para
3
with respect to
the business
case for health
information
technology, we
found little
information
that could
empower
stakeholders
to judge for
themselves the
financial
effects of
adoption. Pg
749 col 1 last
para
use. Pg 748 col
2 para 3
In many
important
domains we
found few
studies. This
was particularly
true of health
information
technology
applications
relevant to
consumers and
to
interoperability,
areas critical to
the capacity for
health
information
technology to
fundamentally
change health
care. 748/749
col 2 last para
and col 1 1st
para.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 193
who will pay
and who will
benefit? Pg 749
col 1 para 2
Regarding the
former
(benefits), a
disproportionat
e amount of
literature on the
benefits that
have been
realised comes
from a small set
of early adopter
institutions that
implemented
internally
developed
health
information
technology
systems.
These
institutions had
considerable
expertise in
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 194
health
information
technology and
implemented
systems over
long periods in
a gradual,
iterative
fashion.
Missing from
this literature
are data on how
to implement
multifunctional
health
information
systems in
other health
care settings.
Pg 749 col 1
para 3.
The limited
quantitative and
qualitative
description of
the
implementation
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 195
context
significantly
hampers how
the literature on
health
information
technology can
inform decision
making by a
broad array of
stakeholders…
pg 749 col1
para 3
Time utilization
p 748 column 1,
para
1…..relative
decreases in
other outcomes
were as follows:
medication
turnaround
time, radiology
completion
time, results
reporting time,
nurse
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 196
documentation
time
the major effect
of health
information
technology on
quality of care
was its role in
increasing
adherence to
guideline- or
protocol based
care (p 744,
final para).
the second
theme showed
the capacity of
health
information
technology to
improve quality
of care through
clinical
monitoring
based on large
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 197
scale screening
and
aggregation of
data pg 746 col
1 para
2……examples
include: using
an electronic
health record to
identify adverse
drug events (pg
746 col 1 para
2); the role of
health
information
technology
surveillance
systems in
identifying
infectious
disease
outbreaks (pg
746 col 1 para
4); third health
information
technology
mediated effect
on quality was
a reduction in
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 198
medication
errors pg 746
2nd col para 2.
Effects on
efficiency;
utilization of
care; provider
time. (p 746,
2nd column,
final para.)
studies
examined the
effect of
systems on
Utilization of
care. P 747 2nd
column.
To date the health information technology literature has shown many important quality and efficiency related benefits
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 199
199
as well as limitations relating to generalisability and empirical data on costs.. Studies from 4 benchmark leaders demonstrate that implementing a multifunctional system can yield real benefits in terms of increased delivery of care based on guidelines (particularly in the domain of preventive health), enhanced monitoring and surveillance activities, reduction of medication errors, and decreased rates of utilisation for potentially redundant or inappropriate care. Pg 748 col 1 last para.
More
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 200
information is
needed
regarding the
organisational
change,
workflow
redesign,
human factors,
and project
management
issues involved
with realising
benefits from
health
information
technology pg
749 col 2 para
2
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 201
Conditions
Prior To
implementation
Cost Need and
importance
of validation
and
evaluation
Attitudes Ease
of Use
of
System
Security/Confideniality
and standards
Education
and
Training
Technological
Issues
Communicatio
n Issues
Organisational
Issues
Other
3 Hebert
2006
�
Decision
framework can
form the basis
of a business
plan for
delivering
home care that
addresses
service
objectives,
costs , and
national
outcome
indicators.
Pg 792 para 2
Inertia and
�
Project
establishment
costs;
equipment
costs,
maintenance
costs,
communication
and staffing c
costs. Pg 790
para 3
Underlying
service delivery
model also
affects
costs…so
study context
important pg
�
Home
telehealth
evidence:
what are the
most
effective
technologies
to support
provision of
services. Pg
791 para6
Does the
evidence
support
using the
technology to
achieve
desired pt
�
Are the
organisation,
health care
providers
and clients
ready to
adopt
technology?
Pg 791
para7
Appropriate
allocation of
staff time to
the project pg
790 para 4
Radical re-
engineering of
the service pg
790 para 9
Context:
stability
of
client’s
condition
and
nurses
ability to
co-
ordinate
more
than one
visit
Pg 791
para9
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 202
202
resistance to
change within
the healthcare
system pg792
last para
790 para 5
outcomes or
reduce
delivery
costs pg 791
para 7
Using theory
driven
program
development
helps
decision
makers
develop clear
expectations
for realistic
outcomes of
using the
technology
pg 791 col 2
section F5.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 203
203
Conditions Prior
To
implementation
Cost Need and importance
of validation and
evaluation
Attitudes Ease of Use
of System
Security/Confideniality
and standards
Education and
Training
Technological
Issues
Communication
Issues
Organisational
Issues
Other
4 Hilty
2002
�
Some obstacles
eg access to
broad bandwidth
lines will reced
ewith the
advancement of
technology.
�
Costs
should be
considered
for patients,
clinics,
providers
and society
at large pg
543 para 5.
�
Long term
sustainability requires
collection of data and
feedback for clinical,
technical and
administrative staff
pg 544 para2.
evaluate options,
implementation and
maintenance of
telepsychiatry with a
team of clinicians,
technicians and
administrators in both
the hub and the
spoke sites. Pg 544
table VI
�
Physicians
may have
anxiety from
the
psychologic
al
resistance
to a change
in the status
quo pg 543
para 6
Suitable
clinical
chamption
is vital pg
544para 2
satisfaction
�
Satisfaction
with
telepsychiatry
was lower
than in person
consultation in
terms of ease
of use with the
process, pg
537 para 5
�
Concerns about
security pg 543 para 6
Legal issues pg 542
para 6
�
Standards are needed
for electronic
information exchange
in addn to established
provisions for security
and privacy pg 543
para 6
�
Lack of
experience or
training;
Inadequate
typing skills
pg 543 para 6
Inadequate
technical
support in
initiation,
maintenance,
and trouble
shooting
emergencies
alienates pts
and clinicians
alike.
Pg 544 para 1
�
with regard to
technical
issues,
programmes
need to review
a variety of
vendors
products
before making
a purchase,
involve the
clinical staff
who will use
the product
from the
beginning. Pg
544 col 2 para
1
Technical
Telepsychiatry
appears to allow
for the building
of relationships
based on the
advantage of
creating a social
presence and
providing more
visual cues than
telephone and
the
disadvantages of
limiting
nonverbal
communication
compared with in
person care. Pg
543 para 3
Effects of
telepsychiatry on
interpersonal
Incentives for
each of the
parties
involved pg
544 para 2
Inadequate support from the specialists providing the service pg 543 para 7
Many
programmes
fail because of
inadequate
financial and
other
administrative
support from
the
leadership, or
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 204
204
with
telepyschiat
ry was
lower than
in person
consultation
in terms of
ease of use
with the
process pg
537 para 5
Training
practitioners
to practice
telepsychiatry
requires
comfort with
the
equipment,
adapting to its
clinical
practice and
being aware
of its
limitations. Pg
544 para 2
issues under-
reported
including
bandwidth,
audio quality,
FPS, size of
transmitted
video image
spped of
computed and
name and
make of
CODEC and
other
equipment pg
543 para 4
promptly solve
technical
problems pg
544 col 2 para
1
interactions pg
540 para3
Effect on
communication
and relationships
pg 539 para 4
one concern
about
telemedicine is
that the
technology may
adversely affect
the development
of a positive
therapeutic
alliance.
Telepsychiatry
seems to have
positive and
negative effects
on
communication
pg 539 col 2
para 3
the fact that
telemedicine
is not a match
for the overall
mission of the
organisation
Pg 543 para 7
diversifying
personnel is
helpful to
sustain the
programme
despite
departures pg
544 col 1
para 2
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 205
205
Conditions Prior
To
implementation
Cost Need and importance
of validation and
evaluation
Attitudes Ease of Use
of System
Security/Confideniality
and standards
Education and
Training
Technological
Issues
Communication
Issues
Organisational
Issues
Other
5 Jennett
2004
�
Important that all
people involved
in health care
particularly
policy and
decision-makers,
start to consider
telehealth as an
integral part of
… health
systems.
(p79, col2,
para4)
This can be
achieved thru
aligning
telehealth
initiatives with
suitable
frameworks
for
economic
analysis
need to be
developed
that capture
non
monetary
and
unintended
consequen
ces as well
as
monetary
measures.
Pg 80 table
2 C
Evaluation.
Evaluation and
research….driving
forces for telehealth
as …..and proof of
technological
feasibility.
Pg 82 para9
suitable outcome
indicators, measures
and reliable and valid
instruments for
socioeconomic
benefit of telehealth
must be identified,
defined, and
consistently applied
within a recognised
framework that asks
relevant questions.
Pg 80 table 2 C
evaluation
It is important that professional and regulatory groups become involved in e-policy decision making so that telehealth can be applied to its full extent pg 82 col 1 para 1
take note of
patient and
professional
s’ views
(p82, col 1
para1)
Standards are needed
to foster use of
technology in specific
clinical areas pg 82
para 1
Barriers identified
were… privacy and
security
It is imp to assess risk
prior to
implementation pg 82
para 7
Barriers identified
were… privacy and
security,
accreditation,
intellectual property
�
Various
strategies,
such as
training and
education,
may enhance
trusting
relationships
pg 82 para2
to facilitate
access to
many
telehealth
applications
increased
broadband
connectivity is
needed,
particularly to
rural and
remote
communities.
Pg 80 table
2A policy.
collaboration,
partnerships and
sharing:
interjurisdictional
sharing of skills,
information, and
services through
telehealth
programs
creastes
opportunities to
improve health
care.
Collaboration
between industry
and health care
organisations
holds several
advantages,
such as
opportunities to
test technologies
and the ability to
telehealth
applications
are more
likely to be
successful, in
terms of cost
and
sustainability,
if they are
considered to
be part of the
larger domain
of e-health.
Programs
implemented
and evaluated
as
independent
initiatives are
at greater risk
of failure.
Important that
all people
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 206
206
initiatives with
existing strategic
health plans. Pg
80 table 2A
policy.
establishment of
a policy forum
that focuses on
telehealth policy
would facilitate
these needs pg
80 table 2A
policy.
partnerships in
telehealth should
be established
where there are
opportunities to
improve
efficiency in
health care and
decrease
duplication. Pg
80 table 2A
policy
evaluation
telehealth programs
should implemented
in a culturally aware
and culturally
sensitive manner pg
80 table 2C
Evaluation
evaluations should
include examination
of the social,
organisational and
policy aspects of
telehealth pg 80 table
2 C evaluation.
and copyright (p82,
col 1 para1)
and the ability to
communicate
local needs pg
82 col 1 para 5
effective
communication
about the goals,
rationale,
benefits and
limitations of the
program, and
training and
education may
enhance trusting
relationships pg
82 col 1 para 2
As teleheatlh
continues to
evolve, input
from all
stakeholders…in
to policy
development is
required.
Consideration of
all people
involved in
health care
particularly
policy and
decision-
makers, start
to consider
telehealth as
an integral
part of
….health
systems. Pg
79 col 2 para
4
organisational
readiness,
which refers to
the fit between
the telehealth
application
and the
processes,
and standards
within the
organisation,
is also
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 207
207
establishing a
guiding
philosophy while
planning for
implementation
is the key to
success pg 80
col 2 para 1
needs as well as
practical
experience is
essential for a
meaningful
exchange of
information and
views. Pg 80
table 2A policy
important pg
82 col 1 para
3
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 208
Conditions Prior
To
implementation
Cost Need and importance
of validation and
evaluation
Attitudes Ease of Use
of System
Security/Confideniality
and standards
Education and
Training
Technological
Issues
Communication
Issues
Organisational
Issues
Other
6 Jennett
2005
�
Turf, practice
context,
ownership…pg 3
last para
organisational
readiness and
system
readiness pg 5
discussion para
1
�
Funding pg
5 para3
�
Projection of benefits
pg 4 para3
�
Innovators
and
champions
pg 5 para3
Strong
leadership
pg 5 disc
para 2
perceived
need for
improveme
nt/awarenes
s of risks
and benefits
pg 5
discussion
para 2
Inefficient
technology
Pg 5 para3
Assessment of risk
pg 4 para 2
sense of risk pg 5
discussion para 2
Awareness
and education
pg 4 para 2
inefficient
technology pg
5 discussion
para 2
organisational
readiness pg
5 discussion
para 1
demands on
time pg 5
discussion
para 2
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 209
resistance
to change
pg 5 para 3
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 210
Conditions Prior
To
implementation
Cost Need and
importanc
e of
validation
and
evaluation
Attitudes Ease of Use of
System
Security/Confidenialit
y and standards
Education
and Training
Technological
Issues
Communicatio
n Issues
Organisationa
l Issues
Other
7 Johnso
n 2001
�
Situational –
external
environmental
factors affecting
access to or use
of technology. Pg
1para 5
Access to IT is
influenced by the
financial situation,
the location of the
practice (rural vs
urban), the size of
the practice, and
the average age
of the PHCPs –
with older PHCPs
less likely to have
�
Situational –
economic
realities. Pg
1para 5
Time pressure
represents the
most
signigicant
barrier to the
adoption of
potentially
useful
technologies.
Today’s
PHCPs spend
an average of
17 mins per
patient
�
�
attitudinal
behaviours or
opinions
contrary to
those needed
to adopt a
technology pg
1 para 5.
Knowledge
and attitudinal
barriers pg
1375 col 2
para 4
Lack of insight
about the
benefits of IT
interventions
succeeded
when decision
support was
provided to
clinicians
automatically pg
3 col 2 para 6
similarly
systems that
were provided
as an integrated
component of
charting were
significantly
more likely to
succeed than
stand alone
Legal – regulated or
unregulated practices
that affect use of a
technology pg1 para
5
The diversity of our
health care
environment, in terms
of policies, practice
styles and revenue
generating activities
make it difficult to
achieve consensus
about what or how
technology should be
applied to the field.
Pg 1375 col 1 para 5
Cognitive
and/or
physical
insufficient
skills or ability
to use a
technology pg
1 para 5
Other
environmenta
l barriers that
are critical to
primary care
practices
include the
cost of
training
personnel
(1375, col 2,
interventions
succeeded
when decision
support was
provided to
clinicians
automatically pg
3 col 2 para 6
similarly
systems that
were provided
as an integrated
component of
charting were
significantly
more likely to
succeed than
stand alone
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 211
less likely to have
access to (or
comfort with)
technology. Pg
1375 col2 para1
Collaborators with
stakeholders –
professional
societys such as
the American
Academy of
Pediatrics….have
assumed a
leadership role in
taking steps to
promote the
adoption of IT.
These societies
have begun to
collaborate with
industry,
government, and
consumer
leaders.
Collaboration…..i
s critical both to
short term
patient
encounter and
are
reimbursed
based on the
time needed
by each
patient.
Studies have
shown that
time pressure
clearly affects
the tolerance
of providers
for
technologies
such
as….computer
based
documentatio
n tools pg
1375 para 4
IT is neither a
one time nor a
low risk
investment.
The sources
benefits of IT
pg 1376 col 1
para 2
IT advocacy.
In concert with
increasing
awareness of
IT, we need to
establish a
network of IT
advocates who
can facilitate
the movement
of more
apprehensive
colleagues
with the help of
other groups.
Pg 1378 col 1
para 4
as our new
systems affect
larger, more
heterogenous
groups of
stand alone
systemes pg 3
col 2 para 6
systems that
used a
computer to
generate the
decision support
were
significantly
more effective
than systems
that relied on
manual
processes pg 3
col 2 para 6
systems that
prompted
clinicians to
record a reason
when not
following the
advised course
of action were
significantly
The adoption of IT
often is impeded by
questions about
liability associated
with the dissemination
of information as well
as concerns about
what balance
between security and
and access is
acceptable to
consumers.
Concerns about
security, patient
confidentiality, and
liability have affected
the PHCPs
confidence in using
email to communicate
with patients. Pg 1375
col2 para3
para 1)
When
surveyed,
PHCPs
identify a lack
of IT training
as a major
barrier to
using
technologies
they consider
valuable
(1375, col 2,
para 2)
Continuing
medical
education
courses
should be
developed
that teach not
only
computer
literacy skills
but also cover
stand alone
systemes pg 3
col 2 para 6
systems that
used a
computer to
generate the
decision support
were
significantly
more effective
than systems
that relied on
manual
processes pg 3
col 2 para 6
systems that
prompted
clinicians to
record a reason
when not
following the
advised course
of action were
significantly
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 212
successes….and
longer term
goals….. pg 1378
col 2 para3
of this risk
include the
financial cost
of these
systems to a
practice or a
payer and the
uncertain cost
benefit. Pg
1375 para5
Other
environmental
barriers that
are critical to
primary care
practices
include the
expense of
systems9 and
the cost of
customizing
systems for a
PHCP-based
practice
(1375, col 2,
para 1)
people and
more
organisational
areas, the
major
challenges to
systems
success often
become more
behavioural
than technical..
a “technically
best” system
can be brought
to its knees by
people who
have low
psychological
ownership in
the system
and who
vigorously
resist its
implementatio
n (1375 col 2
para 5)
more likely to
succeed than
those that
allowed the
system to be
bypassed
without
recording a
reason to
pg 3 col 2 para
6
systems that
provided a
recommendatio
n were more
likely to succeed
than systems
that provided
only an
assessment pg
3 col 2 para 6
issues such
as:
confidentiality
and security
on the
Internet, as
well as
existing
policies
affecting the
use of IT in
medicine;
benefits of
adopting
specific IT
tools; how to
evaluate IT
solns; where
to learn more
about IT.pg
1376 col 2
para 4
more likely to
succeed than
those that
allowed the
system to be
bypassed
without
recording a
reason to
pg 3 col 2 para
6
systems that
provided a
recommendatio
n were more
likely to succeed
than systems
that provided
only an
assessment pg
3 col 2 para 6
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 213
These
problems
include
working in a
delivery
system that
rewards time
efficiency
more than the
quality of care
provided, and
incurring the
financial
burden of
costly IT with
a limited
potential for
return on
investment
(pg 1375, col
1 para 1)
access to IT is
influenced by
the financial
situation pg
137
their lack of
ownership may
be due to a
variety of
factors,
including a
lack of insight
about the
benefits of IT,
their concern
about the
magnitude of
change
caused by IT,
and their
ambivalence
about the
processes that
IT is designed
to improve (pg
1376, col 1
para 1)
Appreciating
the magnitude
of change .
Primary care
physicians
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 214
137
5 col 2 para 1
may be lesss
enthusiastic
about the pace
with which
benefits are
realised after
implementing
IT. For eg.
Physicians
have found
that newe
systems result
in increased
patient waiting
time and staff
workload for a
while after the
system is
implemented.
pg 1376 col 2
para 2.
access to IT is
influenced
by….the
average age of
the PCHPs –
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 215
with older
PHCPs less
likely to have
access to (or
comfort with)
technology pg
1375 col 2
para 1
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 216
Conditions Prior
To
implementation
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of Use of
System
Security/Confidenialit
y and standards
Education and
Training
Technological
Issues
Communication
Issues
Organisational
Issues
Other
8 Kawamoto
2005
�
Alignment of
decision support
objectives with
organisational
priorities and
financial
interests pg 3
table 2
System
developed
through iterative
refinement
process pg 3
table 2
Active
involvement
of local
opinion
leaders pg
3 table 2
a common
theme among all
four features is
that they make it
easier for
clinicians to use
a decision
support
system….as a
general principle
then our findings
suggest that an
effective clinical
decision support
system must
minimise the
effort required by
clinicians to
receive and act
on system
recommendation
s. Pg 7 col 1
para 5
assessments and
recommendations are
accurate pg 3 table 2
System
developed
through
iterative
refinement
process pg 3
table 2
interventions
succeeded
when decision
support was
provided to
clinicians
automatically
pg 3 col2 para
6
similarly,
systems that
were provided
Alignment of
decision
support
objectives with
organisational
priorities and
financial
interests pg 3
table 2
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 217
para 5
clinician –
system
interaction
features: 1.
saves clinicians
time or requires
minimal time to
use; clear and
intuitive user
interface with
prominent
display of advice
pg 3 table 2
interventions
succeeded when
decision support
was provided to
clinicians
automatically pg
3 col2 para 6
similarly,
systems that
as an
integrated
component of
charting were
significantly
more likely to
succeed than
stand alone
systems pg 3
col 2 para 6
….computer
to generate
the decision
support were
significantly
more effective
than systems
that relied on
manual
processes pg
3 col 2 para 6
systems that
provided a
recommendati
on were more
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 218
were provided as
an integrated
component of
charting were
significantly
more likely to
succeed than
stand alone
systems pg 3 col
2 para 6
system is fast –
pg 2 table 2
computer to
generate the
decision support
were significantly
more effective
than systems
that relied on
manual
processes pg 3
col 2 para 6
systems that
likely to
succeed than
systems that
provided only
an
assessment
pg 3 col 2
para 6
systems that
prompted
clinicians to
record a
reason when
not following
the advised
course of
action were
significantly
more likely to
succeed that
allowed the
system to be
bypassed
without
recording a
reason pg 3
col 2 para 6
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 219
prompted
clinicians to
record a reason
when not
following the
advised course
of action were
significantly
more likely to
succeed that
allowed the
system to be
bypassed
without recording
a reason pg 3
col 2 para 6
systems that
provided a
recommendation
were more likely
to succeed than
systems that
provided only an
assessment pg 3
col 2 para 6
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 220
Conditions
Prior To
implementatio
n
Cos
t
Need and
importanc
e of
validation
and
evaluation
Attitudes Ease of Use
of System
Security/Confidenialit
y and standards
Education
and
Training
Technologica
l Issues
Communicatio
n Issues
Organisational
Issues
Other
9 Kukafk
a 2003
�
factors within
the
organisational
infrastructure
required to
enable usage
eg available
resources,
supportive
policies and
accessibility to
the IT system.
Pg 224 col 1
para 1
in addition,
they must
ensure that the
users are
�
predisposing
factors….
Include
perceived
usefulness,
pg 223 col2
para2
in addition,
they must
ensure that
the users are
prepared in
terms of
knowledge of,
and belief, in
the the new
system pg
226 col 2 1st
predisposing
factors….
Include,
ease of
use…pg
224, col 1
para1
Construction
of the
system’s
decision-
support rules
should be
consistent,
salient, user-
friendly and
customizabl
e for a wide
range of
skills that
an
individual
needs to
use a
system pg
224 col 1
para 1
usage
inducing
strategies,
eg skills
training pg
223 col 2
para 2.
phase 4:
factors
issues may relate to
phase 1:
organisational
needs and goals, in
terms of engaging
stakeholders. Pg
222 para 3
phase 2:
organisational
needs amenable to
IT system
solutions…Detailing
the system
….functionality…an
d how they relate to
perceived needs of
end users pg 223
para 2
phase 5:
system use
inducing
strategies
focuses on
developing
and
implementin
g
approaches
that are
proactive
and
specifically
targeted to
influencing
favourably
the
predisposing
, enabling
and
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 221
users are
prepared in
terms of
knowledge of,
and belief, in
the the new
system pg 226
col 2 1st para
the healthcare
organisation
also must
prepare its
workforce for
major changes
that are
associated
with the
instalment of a
new
innovation.
They must
prepare the
environment
for change and
then assist in
the adoption of
that change by
providing the
226 col 2 1st
para
reinforcing
factors follow
a behaviour
and provide
the reward or
incentive for
the repetition
or
persistence of
the
behaviour. Pg
224 col 1
para 1
is more
clearly
reflected in
phase 3:
behaviours
linked with
system use,
where
reference is
made to
collaboration
users (p101
col 2 para 3)
associated
with
behaviours
under
“enabling
factors”
,examples
of which
include
skills
…..that
enable IT
usage. Pg
223 para 4
para 2
phase 3 :
behaviours linked
with system use, to
define the actual
steps through which
a system user will
have to move pg
223 3rd para
factors within the
organisational
infrastructure
required to enable
usage eg available
resources,
supportive policies,
and accessibility to
the IT system pg
224 col 1 para 1
phase 4: factors
associated with
behaviours under
and
reinforcing
factors
identified in
phase 4.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 222
providing the
economic
resources,
incentives and
social support
to facilitate the
change pg 236
col 2 1st para
Phase 1 an
assessment of
the
organisational
needs and
goals pg 222
col 2 para 1
collaboration
patterns,
practical and
social roles
pg 223 para 3
phase 4:
“predisposing
factors are
mostly
psychological
, including
cognitive
dimensions of
knowing,
feelings,
believing, and
having a
sense of
efficacy. They
are the
antecedents
to behaviour
that provide
the basis for
motivation.
Pg 223 col 2
last para
“enabling factors”
,examples of which
include… other
organisational
structure that enable
IT usage. Pg 223
para 4
the results of our
analysis indicate the
necessity for
developers to focus
on the needs and
goals of the
organisation (phase
1) pg 236, col 1 last
para
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 223
Conditions
Prior To
implementation
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of Use
of System
Security/Confidenialit
y and standards
Education and
Training
Technologica
l Issues
Communication
Issues
Organisational
Issues
Other
10 Leat
t
2006
�
Strong
management
commitment
(p96, under
mgt support-
EMR
Management
support should
involve clinical
as well as
administrative
leadership pg
96 col 2 para 5
Size of practice
and
organisation
(p98, para1&4)
�
financial
resource
availability –
commit
sufficient
financial
resources for
hardware,
software, and
reduced
productivity;
avoid
underestimatin
g necessary
financial
resources;
include costs
for training,
technical
support, and
productivity
losses,
integrate CQL
processes fig 2
pg 97
Employment of
continuous
quality
improvement
processes to
routinely
monitor and
assess
implementatio
n processes
and new
technology
(p100 col 1,
para 1 under
MAR)
���
Since
experience
has shown
that
resistance
to CPOE
comes
primarily
from
physicians
and other
staff who do
not want to
change their
work
processes
(96, under
mgt
support-
CPOE)
�
System
accessibility,
efficiency of
use and
ease of
navigation
(p99 col 2
para 2)
construction
of the
systems’
decision
support roles
should be
consistent,
salient, user
friendly, and
customisabl
e for a wide
range of
develop new practice
standards and
routines fig 2 pg 97
Scheduling
fewer patients
during the
learning period
and ensuring
protected and
adequate time
for training
(p98 para2;
p98 col2 –
p99). Training
initially but
also ongoing
(also p100 col
1 para 2)
Use of one-to-
one
implementatio
n training
sessions (p100
col1 para1)
involve key
individuals in
the EMR
system
design fig 2
pg 97
Ongoing
technical
support and
back-up (p99
col2 para3;
p100 col 1
para2)
Iterative
refinements
or revisions
to the system
(p101 col2
para 3)
communicate
about
expectations and
experiences fig 2
pg 97
capitalise on the
ability of
individuals holding
dual roles (ie
clinical and
administrative) to
bridge gaps
between groups of
stakeholders fig 2
pg 97
Role of leaders in
communicating
and connecting
with staff and
addressing
Establishment
of
interdisciplinary
committees
showing
support (p97,
last para; p100
top of col 2)
‘studies
identified the
importance of
organizational
preparation for
reductions in
productivity,
both during and
after
implementation
, as critical to
the effective
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 224
‘studies
identified the
importance of
organizational
preparation for
reductions in
productivity,
both during and
after
implementation
, as critical to
the effective
implementation
of EMR’ (p98
para2)
Culture of
change and
innovation (p98
col2 para2)
also fig 2. pg
97
Communicating
the reasons for
adopting and
losses,
emphasise
patient safety
gains to justify
financing;
secure
sufficient
resources for
implementation
processes FIG
1 pg 97
Cost and
funding (p98,
all of col 1)
Users’
expectations
of the
system (p99
col 1 para3)
Shared
belief in
system’s
potential
(p98 col2
para4)
Physician
champion
(p96, under
mgt
support-
EMR)
Previous
experience
of
computers –
mixed
findings
users pg 101
col 2 para 3
design the
system for
efficiency of
use and
avoid
interfering
with patient
care fig 2 pg
97
provide
adequate,
timely and
ongoing
training,
protect time for
training, fig 2
pg 97
ensure EMR
system back
up fig 2 pg 97
provide
ongoing on
site technical
support fig2
pg 97
involve key
individuals in
the EMR
system
design
design the
system for
efficiency of
use and
avoid
interfering
with patient
care fig 2 pg
97
addressing
workflow concerns
(p97, para1)
it is also important
to maintain
bidirectional
communication
throughout the
process to enable
staff to see current
measures
indicating cost
savings and error
reductions, as well
as to ensure that
staff are providing
input about the
system’s
usefulness and
potential
modifications(p10
0 col1 para3)
Physicians’
concerns that
system would
interfere with or
implementation
of EMR’ (p98
para2)
Shifting
organisational
priorities to
support system
(p98 col 1 last
para)
Substantial
reconfiguration
of roles,
responsibilities
and work tasks
among staff in
various
departmental
units (p100 col
2 last para
before
discussion)
develop
committees
and teams that
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 225
adopting and
implementing
CPOE .In
addition to
conveying
expected cost
savings and
reductions in
errors prior to
implementation
, however ,
(p100 col1
para3)
Partnerships
between
vendors and
purchasers
(p100 col 2 top)
Preparation of
staff for
changes in
responsibilities
as well as
patterns and
(p99 col 1
para2)
foster
positive
perceptions
of MARS
potential to
decrease
medication
errors and
turnaround
time for
medication
orders fig 2
pg 97
staff
resistant to
change pg
96 under
mgt support
CPOE
encourage
participation
in the design
of the system
by clinicians
pg 97 fig
2/inclusion of
appropriate
individuals in
the design of
the system
pg 99 col 2
para 2
inclusion of
appropriate
individuals in
the design of
the system
pg 99 col 1
last para.
construction
of the
systems
decision
interfere with or
negatively impact
the physician-
patient encounter
(p99 col 2 para 2)
and teams that
meet and work
over an
extended
period of time
fig 2 pg 97
develop formal
goals,
objectives and
key indicators
of success fig 2
pg 97
foster a climate
reflecting that
the new system
is being
promoted
supported and
rewarded fig 1
pg 97
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 226
priorities of
work (p101 col
2 just before
conclusion)
Inclusion of
appropriate
individuals in
the design of
the system
(RI?) (p99 col 1
last para)
involve key
individuals in
the EMR
system design
design the
system for
efficiency of
use and avoid
interfering with
patient care fig
2 pg 97
solicit clinician
input on system
decision
support rules
should be
consistent,
salient, user
friendly and
customizable
for a wide
range of
users pg 101
col 2 para 3
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 227
design to
facilitate buy in
fig 2 pg 97
encourage
participation in
the design of
the system by
clinicians pg 97
fig 2/inclusion
of appropriate
individuals in
the design of
the system pg
99 col 2 para 2
clear and
formal goals
and objectives
and
communication
strategies pg
100 col 1 under
MAR
anticipate
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 228
challenges to
implementation
fig 2 pg 97
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 229
Conditions
Prior To
implementatio
n
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of Use
of System
Security/Confidentialit
y and standards
Education
and Training
Technological
Issues
Communicatio
n Issues
Organisational
Issues
Other
11 Lu
2005
Cost-saving
(sec 4.1.1)
Cost-
effectivenes
s (Table 2)
Maintenance
: repair costs
constitute
another
barrier
(Table 4 pg
415)
Clinical
impact: the
literature
suggests that
utilisation of
PDAs saves
clinicians time
in regard to
accessing,
retrieving, and
recording data
4.1.5 pg 414
Clinical impact
(quality of
care) It can
help health
care providers
to better take
care of their
patients by
improving their
Personal
factors (eg
comfort with
technology,
comfort with
device, age,
memory
problem
(forgot to
carry the
device):
Dependency
or over
reliance on
the device
table 4 pg
415. (Table
4) pg 415
lack of
needs or
motivation
Usability pg
409
summary.
Time-saving
for
physicians,
easily
incorporated
into workflow
(4.1.3)
Time saving
table 2 pg
413
Real-time
access to
patient
information
Security concerns.
Summary pg 409
Security and speed of
wireless transmission:
an integrated
input/output device
with data encryption at
both ends of
transmission should
be developed to meet
security needs table 4
pg 415.
Experts estimate that
wireless technology
will become an
essential component
of hospital operations
within the next few
years, after some
Educational
benefit (-
reported
improvemen
t in
participants
educational
experience
(4.1.2
Seamless
integration of
PDA
technology with
hospital
information
systems
summary pg
409
Barrier in
cases of
technology not
integrated with
EMRs (Table
4)
Inadequate
technology
support or
access barriers
Benefits in
communication
(Table 2) pg
413
alternative
methods used
or competing
technologies
eg some
physicians
prefer to write
notes on paper
than PDAs.
Table 4 pg 415
Lack of
technical and
organisational
support
summary pg
409 summary
Organisational
barriers, eg.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 230
personal and
professional
information
mangement,
providing
medical
decision
support via
acces to
evidence
based
materials and
allowing
remote access
to patient
data.(4.1.5) pg
414
Evidence-
based practice
support (Table
2) pg 413
Ineffectivenes
s table 4 pg
415
Table 4 pg
415
Most health
professional
s perceive
PDAs as
useful tools
that can
enhance
their practice
by offering
mobility and
functionality
in a small
device that
fits into one’s
pocket. 4.2
pg 414
Negative
patient
perception
(Table 4)
(Table 2)
Mobility: In
the
healthcare
environment,
PDAs allow
the clinicians
to have
access to
patient
information
whenever
and
wherever
they prefer,
to record
patient data
in real time,
to look up
refs and to
communicat
e with
colleagues.
Table 2 pg
413
Data entry –
unintuitive
major issues,
bandwidth, availability
and security are
resolved. Pg 416 col 1
para 2.
Error reduction (4.1.4)
Increased safety or
reduce medical errors.
Table 2 pg 413.
Table 4 pg 415
Physical
Design : size,
weight and
small screen.
Table 4) pg
415
Technical
difficulties:
difficulties in
gaining internet
access at
times; and
synchronizatio
n with home
computers
table 4 pg 415.
Experts
estimate that
wireless
technology will
become an
essential
component of
barriers, eg.
Lack of
institutional
support and
concerns
about legal
issues. Table 4
pg 415
Some barriers
can be
eliminated by
organisational
changes such
as providing
the necessary
infrastructure
for the
handhelds,
technical
support, and
funding for the
devices and
software
applications.
Pg 416 col 2
para 1.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 231
wider
organisational
or institutional
adoption is
relatively slow,
perhaps
because there
still relatively
little published
evidence of a
positive impact
of handheld
computers on
patient
outcome.
More PDA
evaluation
studies are
also
encouraged to
explore
expanding
roles and
clinical
impacts of the
technology in
health care.
and not easy
to use table
4 pg 415
Usability
issues, eg
difficult in
use, limited
memory,
screen size.
Table 4 pg
415
For example,
Larkin
reported that
if a PDA
could not fit
into a
physician’s
workflow
seamlessly
or if it
required
extra effort,
the physician
was less
likely to use
component of
hospital
operations
within the next
few years, after
some major
issues,
bandwidth,
availability and
security are
resolved. Pg
416 col 1 para
2.
Greater
efficiency and
increased error
reduction will
be achieved if
data capture or
data entry are
permitted
wherever and
whenever it is
available table
2 pg 413.
More and
more health
care
professionals,
clinics and
hospital
organisations
are
considering
the purchase
and
implementatio
n of handheld
technologic
solutions to
save time,
improve
medical data
access, assist
with medical
information
management,
improve
patient care
efficiency and
reduce
medical errors.
Pg 416 col 1
para 3
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 232
232
Pg 417 col 2
para 1.
likely to use
the device.
Pg 416 col 1
para 4
section 4.3
Advances in
technology
and usability
help
overcome
some of the
barriers to
adoption pg
416 col 1 last
para.
Customisation:
various
functions or
programs can
be added to
support
different
medical
specialities.
Table 2 pg 413
Delicate
devices: fears
of breaking the
device make
some users
limit their uses
to avoid
damaging it.
Table 4 pg
415.
Personal
factors:
physical factors
eg large fingers
(too big for the
Better
designed PDA
hardware and
software
applications
summary pg
409
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 233
buttons) and
poor eyesight
(cannot read
the small fonts)
table 4 pg 415
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 234
Conditions Prior
To
implementation
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of Use
of System
Security/Confidenialit
y and standards
Education and
Training
Technological
Issues
Communication
Issues
Organisational
Issues
Other
12 Ohinmaa
2006
Planning
readiness (incl.
telemedicine
strategic plan,
needs
assessment
analysis and a
business plan)
Workplace
readiness (incl.
designating a
telemedicine co-
ordinator and
ensuring change
management
readiness) pg 42
para 2
Statewide
cooperative
Allowed more
patients to be
treated at
lowered costs
pg 42 para3
Reimburseme
nt pg 42 para
4
For costing it
is imp that the
non clinical
use be
considered in
the allocation
of investment
as well as line
and staff
assessment
of these
programmes
had shown
significant
cost savings,
a decrease in
……turnaroun
d time and
improved
productivity
pg 42 para 6
Lack of “buy
in” ,
resistance to
change pg 42
para2
Ownership,
attitudes of
participants,
professional
connections
remote
location pg 42
para 4
Significantly
reduced time
for the PACS
support staff
to transfer
imaged….red
uced the
clinical
decision-
making time
(esp in
neurosurgery)
pg 42 para1
Systems have been
shown to be safe pg
42 para3
Liability requirements
pg 43 para 5
Lack of staff
preparation pg
42 para 2
Limitations of
technology
associated
with pt
selection (eg
severely
demented pts
who could not
sit still) pg 42
para4
Communication
between
clinicians has
also improved pg
42 para 1
one of the
biggest
difficulties in
adapting
telemedicine
applications
from abroad is
the difference
in the health
care delivery
systems. Pg
43 para 4 col
1
Improved
productivity in
healthcare pg
42 para6
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 235
systems that
provide
scheduling
support pg 43
para 6
Changes in health
care legislation by
government
agencies pg 42
para 5
Directed,
systematic
government
policy aiming to
increase
investment in
technology can
enhance
networking and
collaboration
within the
healthcare
system pg 43
para 1
Government
charges for
each type of
service. Pg 43
6
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 236
money to build
infrastructures
such as
broadband
networks or
statewide
cooperative
systems that
provide
scheduling
support. Pg 43
para6
Changes in health
care legislation by
government
agencies pg 42
para 5
Workplace
readiness (incl.
designating a
telemedicine co-
ordinator and
ensuring change
management
readiness) pg 42
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 237
para 2
Distance is a key
factor in the use
of telemedicine
pg 43 para 2
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 238
Conditions
Prior To
implementation
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of Use
of System
Security/Confidenialit
y and standards
Education and
Training
Technologica
l Issues
Communicatio
n Issues
Organisationa
l Issues
Other
13 Papshev
2001
Fragmentation
of the health
care market
(p731) and
table 3 pg 729
Regulatory
indecisiveness
(p 733)
regulatory
indecisiveness
table 3 pg 729
and pg 735
summary
Considerable
funding
requirements,
segmentation
of healthcare
markets, lack
funding
requirement
s table 3 pg
729
considerable
investment
capital pg
735
summary
Funding
requirement
s (p731)
Retail profits
may be
unfavourably
impacted
(p733,
column 2,
para 1).
Unification of
point of care
and point of
service
processes (p
728). “electronic prescribing would enable prospective drug utilisation review (DUR) to service its intended purpose in a most efficient manner. Advise pharmacists of potential therapeutic conflicts that should be resolved
providers
resistance to
change Pg 725
abstract data
synthesis and
table 3 pg 729
and pg 732
and pg 735
summary
potential
solutions
include gaining
stakeholder
support in
implementatio
n of the
technology. Pg
725 abstract
data synthesis
through
eliminating
these
redundant,
time
consuming
steps,
electronic
prescribing
should allow
practitioners
to focus on
providing
care rather
than on data
management
. Pg 729 col
1 para 1
Efficiency,
productivity
(p728 final
Lack of technology
standardization
(p732) and table 3 pg
729 and pg 735
summary
potential solutions
include establishing a
standardising
warehouse or a
router. Pg 725
abstract data
synthesis and pg 733
Familiarizatio
n with the
technology (p
733 line 3)
Products and
technology
pp 726 – 7
Pharmacists:
potential for
improved
communication
with the patient
(p733 para 3
column 1).
Improved
quality of care
pg 729. and
table 3 pg 729
and pg 735
summary
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 239
of technology
standardisation
, and regulatory
indecisiveness
create
boundaries to
the widespread
use of
automated
prescribing
Pg 725 abstract
data synthesis
resolved by physicians. If DUR initiated at the point medications are prescribed, a higher intervention rate could be achieved.
Reduction of
medication
errors (p 729)
and table 3 pg
729 and pg
735 summary.
computerised
prescribing
can provide
immense
benefits to
healthcare
providers,
patients and
managed
care. Pg 735
summary.
Stakeholder
buy in.
Stakeholders
include MCOs,
legislators,
providers and
patients pg
734
Increased
patient
satisfaction (p
731)
“facilitating
seamless
resolution of
formulary and
prior
authorization
issues …
healthcare
providers had
more time to
devote to their
patients, while
patients were
less frustrated
in efforts to
paragraph)
elimination of
time gap
between
point of care
and point of
service table
3 pg 729 and
pg 735
summary
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 240
“a means of monitoring and restricting physicians’ prescriptive authority” p 733 para 1.
receive
medications
prescribed by
their
physicians”.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 241
Conditions
Prior To
implementatio
n
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of Use of
System
Security/Confidenialit
y and standards
Educatio
n and
Training
Technological
Issues
Communicatio
n Issues
Organisational
Issues
Other
14 Peleg
2006
Establish
public policies
that provide
incentives for
implementing
CDSS p 76
middle column
and
designin
g them to
be cost
effective
pg 73
middle
col para
1
lack of
improved
performance
could be due
to any number
of factors,
such as lack of
support among
colleagues pg
78 col 1 para 1
lack of
improved
performance
could be due
to any number
of factors,
such as
human-
computer
interface
“include
patients’
perspective
s of their
health
problems
and
preferences”
p 76 column
3
“physicians
ability to
change the
knowledge
base” p 76
3rd column
para 2
timely advice
should be
provided p 76
middle column
and last col
speed is
everything p 76
middle column
bottom
anticipate
needs and
deliver in real
time p 76
middle column
bottom
simple
interventions
sharing is enhanced
through standards pg
74 col 2 para 3
implementation of
CDSSs are greatly
aided by
standardization in
information system
infrastructure,
including standard
terminology, data
model, data exchange
format, and other
clinical information
systems services. Pg
77 col 1 para 2
in which
modellers
represent the
medical
knowledge that
enables the
CDSS to deliver
appropriate
decision support
services during
the care
process. Pg 73
col 3 .
provision of a
direct
recommendatio
n rather than
just an
assessment that
is presented to
the clinician for
in this context,
the goals of
modern cdss
go beyond the
original focus
of producing
expert level
advisories and
extend to
include….
facilitating
communication
among
providers pg 73
middle column
para 1
that
implementers
of CDSS can
use to work
through the
process of
identifying
stakeholders,
pg 73 col 3
bottom
workflow
integration
should be
considered. p
76 middle
column and
also bottom of
column and
also 3rd column
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 242
interface
problems or
lack of time pg
78 col 1 para 1
when
evaluators
were not also
the system
developers,
the proportion
of systems
reporting
improvement
dropped
significantly pg
78 col 1 para 2
it is difficult to
distinguish
effects caused
by the CDSS
from effects
caused by the
change in the
work practices
respond to
user needs
p 76 last
column 2nd
para
Need to
consider
patient
preferences
pg 73
middle col
para 1
work best p 76
middle column
p 76 middle
column bottom
automatic
decision
support as part
of clinician
workflow pg 76
col 3 para 3
provision of
decision
support at the
time and
location of
decision making
pg 76 col 3 para
3
The
implementation
s of these
rationalistic
technological
consideration pg
76 col 3 para 3
to be effective
and successful
CDSSs need to
be integrated
into health
information
systems that
supply the
patient data
CDSSs need.
Pg 77 col 1 para
2
Lots of technical
aspects
The knowledge
modelling tasks
involves
elicitation,
representation,
sharing,
evolution, and
delivery of
a shift toward
specialised
and focused
system,
interacting
systems that
are integrated
into the clinical
environment
and workflow
pg 73 col
middle para1
“elucidates …
the information
flow and the
work flow, the
roles and
responsibilities
, and the
communication
and co-
ordination
patterns of the
care process”
p73 3rd
column.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 243
induced by the
implementatio
n of CDSS.
Pg78 col 1
para 3
Clinical effects
and costs of
the system
should be
evaluated p 76
middle column
the importance
of evaluating
CDS pg 73
middle column
para 1
that
implementers
of CDSS can
use to work
through the
process of
monitoring
interventions
required a
“disciplined
practice” where
clinicians
entered well
defined input
data at
appropriate
times and the
output of the
systems is
realisable in the
clinic. The
conflict between
these
requirements
and the
evolving,
contingent,
emergent
nature of
medical work
contributed
towards
difficulties in the
adoption of
CDSSs/ pg 73
col 1 bottom of
delivery of
knowledge to
users pg 74 col
1 para 2
The system
should be
developed with
an ability to be
maintained and
extended p 76
middle column
Capture
evidence in
machine
interpretable
knowledge
bases p 76
middle column
Develop
maintainable
foundations for
computer based
decision support
The
introduction of
the system
caused an
increase in the
number of
coordination
and verification
requirements,
sharing of login
session by
different users,
and the
disruption of
workflow due
to the
geographical
locations of the
clinical
workstations.
Pg 78 col 2 1st
para
“potential
affect on the
division of
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 244
specific CDS
interventions
pg 73 col 3
bottom
clinical effects
and costs of
the system
should be
evaluated p 76 middle column mentioned twice in this section
monitor
impact, get
feedback and
respond p 76
middle column
bottom
the complexity
of medical
practices and
the high cost
of
implementing
page.
p 76 middle
column
Flexibility p 76
last column 2nd
para
work among
care providers”
p 73, middle
column, top
“how CDS
would shape,
and in turn be
shaped by the
organizational
structure and
practices of
providers” p 73
middle column
top
Elucidates the
organization
goals, p73 3rd
column near
top
….that
implementers
of CDSS can
use to work
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 245
implementing
CDSSs make
evaluation of
CDSSs both a
challenge and
a necessity.
Pg 77 col 2
bottom para
through the
process of
determining
the goals and
objectives of
the CDSS,
cataloguing the
host
information
systems
capabilities
and selecting,
deploying
specific CDS
interventions
pg 73 col 3
bottom
the main
challenge in
implementing
CDS is not so
much trying to
fit CDS into
existing
workflow, as it
is managing
the ongoing
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 246
process of
organisational
development
that was
triggered by
the CDS
intervention.
Pg 74 col 1
para 1
CDSSs are
part of a
knowledge
management
toolkit that a
healthcare
organisation
can employ to
deliver the
“right
knowledge to
the right
people in the
right form at
the right time.
Pg 73 last col
near top
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 247
much of the
current
research
emphasises
the importance
of modelling
the integration
of cdss with
the
organisational
workflow and
information
systems. Pg
75 col 2
bottom
integration into
IT environment
workflow
integration
should be
considered. p
76 last column
2nd para
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 248
Conditions
Prior To
implementatio
n
Cost Need and
importanc
e of
validation
and
evaluation
Attitudes Ease of Use of
System
Security/Confidenialit
y and standards
Education
and
Training
Technological
Issues
Communicatio
n Issues
Organisationa
l Issues
Other
15 Shekell
e 2006
Additional
barriers
included
difficulty with
financial
incentives Pg
57 para 3
Cognitive and
or physical
barriers
include
physical
disabilities .pg
57 para 2
Challenges for
adoption of
electronic health
records…..include
d cost, . Pg 57
bottom para
Misalignment of
costs and benefits
(p58 top line)
Situational barriers
included time and
financial
pressures,
unproven return on
investment,
insufficient access
to the internet or to
Additional
barriers
included
difficulty with
physician
attitudes. Pg 57
para 3
physician and
organizational
resistance due
to the perceived
negative impact
on the
physician’s
workflow” p 57,
para 3.
satisfaction
satisfaction
(physician) was
correlated most
strongly with the
ability of the HIT
system to
perform tasks in
a
“straightforward”
manner pg 57
para 1
Liability barriers
included ….Concerns
about privacy and
confidentiality (p 57,
bottom para) and pg
57 para 2
Challenges for
adoption of electronic
health
records…..included
cost, technical issues,
system
interoperability. Pg
57 bottom para
Cognitive
and or
physical
barriers
include
insufficient
computer
skills.pg
572nd
para).
Lack of a
well
trained
clinical
informatic
s
workforce
to lead the
process
Situational barriers
included software
not being
supportive of
pediatric practice
needs Pg 57 para
2
Challenges for
adoption of
electronic health
records…..include
d cost, technical
issues, system
interoperability. Pg
57 bottom para
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 249
computer
technology in the
office setting, the
prohibitive cost of
information
technology for
practices. Pg 57
par 2 high cost pg
57 para 3
(physician) was
correlated most
strongly with the
ability of the HIT
system to
perform tasks in
a
“straightforward”
manner pg 57
para 1
(p57
bottom
para)
Additional barriers
included
complementary
changes in
support, electronic
data exchange. Pg
57 para 3
Product / vendor
immaturity pg 57
para 3
Additional barriers
included difficulty
with technology,
Pg 57 para 3
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 250
Conditions
Prior To
implementation
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of
Use of
System
Security/Confidenialit
y and standards
Education and
Training
Technologica
l Issues
Communication
Issues
Organisational
Issues
Other
16 Studer
2005
Prior to
implementation
it may be
beneficial to
assess
expectations
and concerns
regarding EMR
system
implementation
. Pg 95 col 2
para4
Appropriate
individuals
throughout the
organisation
should be
included in the
design of the
EMR sytem
and
Lack of
financial
incentives and
rewards is a
barrier (p 96
col 1 para 4)
Financial
resource
Availability.
Costs
Associated
with EMR
System Start
UP and
Ongoing
Maintenance
are a major
barrier (p 94
col 2 last
para– 95 col 1
1st para);
It is important
to determine
the effect of
EMR
implementatio
n of patient
safety, clinical
quality and
costs. Pg 97
col 1 para 3.
“A
physician
champio
n is
essential”
p 94
column 2
para 6
The EMR
system
must be
accessible
, efficient
to use and
not
interfere
with or
negatively
impact the
physician
– patient
encounter
(p 96,
column 1,
bottom
para).
EMR system
downtime: exposed
physicians and
practices to risk of
patient injury and
liability (p 96, column
2).
concerns regarding
the privacy and
confidentiality of
patient information
stored in an EMR pg
96 col 2 para4
lack of standards for
data coding and
exchange pg 96 col 2
para 4
Subsequent
communication
and training
should address
unrealistic
expectations
and concern
and convey a
clear statement
of the goals for
and anticipated
benefits of
EMR
implementation
. Pg 95 col 2
para4
Training for
EMR system
users should
be adequate,
timely, tailored
EMR system
downtime:
exposed
physicians
and practices
to risk of
patient injury
and liability (p
96, column
2).
Subsequent
communication
and training
should address
unrealistic
expectations
and concern
and convey a
clear statement
of the goals for
and anticipated
benefits of
EMR
implementation
. Pg 95 col 2
para4
the importance
of considering
work flow
design as part
of the EMR
implementatio
n process pg
96 col 2 para 4
Management
support “strong
management
commitment is
critical and
should be
broadly
communicated
through the
organization” p
94. col 2 para
4
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 251
implementation
planning
process and
physician buy
in must be
obtained. Pg 96
col 1 para 6
Implementation
climate theme:
organisations
with a culture of
change and
that value
innovation may
have a greater
likelihood of
effectively
implementing
and emr
system (p 95,
column 1, para
5).
1st para);
Financial
resource
Availability
Theme
Number 2 :
The
Commitment
of Adequate
Financial
Resources,
including both
hardware and
software costs
and costs
associated
with reductions
in productivity
during and
after EMR
system
implementatio
n are vital. Pg
95 col 1 para 2
… available on
an ongoing as
needed basis
and include
simulated
patient
encounters
(p95 column 1,
bottom para).
There must be
Sufficient
protected time
for training for
all EMR system
users. (p 95
column 2, para
2 )
On going, on
site technical
support is
critical pg 96
col 1 para 2
4
preparation for
reductions in
productivity
during and
after
implementatio
n critical to
effective
implementatio
n (p 95, para
3).
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 252
Conditions Prior
To
implementation
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of Use of
System
Security/
Confideniality and
standards
Education and
Training
Technological
Issues
Communication
Issues
Organisational
Issues
Other
17 Vreeman
2006
End user
participation in
the development
process
fundamental to
implementation
success…for eg
a committee with
stakeholder and
user
representatives
that oversaw
system
implementation
P441 PLUS
para7 end user
participation in
the development
process. Pg 441
col 2 para3/
table 2 pg
440/pg 442 col 2
para
3recommendatio
financial
implications of
implementing
EHRs…the
financial
incentives and
disincentives to
implementation
are a major, if
not the most
important
concern. Pg 443
col 1 para 3
�
Computer
systems have
the potential to
introduce errors
pg 443 col 1
para 2
Plan and test
carefully to
ensure adequate
software and
hardware system
performance pg
443 col 2 para 6
Incorporating
workflow
analysis into
system
design…noted
�
Improved reporting
capabilities
…..facilitated
clinical decision
making for
individual patients.
Pg 438 col 2 para
5 and
Workflow or
behaviour
modification pg
441 para 2 eg.
Using the system
challenged
therapists prior
practices with
regard to
documentation and
reqd them to
change their
traditional
practices
Improved data
accuracy pg 440
para 4/ table 2 pg
440/ pg 442 col 2
para 2
security, privacy,
and confidentiality
were not prominent
issues in the
studies we
reviewed. Pg 443
col 1 para 3
likewise there is
widespread
recognition of the
crucial role of data
standards in health
care pg 442 col 2
para 3.
Workflow or
behaviour
modification pg
441 para 2…The
automated
process shifted
much of the
responsibility
…creating
workflow
inefficiencies
that reqd special
training and
system
modifications.pg
441 col 1 para 2/
table 2 pg 440
Staff training pg
441 para 6 and
pg 442 col 1
para 1./ table 2
pg 440
Software or
hardware
inadequacy.
For example
using donated
equipment or
using systems
with shared
mainframes
causing
problems pg
441 col 1 para
4/ table 2 pg
440
recommendati
ons: pursue
the efficient
capture of
coded data
pg 443 col 2
para 7
Improved inter-
departmental
communication
pg 440 para 3/
table 2 pg 440
enhanced
communication
between
physicians and
nurses was
noted…pg 442
col 2 para 1
Improved
reporting
capabilities
…cited the
capability for
more
comprehensiv
e reporting
that integrated
clinical and
administrative
data as a key
benefit.
…Helped
clinicians and
administrators
to be aware of
the current
departmental
workload
which helped
provide the
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 253
3recommendatio
ns: include end
users, especially
clinicians into in
the system
development
activities. pg 443
col 2 para 3
that workflow
shifts caused by
the system could
decrease clerical
work while
simultaneously
increasing the
responsibilities
of higher paid
professional
staff. Pg 442
para 2/ table 2
pg 440
Incorporating
workflow
analysis into
system
design…of
identifying
needed data
elements,
identifying how
collected, and
determining
whether
additional
elements are
practices
recommendations:
commit to data
standards pg 444
col 1 para 2
pg 440
Staff training
requiring
overtime on
nights and
weekends pg
441, para6.
recommendation
s: devote
significant
resources to
training pg 443
col 2 para 5
provide the
rationale for
how pts and
therapists
were
scheduled. Pg
438 col 2 para
5 and 6
Workflow or
behaviour
modification
pg 441 para
2…The
automated
process
shifted much
of the
responsibility
…creating
workflow
inefficiencies
that reqd
special
training and
system
modifications.
pg 441 col 1
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 254
needed to
support the
desired
functionality.. Pg
442 para 2/ table
2 pg 440/pg 442
col 2 para 3
para 2/ table 2
pg 440
Incorporating workflow analysis into system design…noted that workflow shifts caused by the system could decrease clerical work while simultaneously increasing the responsibilities of higher paid professional staff. Pg 442 para 2/ table 2 pg 440
computer
systems have
the potential
to disrupt
workflow pg
443 col 1 para
2
computer systems have
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 255
the potential to disrupt workflow pg 443 col 1 para 2
recommendati
ons:
incorpororate
workflow
analysis into
the system
design and
implementatio
n pg 443 col 2
para 3
recommendati
ons: devote
significant
resources to
training pg
443 col 2 para
5
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 256
Conditions
Prior To
implementatio
n
Cost Need and
importance of
validation and
evaluation
Attitudes Ease
of Use
of
Syste
m
Security/Confideniality
and standards
Education
and Training
Technologica
l Issues
Communicatio
n Issues
Organisational
Issues
Other
18 Yarborough
2007
Perceived
organisational
support are
positive
predictors of
perceived
usefulness. Pg
660 para 2
The most
common
reason cited
by physicians
for resisting
communicatio
n with patients
is the lack of
reimbursement
for providing
such a service.
Pg 651 para 3
Cost as a
barrier pg 659
para 2
Costs, lack of
standards
as…major
another major
impediment to
physician
technology
utilisation is
the lack of
documentatio
n suggesting
that available
technologies,
do, in fact,
increase
quality of care
pg651 para
3/lack of
empirical
evidence
linking
information
systems to
quality or
financial
improvements
Personal
characteristics
have also
been identified
as barriers to
physician
technology
acceptance pg
661 para
2….examples
anxiety
regarding
computers,
value a
physician
places on a
close patient
relationship,
lack of
familiarity and
knowledge of
available
resources pg
Reliability and
dependency as the main
concerns with adopting
such technology
(handheld devices) pg
662 para 1
Fears about litigation,
privacy/confidentiality….
. also serve as barriers
to physician technology
utilisation. Pg 651 para 3
Confidentiality and clear
policies regarding error
reporting pg 661 para 1
Lack of infrastructure or
regulations to deal with
such issues (litigation)
Lack of
adequate
organisationa
l support
….also
presence of
appropriate
training and
infrastructure
pg 660 para 2
The ability to
customise
and organise
the
knowledge
captured on a
local level is
critical for
physician
technology
acceptance
pg 664
summary
Poor system
fit with
practice
patterns pg
659 para 2
Misinformation
….also serve
as barriers to
physician
technology
utilisation. Pg
651 para 3
The unique
relationship
between a
physician and
his or her
patient is
different from
that found in
other
industries.
Many
physicians
value this
interaction and
Interruption of
traditional
practice
patterns pg 656
para2-5…in
particular
focusing on
systems taking
more time…eg
more time per
physician per pt
when using
information
systems
Time as a
barrier to usage
of handheld
computers pg
659 para 2
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 257
barriers to IT
adoption. Pg
660 para 3
The cost of
technology
adoption has
been largely
ignored in the
literature
specific to this
framework
(TAM)..the
cost of
physician time
and change in
practice
patterns have
been identified
as significant
barriers to
technology
acceptance.
pg 667 para
2/3/pg 667
para 3
. Pg 659 para
4, pg 664
summary
Although the
benefits of
technology
adoption are
widely
prophesized,
little to no
empirical
evidence to
support such
claims are
offered in the
literature. Pg
651 para 3.
Lack of
evidence of
benefits of IT.
Pg 656 para
2. and pg 659
para 4…lack
of empirical
evidence
linking info
661 para
2/pg664
summary
The most
common
reason cited
by physicians
for resisting
communicatio
n with patients
is the lack of
reimbursement
for providing
such a service.
Pg 651 para 3
The report
also suggests
that physician
satisfaction is
highly
correlated with
efficiency. Pg
656 para 4
also serve as barriers to
physician technology
utilisation. Pg 651 para 3
Costs, lack of standards
as…major barriers to IT
adoption. Pg 660 para 3
Computer experience
positive predictors of
perceived usefulness.
Pg 660 para 2
Social and personal
norms appear to predict
technology acceptance
while physician self
identity factors actually
suppress technology
acceptance. Pg 661
para 3
are hesitant to
give it up or
take time away
from the
experience due
to new
information
technologies.
Pg 667 para 3
Lack of time
as…major
barriers to IT
adoption. Pg
660 para 3 /pg
664 summary
Organisational
characteristics
significantly
influence
technology
acceptance pg
660 para 3.
For the
implementation
s of CPOE or
EMRs to be
successful, a
physician’s
practice
environment
must have a
collaborative
organisational
culture that
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 258
linking info
systems to
quality or
financial
improvements
.
Physician
perceptions of
the utility of
new
technology
must be
measured pg
668 para 2.
emphasises
teamwork. Pg
664 summary
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 259
Conditions Prior
To
implementation
Cost Need and
importance of
validation and
evaluation
Attitudes Ease of Use
of System
Security/Confidenialit
y and standards
Education and
Training
Technological
Issues
Communication
Issues
Organisational
Issues
Other
19 Yusuf Installation
strategy pg 264
table 1
Resources
pg 264
table 1
User
perception pg
264 table 1
User attitudes
pg 264 table 1
Leadership
and support
pg 264 table 1
/pg 265 col 1
para 3.
Ease of use
pg 264 table
1/pg264 para
1 col 1
Time
(response and
turnaround)
pg 264 table 1
/pg 264 col 1
para 3
System
Flexibility pg
262/pg 262
col2 para 2
System
Usefulness pg
264 table 1
User training,
roles, skills pg
262 col 2 para
4and5/ pg 264
table 1
Successful
implementations
attributed to
provision of
extensive,
continuous user
training and
support pg 264
para 6
Clarity of system
purpose pg 262
table 1/ pg 262
col 1 para 2
System purpose
pg 264 table 1
Organisational
factors
(leadership
and support,
internal
communicatio
n) pg 264
table 1
User
involvement
pg 264
table 1
Organisational
factors (inter
organisational
system) pg
264 table 1
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 260
264 table 1
/pg 264 para 2
col 1
Information
accessibility
and relevancy
pg 262 para 3
Information
quality/comple
teness/legibilit
y/format table
1 pg 264
Problems in
the EMR
implementatio
n indicated a
design that
interfered with
clinical
practices it
was supposed
to support pg
265 col2 para
Technical
support pg
264 table 1
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 261
1
Ability of
system to fit
with clinical
practice pg
265 para4/
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 262
Appendix 6 WP2 Information letter Task Group
Mair Page 1 06/01/2009 Version 2.
Understanding the Implementation and Integration of E-Health Services; Information Sheet for Task Group Participants Workpackage 2.
Thank you for taking the time to consider participating in this research project which has been funded by the NHS Service Delivery and Organisation R&D Programme. Before you decide it is important for you to understand why the research is being done and what it will involve. Please take time to read the following information carefully. Talk to others about the study if you wish.
This information sheet explains the purpose of the study, what will happen to you if you take part, and gives more detailed information about the conduct of the study.
Ask us if there is anything that is not clear or if you would like more information you can discuss this with our researcher. Take time to decide whether or not you wish to take part.
What is the purpose of the study?
The study examines those factors that act as facilitators and barriers to the use of e-Health systems by health care professionals. E-Health covers a range of types of service. In this project we will be looking at four specific aspects of e-Health that are particularly important to the NHS:
1. Management systems (this includes electronic medical record systems);
2. Communication systems (includes real time or store and forward communication systems such as e-mail, telecare and telemedicine systems)
3. Decision support systems.
4. Information systems (including use of the internet).
In this study we are looking particularly at: uptake of different types of e-Health services; roots of resistance (if any); views about the effects of e-Health services on the patient/provider relationship; and training and support needs.
Why have I been chosen?
We are interested in hearing about the experiences of health care professionals from a range of backgrounds and with different levels of experience with the range of e-Health systems listed above in order to get as clear an understanding as possible of the key issues facing health professionals when new e-Health services are introduced. You have been chosen because you are an active NHS health care professional.
What do I have to do?
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 263
If you are interested in taking part or would like to speak to someone about the study you should contact either Prof Mair or Dr Anderson, one of the researchers, working on the project.
If you decide to take part they will then arrange an appointment to discuss study participation, via telephone, at a time that is convenient to you. Study participation will involve attending a single focus group session with up to seven other health professionals.
The Focus Group will last approximately two hours and will take place at a local venue (travel expenses will be paid). Prior to attending the focus group you will be sent summary information relating to a literature review that has been recently undertaken about barrier/facilitators to e-Health service utilisation. During the focus group you will be asked to comment on the information provided and to provide your views regarding:
� ̈ Barriers to using e-Health systems in practice.
� ̈ Ways of helping health professionals to integrate use of e-Health systems within their routine work.
� ̈ Types of support necessary to help with successful introduction of e-Health services.
� ̈ Potential effects, if any, of the different types of e-Health systems (electronic records, telecare, decision support, and the internet) on the health professional/patient relationship.
When you arrive for the focus group, the researcher will ask you to fill in a consent form. This is to show that you understand what the study is about, that you are happy to participate, and that you are happy for us to record the focus group.
We will give you a copy of this information sheet and the consent form to keep.
What will happen during the focus group?
The focus group will consist of questions and discussion relating to the use of different types of e-Health services by health professionals as outlined above. The focus group will last about 2 hours and will be tape recorded.
Why is the focus group recorded?
We record the focus groups because it is hard to take notes of what people say, listen carefully and think all at the same time! After the focus group the recording will be listened to carefully and every word that participants and the researcher say will be typed down. We use this written record, to help us remember what people said. Once the study has concluded, all the original audio recordings will be destroyed.
Are there any benefits to helping with this work?
Although there may be no direct benefits to you personally, we hope that you find participation an interesting experience. The subject matter is relevant to your work as across the United Kingdom the NHS is committed to the adoption of new e-Health Services such as the use of electronic medical records and home telecare technologies. Your involvement will give us a better of understanding of the problems faced by health professionals when using new e-Health services and how difficulties, if any, can best be overcome. We believe this will be of use to health care professionals and
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 264
the wider NHS, providing a better understanding of the needs of health professionals whenever new e-Health services are being introduced and how best to support the introduction of such services.
Are there any disadvantages to helping with this research?
The main disadvantage is the time it will take; the focus group usually lasts about two hours and there will be some documentation to read prior to the meeting. We are however able to offer remuneration for the time of the interview.
Will my taking part in the study be kept confidential?
Yes. Everything you tell us during the focus group is completely confidential. All personal information – your name, names of colleagues or patients - or anything else which might identify you will be removed so that no-one can identify you personally. Only anonymised data from the focus groups will be used in future publications from the research team so that individual participants cannot be identified. All recordings and written records are treated as confidential material. They are stored securely within the University under the supervision of Prof Frances Mair and will not be played or shown to anyone outside the research team. .
Who is organising and funding the research?
The research if funded by the NHS Service Delivery and Organisation R&D Programme and is being organised by the University of Glasgow (Division of Community Based Sciences).
Who has reviewed the study?
This study was given a favourable review by the NHS Service Delivery and Organisation R&D Programme and has had full ethical approval.
How can I get more information about the study?
If you want to know about what the focus group will involve or about the study in general or you have any concerns about any aspect of the study, please get in touch with us. Our telephone number is: 0141 330 8317, please ask for Professor Frances Mair or Dr George Anderson. Our address is: Department of General Practice & Primary Care, University of Glasgow, 1 Horselethill Road, Glasgow G12 9LX
WP1 Invitation letter Task Group
General Practice and Primary Care Division of Community Based Sciences University of Glasgow 1 Horselethill Road Glasgow G12 9LX
Dear ,
Re: Research participation
We are writing to you to ask for your help with a research study that is being funded by the NHS Service Delivery and Organisation R&D Programme, and conducted by the Universities of Glasgow, Newcastle upon Tyne, Dundee and University College of London.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 265
The study is about NHS staff members’ experiences of using e-Health services in their every day work. e-Health covers a range of types of service, but for example, includes electronically facilitated systems for management (eg. electronic medical record systems), communication (eg. email/telecare/telemedicine), decision support (i.e. computerised programmes to guide decision-making) and other information systems, such as use of the internet and database systems. We are interested in gaining a better understanding of how NHS staff use such systems in their every day work, including perceived barriers to, and facilitators of effective use of e-Health systems for health care provision. We hope that the results of our research will be very important for guiding NHS managers and policy makers in ways of introducing e-Health services into the NHS so that it is easier for staff to use them as part of their every day working practice.
We are therefore inviting you to take part in this study by agreeing to take part in a task group (with between 4 and 7 other health professionals) to discuss your views and experience, if any of using e-Health technologies in your work. A detailed information sheet is provided so that you can gain a better understanding of what your participation in this study would involve, and thus enable you to make an informed decision about whether or not you would like to take part. If, however, you need further information to make a decision, or simply wish to speak to someone about this research, contact details are provided and we are happy to answer your queries.
We sincerely hope that you will be able to help with this research.
Yours faithfully,
Dr George Anderson
On behalf of Professor Frances Mair.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 266
Appendix 7 WP2 interview Invitation letter
General Practice and Primary Care
Division of Community Based Sciences
University of Glasgow
1 Horselethill Road
Glasgow
G12 9LX
Dear ,
Re: Research participation
We are writing to you to ask for your help with a research study that is
being funded by the NHS Service Delivery and Organisation R&D
Programme, and conducted by the Universities of Glasgow, Newcastle upon
Tyne, Dundee and University College of London.
The study is about NHS staff members’ experiences of using e-Health
services in their every day work. e-Health covers a range of types of service,
but for example, includes electronically facilitated systems for management
(eg. electronic medical record systems), communication (eg.
email/telecare/telemedicine), decision support (i.e. computerised
programmes to guide decision-making) and other information systems, such
as use of the internet and database systems. We are interested in gaining a
better understanding of how NHS staff use such systems in their every day
work, including perceived barriers to, and facilitators of effective use of e-
Health systems for health care provision. We hope that the results of our
research will be very important for guiding NHS managers and policy
makers in ways of introducing e-Health services into the NHS so that it is
easier for staff to use them as part of their every day working practice.
We are therefore inviting you to take part in this study by agreeing to take
part in an interview to discuss your views and experiences of using or
implementing e-Health services. A detailed information sheet (Version 1,
dated 30/07/07) is provided so that you can gain a better understanding of
what your participation in this study would involve, and thus enable you to
make an informed decision about whether or not you would like to take
part. If, however, you need further information to make a decision, or
simply wish to speak to someone about this research, contact details are
provided and we are happy to answer your queries.
We sincerely hope that you will be able to help with this research.
Yours faithfully,
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 267
Professor Frances Mair
interview gp info
Mair Page 180 21/01/2009
Understanding the Implementation and Integration of E-Health Services;
Information Sheet for Interview Participants Workpackage 2.
Thank you for taking the time to consider participating in this research
project which has been funded by the NHS Service Delivery and
Organisation R&D Programme. Before you decide it is important for you to
understand why the research is being done and what it will involve. Please
take time to read the following information carefully. Talk to others about
the study if you wish.
This information sheet explains the purpose of the study, what will happen
to you if you take part, and gives more detailed information about the
conduct of the study.
Ask us if there is anything that is not clear or if you would like more
information you can discuss this with our researcher. Take time to decide
whether or not you wish to take part.
What is the purpose of the study?
The study examines those factors that act as facilitators and barriers to the
use of e-Health systems by health care professionals. E-Health covers a
range of types of service. In this project we will be looking at four specific
aspects of e-Health that are particularly important to the NHS:
Management systems (this includes electronic medical record systems);
Communication systems (includes real time or store and forward
communication systems such as e-mail, telecare and telemedicine systems)
Decision support systems.
Information systems (including use of the internet).
In this study we are looking particularly at: uptake of different types of e-
Health services; roots of resistance (if any); views about the effects of e-
Health services on the patient/provider relationship; and training and
support needs.
Why have I been chosen?
We are interested in hearing about the views of health care personnel who
have expertise or particular experience of some aspect of e-Health services
as described above who are well placed to act as key informants. You have
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 268
been chosen because of your leadership role and/or experience/expertise in
the use of e-Health services.
What do I have to do?
If you are interested in taking part or would like to speak to someone about
the study you should complete the ‘consent to contact’ form included with
this information sheet and return it with your completed questionnaire.
Within a few weeks you will be contacted by Prof Mair or Dr Anderson, one
of the researchers, working on the project.
If you decide to take part they will then arrange an appointment to discuss
study participation, via telephone, at a time that is convenient to you and at
that point a consent form will be mailed to you. This is to show that you
understand what the study is about, that you are happy to participate, and
that you are happy for us to record the interview.
We will give you a copy of this information sheet and the consent form to
keep.. Study participation will involve participating in a single telephone
interview. The interview will last approximately 45 minutes and will take
place via telephone or the researcher can come to your place of work if that
is preferable. Prior to the interview you will be will be sent summary
information relating to a literature review and five task groups that have
been undertaken with health care professionals about barrier/facilitators to
e-Health service utilisation. During the interview you will be asked to
comment on the literature review and the task group findings and provide
your views regarding:
Barriers to using e-Health systems in practice.
Ways of helping health professionals to integrate use of e-Health systems
within their routine work.
Types of support necessary to help with successful introduction of e-Health
services.
Potential effects, if any, of the different types of e-Health systems
(electronic records, telecare, decision support, and the internet) on the
health professional/patient relationship.
When you arrive for the focus group, the researcher will ask you to fill in a
consent form.
What will happen during the interview?
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 269
The researcher will ask a number of fairly open questions regarding the
findings of the research to date. The aim is to get your views on the study
findings relating to the use of different types of e-Health services by health
professionals as outlined above. The interview will last about 45 minutes
and will be tape recorded.
Why is the interview recorded?
We record the interviews because it is hard to take notes of what people
say, listen carefully and think all at the same time! After the interview the
recording will be listened to carefully and every word that participants and
the researcher say will be typed down. We use this written record, to help
us remember what people said. Once the study has concluded, all the
original audio recordings will be destroyed.
Are there any benefits to helping with this work?
Although there may be no direct benefits to you personally, we hope that
you find participation an interesting experience. The subject matter is
relevant to your work as across the United Kingdom the NHS is committed
to the adoption of new e-Health Services such as the use of electronic
medical records and home telecare technologies. Your involvement will give
us a better of understanding of the problems faced by health professionals
when using new e-Health services and how difficulties, if any, can best be
overcome. We believe this will be of use to health care professionals and
the wider NHS, providing a better understanding of the needs of health
professionals whenever new e-Health services are being introduced and how
best to support the introduction of such services.
Are there any disadvantages to helping with this research?
The main disadvantage is the time it will take; the interview usually lasts
about 45 minutes and there will be some documentation to read prior to the
interview.
Will my taking part in the study be kept confidential?
Yes. Everything you tell us during the interview is completely confidential.
All personal information – your name, names of colleagues or patients - or
anything else which might identify you will be removed so that no-one can
identify you personally. Only anonymised data from the interviews will be
used in future publications from the research team so that individual
participants cannot be identified. All recordings and written records are
treated as confidential material. They are stored securely within the
University under the supervision of Prof Frances Mair and will not be played
or shown to anyone outside the research team. .
Who is organising and funding the research?
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 270
The research if funded by the NHS Service Delivery and Organisation R&D
Programme and is being organised by the University of Glasgow (Division of
Community Based Sciences).
Who has reviewed the study?
This study was given a favourable review by the NHS Service Delivery and
Organisation R&D Programme and has had full ethical approval.
How can I get more information about the study?
If you want to know about what the interview will involve or about the study
in general or you have any concerns about any aspect of the study, please
get in touch with us. Our telephone number is: 0141 330 8317, please ask
for Professor Frances Mair or Dr George Anderson. Our address is:
Department of General Practice & Primary Care, University of Glasgow, 1
Horselethill Road, Glasgow G12 9LX
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 271
Appendix 8 WP2 Summary of Key Findings
A Summary of Our Key Findings.
The main factors to consider, and how they may impede or facilitate the implementation of e-Health systems, as identified in our review are as follows:
Note that these are not shown in a hierarchy of importance.
Conditions prior to implementation.
The literature shows that conditions prior to implementation are regarded as integral to the selection of a site for an e-Health intervention. There are essentially two levels on which the conditions prior to the implementation of an e-Health system will influence its success or failure: these are the conditions within the organisation itself (be it a health trust or a GP surgery), and the broad societal conditions (how strong is the support of national government in the general implementation of e-Health systems
Conditions prior to implementation.
Cost. Education and training.
The need for and impact of validation and evaluation. Security,
confidentiality and standards.
Professional attitudes.
Ease of use of the system.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 272
etc). Thus, the impact of pre-existing conditions, be they internal or external to the setting, cannot be under-estimated.
Cost.
The cost of a system is an inevitable factor to consider in the implementation of an e-Health system. It can be assumed that the term “cost” includes both the financial cost of initial hardware, software and its implementation and cost in terms of time and human resources, though the literature itself does little to differentiate between these types. In terms of telecommunications, there is also the cost of regular usage to consider as well as that of initial implementation, since network tariffs may apply; in terms of other systems there will also be costs associated with procurement and maintenance.
The need for and impact of validation and evaluation.
Validation and evaluation of e-Health systems are important factors in their widespread implementation. Without strong data demonstrating that a system works, improves standards of care, can be used efficiently and easily, and is cost-effective to implement, it is unlikely to win the confidence of policy makers and users. Indeed, lack of validation and evaluation is frequently presented as a barrier to system implementation in the literature, while continued monitoring and evaluation is a facilitator.
. Professional attitudes.
Professional attitudes to the implementation and integration of e-Health systems contain within them a broad array of barriers and facilitators to success. These attitudes may relate to perceptions of the changing relationship between professionals and patients and perceptions as to the effectiveness of the technology. It may be that there exists a general anxiety towards the use of technology, or even that clinicians feel that their roles are undermined by it and that the introduction of new systems implies shortcomings in their own performance. The latter can relate how the e-Health system affects the allocation of tasks and how an organisational structure may change once technology is adopted. It can also relate to unease regarding patient safety, particularly in terms of how professionals can manage a critical situation at a distance.
Ease of use of the system.
The ease of use of an e-Health system, the interaction between the technology and its users, is self-evidently a factor in successful implementation. The system needs to be appropriate to the work and not imposed for its own sake, be compatible with existing systems, and reliable.
Related to the general ease of use of a system is its ability to order and manage information. If the technology is disruptive to established working practices and increases the amount of time and effort required to record, communicate, and find information and indeed arrive at a clinical decision,
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 273
its successful implementation is unlikely. A management or information system may contain insufficient, misleading, or too much information.
Security, confidentiality, and standards.
For successful implementation, a system needs to be not only workable, but also secure. It needs to be safe for both patients and professionals, for patients, in terms of the security of their personal data, and for professionals in terms of not compromising their standards of practice.
Education and training.
The success or failure of a new system will depend on the capabilities of those using it; and new technologies, though they may strive to ultimately simplify the work, must be learnt in the first instance. Lack of skills and knowledge is frequently cited as a barrier to success. Education can be as simple as the communication of information between professionals and organisations for the purpose of influencing system implementation and usage to more formalised practices, such as officially sanctioned system-specific training courses.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 274
Appendix 9 WP2 Task Group Schedule
Initial information gathering:
(Each group member in turn) Who are you and what is your role (this is also for the transcriber)?
What e-Health systems have you used or had experience of?
-What type of system (link to 4 domains)? (2 minutes per participant.)
Introduction:
The main factors to consider, and how they may impede or facilitate the implementation of e-Health systems, as identified by our review are as follows:
Note that these are not shown in a hierarchy of importance.
Central Question(s):
To Start off:
Thinking of a specific system in which you have had involvement, which of these factors, or which factor, would you consider the most important, if any?
(Or which would you most like to talk to about? Which seems most relevant to your experience and the type of system you have been involved with?)
Conditions prior to
implementation.
Cost. Education and training.
The need for and impact of validation and evaluation.
Security, confidentiality and
standards.
Professional attitudes.
Ease of use of the system.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 275
Following on from this, respondents will be asked about each factor specifically; whether they have personal experience of each factor (can they report on real-life instances of such?) and whether they agree with the findings of the review that the factor is an important barrier/facilitator. (1 hour +)
Respondents will be asked for two things: firstly, if they deem anything important to be missing from our findings; secondly, if they deem any of our findings insignificant. (Last half hour.)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 276
Appendix 10 WP2 Interview Schedule
Initial information gathering:
What is your role?
What e-Health systems have you implemented or had experience of?
-What type of system (link to 4 domains)? (5 minutes.)
Introduction:
The main factors to consider, and how they may impede or facilitate the implementation of e-Health systems, as identified by our review are as follows:
Note that these are not shown in a hierarchy of importance.
Central Question(s):
To Start off:
Thinking of a specific system in which you have had involvement, which of these factors, or which factor, would you consider the most important, if any?
(Or which would you most like to talk to about? Which seems most relevant to your experience and the type of system you have been involved with?)
Conditions prior to
implementation.
Cost. Education and training.
The need for and impact of validation and evaluation.
Security, confidentiality and
standards.
Professional attitudes.
Ease of use of the system.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 277
Following on from this, respondents will be asked about each factor specifically; whether they have knowledge or personal experience of each factor (can they report on real-life instances of such?) and whether they agree with the findings of the review that the factor is an important barrier/facilitator.
Respondents will be asked for two things: firstly, if they deem anything important to be missing from our findings; secondly, if they deem any of our findings insignificant. (25 minutes.)
Finally, respondents will be asked to describe how to integrate a system
successfully, and whether the same factors are of equal importance across
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 278
Appendix 11 WP3 Sample characteristics of expert survey participants
Sample characteristics of expert survey participants
Location of Residence %
USA 37
UK 27
Canada 13
Europe (excluding Scandinavia) 10
Australia/New Zealand 8
Scandinavia 6
Research background
Medical 32
Social science 24
Informatics 21
Nursing 11
Economics 2
Health Services Research 5
Non-specific 6
Sex
Male 59
Female 41
E-health domain Mostly (%) Partly (%) Not at all (%)
Management Systems 29 46 25
Communication Systems 44 32 24
Computerised decision support systems 14 38 48
Web based Information Resources 22 29 49
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 279
279
Appendix 12 WP3 Expert sample: Factor rating scores: frequencies, means and standard deviation
Expert sample: Factor rating scores: frequencies, means and standard deviation
Score* frequencies % Q.
N Mean score
(SD) 0 1 2 3 4 dk
Q3 Impact of the system on existing ways of working (CI) 63 3.49 (.759) 0 1.6 11.1 23.8 63.5 0
Q27 Ease of using the system (IW) 63 3.44 (.616) 0 0 6.3 42.9 50.8 0
Q2 Allocation of organizational effort to the system (CI) 62 3.32 (.845) 0 4.8 9.5 33.3 50.8 1.6
Q8 Additional workload created by the system (SW) 61 3.20 (.771) 0 1.6 15.9 41.3 38.1 3.2
Q4 Balance of effort against rewards of using the system (CI) 63 3.08 (.848) 0 4.8 17.5 42.9 34.9 0
Q23 Perceived impact of the system on ways of working with patients (IW) 62 2.90 (.970) 1.6 4.8 27.0 33.3 31.7 1.6
Q1 Allocation of financial resources to the system (CI) 61 2.90 (.87) 0 3.2 31.7 33.3 28.6 3.2
Q6 level of co-operation required from others within the organisation, in using the
system (SW)
63 2.89 (.918) 0 9.5 19 44.4 27.0 0
Q26 Perceived impact of the system on the amount of time spent with patients
(IW)
63 2.87 (1.008) 1.6 9.5 19 39.7 30.2 0
Q19 Availability of technical expertise in using the system (RI) 63 2.84 (1.003) 0 11.1 25.4 31.7 31.7 0
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 280
280
Q11 Obtainability of new skills required to use the system (SW) 63 2.79 (.986) 0 14.3 17.5 42.9 25.4 0
Q22 How flexibly the system can be used for conducting work (IW) 62 2.79 (.908) 1.6 7.9 19.0 50.8 19.0 1.6
Q24 Perceived impact of the system on outcomes for patients (IW) 60 2.72 (1.075) 3.2 7.9 28.6 28.6 27.0 4.8
Q15 Individuals’ perceptions of the efficiency of using the system (RI) 63 2.67 (.916) 0 12.7 25.4 44.4 17.5 0
Q10 Compatibility of the system with existing skills (SW) 63 2.65 (.786) 0 4.8 39.7 41.3 14.3 0
Q5 Impact of the system on individual’s perceptions of autonomy in their work
(SW)
63 2.62 (.991) 0 14.3 31.7 31.7 22.2 0
Q25 Perceived impact of the system on communication with patients (IW) 62 2.52 (1.083) 3.2 19.0 15.9 44.4 15.9 1.6
Q9 Impact of the system on allocation of work between individuals (SW) 60 2.47 (.965) 3.2 9.5 34.9 34.9 12.7 4.8
Q13 Individuals’ own confidence in the safety of using the system (RI) 62 2.37 (.945) 1.6 14.3 41.3 28.6 12.7 1.6
Q21 Availability of users’ knowledge of the clinical effectiveness of the system
(RI)
61 2.34 (1.109) 4.8 19.0 25.4 33.3 14.3 3.2
Q16 Impact of the system on the distribution of responsibilities between
individuals (RI)
59 2.34 (.958) 3.2 12.7 36.5 31.7 9.5 6.3
Q20 Availability of an evidence base about the clinical effectiveness of the system
(RI)
62 2.31 (1.249) 9.5 17.5 23.8 28.6 19.0 1.6
Q7 level of co-operation required from others outside the organisation, in using
the system (SW)
62 2.27 (.890) 0 20.6 38.1 31.7 7.9 1.6
Q17 Impact of the system on individuals’ beliefs about their accountability for their
work (RI)
62 2.26 (1.023) 6.3 12.7 38.1 31.7 9.5 1.6
Q12 Impact of the system on individuals’ perceptions of personal liability (RI) 59 2.25 (.993) 0 25.4 30.2 27 11.1 6.3
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 281
281
Q18 Impact of the system on individuals’ beliefs about others’ expectations of
their accountability for their work (RI)
60 2.08 (.962) 6.3 15.9 41.3 27 4.8 4.8
Q14 Individuals’ confidence in the safety of others’ use of the system (RI) 60 2.02 (1.033) 6.3 22.2 38.1 20.6 7.9 4.8
*Scale: 0=not at all important; 1=some importance; 2=moderate importance; 3=very important; 4=extremely important
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 282
282
Appendix 13 WP3 Phase 2: QUESTION ANALYSIS BY NPM CONSTRUCT
Phase 2: QUESTION ANALYSIS BY NPM CONSTRUCT
INTERACTIONAL WORKABILITY SKILL-SET WORKABILITY
Existing questions:
1. How flexibly the system can be used for conducting work
2. Perceived impact of the system on ways of working with
patients
3. Perceived impact of the system on outcomes for patients
4. Perceived impact of the system on the amount of time
spent with patients
5. Ease of using the system
‘Additional’ factors mentioned but covered:
possibility to modify technical system (1)
Flexibility of use of system/open access interface (x2) (q.1)
Preferences regarding patient interaction (2)
access to (fast) internet in work setting (5)
User friendliness (5)
Requirement to use multiple tools for one task (5)
Perceived convenience/inconvenience (5)
Existing questions:
1. Impact of the system on individual’s perceptions of autonomy in
their work
2. Level of co-operation required by others in using the system
3. Additional workload created by the system
4. Impact of the system on allocation of work between individuals
5. Compatibility of the system with existing skills
6. Obtainability of new skills required to use the system
‘Additional’ factors mentioned but covered:
Loss of autonomy of practice (1)
Integration of IT in clinical workflow (x2) (3)
user abilities (re educational patient materials; ability to rate info
quality) (5)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 283
283
Mentioned and not covered:
technical performance
RELATIONAL INTEGRATION CONTEXTUAL INTEGRATION
Existing questions:
1. Individuals’ own confidence in the safety of using the
system
2. Individuals’ perceptions of the efficiency of using the
system
3. Impact of the system on the distribution of responsibilities
between individuals
4. Impact of the system on individuals’ beliefs about their
accountability for their work
5. Availability of technical expertise in using the system
6. Availability of evidence about the clinical effectiveness of
the system
‘Additional’ factors mentioned but covered:
Perceived effect on patients’ responsibilities/roles (3)
operation of a back-up system (q.1?)
Mentioned and not covered:
Quality/evidence base of information within the system (3
mentions)
Use is expected as normal practice (maybe q.4?)
Existing questions:
1. Allocation of financial resources to the system
2. Allocation of organizational effort to the system
3. Impact of the system on existing ways of working
4. Balance of effort against rewards of using the system
‘Additional’ factors mentioned but covered:
Incentives for use (1)
Balance of costs/benefits across different stakeholders (4)
Immediate perception of benefit (4)
Mentioned and not covered:
Characteristics of the organization itself (stability; commitment &
champions; leadership; complexity; readiness/fear of change)
Wider contexts outside the organization (policy – broad and e-
Health specific; legal context; compatibility with other
systems/agencies)
Organizational process (Adequate feasibility testing in practice;
commissioner autonomy; targets for use; ongoing technical
support; ‘best practice’ relating to patient-provider email
communications; multi-professional involvement in planning &
development (4 mentions; e-Health embedded in
CPD/education; responsibility for data management)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 284
284
Alignment of system with philosophy of care /health system
priorities and challenges (2 comments)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 285
285
Appendix 14 WP3 Analysis of rating items for Phase 2
Analysis of rating items for Phase 2
Considerations Decision Final Item
Q1
Allocation of financial resources to the system Ranked in top half of table. Correlates with q.2 (0.527) and
q. 18 (0.531)
retain Allocation of financial
resources to the system
Q2
Allocation of organizational effort to the system Third highest mean rating score. Correlates with q.1
(0.527).
retain Allocation of organizational
effort to the system
Q3
Impact of the system on existing ways of working
Ranked no 1 in importance. No r’s > 0.5.
retain Impact of the system on
existing ways of working
Q4
Balance of effort against rewards of using the
system Ranked 5th. Doesn’t correlate well with any other item
retain Balance of effort against
rewards of using the system
Q5
Impact of the system on individual’s perceptions of
autonomy in their work Mid-table in importance ratings. Correlates with q.9 (r
0.573).
retain Impact of the system on
individual’s perceptions of
autonomy in their work
Q6
level of co-operation required from others within the
organisation, in using the system Ranked 8th. Correlates with q.7 (0.560).
Q7
level of co-operation required from others outside
the organisation, in using the system
Correlates with q.6 (0.560), but most correlations near
zero. (ranked 5th from bottom)
Combine 6 and 7
Level of co-operation required
by others in using the system
Q8
Additional workload created by the system Ranked 4th in importance. No r’s above 0.5, but
approaching that on q. 26 and 27.
retain Additional workload created
by the system
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 286
286
Q9
Impact of the system on allocation of work between
individuals
Correlates with q.5 (r 0.573).
retain Impact of the system on
allocation of work between
individuals
Q10
Compatibility of the system with existing skills
Ranked mid-table. Correlates with q.11 (0.519)
retain
Compatibility of the system
with existing skills
Q11
Obtainability of new skills required to use the
system
Ranked 11th. Correlates with q.10 (0.519). Several
significant (but low) correlations with other items.
retain Obtainability of new skills
required to use the system
Q12
Impact of the system on individuals’ perceptions of
personal liability
Ranked 3rd from bottom. Correlates with q. 17 (r .564) &
18 (r .569). Correlations < but approaching 0.5 for q. 13 &
14.
exclude
Q13
Individuals’ own confidence in the safety of using
the system
Ranked mid-table. High r (0.725) with q. 14. Correlates
with q. 18 (0.565). Approaches 0.5 with q.12.
Q14
Individuals’ confidence in the safety of others’ use of
the system
Ranked least important. High r (0.725) with q. 13, and
correlates with q.18 (0.531). Approaches 0.5 with q.12.
Combine 13 and
14
Individuals’ own confidence in
the safety of using the system
Q15
Individuals’ perceptions of the efficiency of using the
system Ranked mid-table. No correlations > 0.5.
Retain Individuals’ perceptions of the
efficiency of using the system
Q16
Impact of the system on the distribution of
responsibilities between individuals
Ranked in bottom half. No correlations > 0.5.
Retain Impact of the system on the
distribution of responsibilities
between individuals
Q17
Impact of the system on individuals’ beliefs about
their accountability for their work Ranked near bottom. High r with Q.18 (0.806). Correlates
with q. 12 (r .564)
retain Impact of the system on
individuals’ beliefs about their
accountability for their work
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 287
287
Q18
Impact of the system on individuals’ beliefs about
others’ expectations of their accountability for
their work
Ranked second bottom. High r with Q.17 (0.806).
Correlates with q. 12 (r .569), 13 (0.565) and q.14
(0.531).
Exclude question
Q19
Availability of technical expertise in using the
system
Ranked in top half. Correlates with q.21 (0.557) & 25
(0.581).
retain Availability of technical
expertise in using the system
Q20
Availability of an evidence base about the clinical
effectiveness of the system
Ranked in bottom half. High r with Q.21 (0.721). Also r
0.619 with Q.24.
Q21
Availability of users’ knowledge of the clinical
effectiveness of the system Ranked in bottom half. High r with Q.20 (0.721). Correlates
with q.19 (0.557), q. 24 (0.517) & q.25 (0.514).
Combine 20 and
21
Availability of evidence about
the clinical effectiveness of the
system
Q22
How flexibly the system can be used for conducting
work Ranked in top half. Correlates with q.23 (0.533).
retain How flexibly the system can
be used for conducting work
Q23
Perceived impact of the system on ways of
working with patients Ranked 6th in importance. Correlates with Q.22 (0.533). &
q.25 (0.586).
Retain Perceived impact of the
system on ways of working
with patients
Q24
Perceived impact of the system on outcomes for
patients Ranked mid-table. Correlates with Q.20 (0.619) & q.21
(0.517).
retain Perceived impact of the
system on outcomes for
patients
Q25
Perceived impact of the system on communication
with patients
Ranked mid-table. Correlates with q.19 (0.581), q. 21
(0.514) & q.23 (0.586)
Exclude (covered
in q 23)
Q26
Perceived impact of the system on the amount of
time spent with patients
Ranked in top half. Approaches 0.50 with q.8. & q.25.
retain Perceived impact of the
system on the amount of time
spent with patients
Q27 Ease of using the system Ranked second highest in importance. Doesn’t correlate retain Ease of using the system
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 288
288
>.05 with any item.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 289
289
Appendix 15 WP3 Final set of TARS items mapped against WP2 data themes
WP3 Final set of TARS items mapped against WP2 data themes
Tars Generic Tars specific WP2 Themes
1. Allocation of financial resources to the system 1. The e-Health system is adequately resourced
financially
TG7: Cost (KI:11 Cost)
2. Allocation of organizational effort to the system 2. Sufficient organizational effort has gone into
supporting the e-Health system
TG9: Organisational drive and
leadership
3. Impact of the system on existing ways of
working
3 The e-Health system is a different way of
working
(KI10 Broader societal/cultural change?)
4. Balance of effort against rewards of using the
system
4. The rewards of using the e-Health system
outweighs the effort
5. The supportiveness or otherwise of the broader
policy context in relation to e-Health
5. Government policy initiatives are supportive of
this e-Health system
KI10 Broader societal/cultural change
6. The technical and organisational compatibility
of the e-Health system with other systems and
agencies that it must relate to for the conduct
of work
6. This e-Health system is technically and
organisationally compatible with other systems
and agencies that we are required to work with
TG3: Collaborative working (KI8:
collaborative/interagency working)
KI6: Relationship with suppliers &
designers
7. How well the e-Health system fits in with
priorities and challenges of the organisation
7. This e-Health system fits in with the priorities
and challenges of our organisation
TG2: Clear purpose
KI5: Clear rationale
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 290
290
8. An organisational culture that is supportive of
change
8. This organisation has a culture that is
supportive of change
TG9: Organisational drive and
leadership
9. An organisational culture of involving staff in
planning and development
9. There is a culture in this organisation of
involving staff in planning and development
TG5: Sense of user empowerment
10. The level of autonomy that the organisation has
in terms of commissioning services
[Item omitted due to irrelevance to study sites]
11. Impact of the system on individual’s
perceptions of autonomy in their work
10. Using the e-Health system makes me feel
autonomous in my work
12. Level of co-operation required by others in
using the system
11. Using the e-Health system requires co-
operation with other staff
TG3: Collaborative working (KI8)
13. Additional workload created by the system 12. The workload involved in using the e-Health
system is manageable
TG6: Ease of use (KI3)
14. Impact of the system on allocation of work
between individuals
13. In using the e-Health system, the allocation of
work between individuals is appropriate
TG3: Collaborative working (?)
15. Compatibility of the system with existing skills 14. The skills I have are appropriate for using the e-
Health system
TG6: Ease of use
KI3: Ease of use
16. Learnability of new skills required to use the
system
15. The skills needed to use the e-Health system
are easily learned
TG6: Ease of use (simplicity & ease to
learn) (K3)
17. Individuals’ own confidence that using the
system does not put patients at risk
16. I have confidence that using the e-Health
system does not put patients at risk
TG4: Standards (clinical risk)
TG1: Patient benefits
KI2: Standards
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 291
291
18. Individuals’ perceptions of the efficiency of
using the system
17. Using the e-Health system is an efficient use of
time
TG6: Ease of use (efficiency)
19. Impact of the system on the distribution of
responsibilities between individuals
18. In using the e-Health system, responsibilities
are divided between individuals appropriately
?
20. Impact of the system on individuals’ beliefs
about their accountability for their work
19. In using the e-Health system, I understand my
accountability for my work
TG4: Standards (?)
21. Impact of the system on individuals’
perceptions of liability
20. In using the e-Health system, I understand my
liability for my practice
TG4: Standards (legality)
22. Availability of technical back-up in using the
system
21. Technical back-up in using the e-Health system
is available if I need it
TG6: Ease of use (tech support)
K9: Technical support
KI6: Relationship with suppliers &
designers
23. Availability of evidence about the clinical
effectiveness of the system
22. I believe there is good evidence about the
clinical effectiveness of using the e-Health
system
TG4: Standards
KI7: Generating new knowledge
KI5: Clear rationale (evidence)
24. How flexibly the system can be used 23. There is some flexibility in how the e-Health
system can be used
TG6: Ease of use
KI3: Ease of use
25. Perceived impact of the system on outcomes
for patients
24. Using the e-Health system leads to positive
outcomes for patients
TG1: Patient benefits
26. Perceived impact of the system on the amount
of time spent with patients
25. Using the e-Health system involves the right
amount of time spent with patients
TG1: Patient benefits
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 292
292
27. Perceived impact of the system on the quality
of the interaction between professionals and
patients
26. In using the e-Health system, the quality of
professional and patient interaction is good
TG1: Patient benefits
28. Ease of using the system 27. The e-Health system is easy to use TG6: Ease of use; KI3: Ease of use
29. The existence of a shared understanding of
what the system is for and how it is to be used
28. The staff who work here have a shared
understanding of what the system is for and
how it is to be used
TG2: Clear purpose
TG8 & KI1: Professional attitudes (unity
of purpose; perceived usefulness)
30. The commitment of individuals to making the
system work
29. The staff here are committed to making the
system work
TG8: Professional attitudes (willingness)
31. The existence of ongoing mechanisms for
monitoring and appraising how the system is
used
30. There are ongoing mechanisms for monitoring
and appraising how this e-Health system is used
TG4: Standards
TG6 Ease of Use
KI7: Generating new knowledge
32. The balance between the needs of individual
users, other users of the system, and the
organization itself [suggested addition by site
contact]
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 293
Appendix 16 WP3 TARS GENERIC
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 294
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 295
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 296
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 297
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 298
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 299
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 300
300
Appendix 17 WP3 Tars Generic: Means, standard deviations and frequencies
WP3 Tars Generic: Means, standard deviations and frequencies
Frequencies (% are approx double)
N* Means Sds
1
2
3
4
5
8 DK
Total
ans
3 Impact of the system on existing ways of working 49 4.57 .677 5 11 33 1 50
9 An organizational culture of involving staff in planning
and development
47 4.57 .677
2 14 31 2 49
15 Compatibility of the system with existing skills 46 4.57 .677 1 11 13 21 3 49
21 Impact of the system on individuals' perceptions of
liability
45 4.57 .677
1 3 4 14 23 3 48
27 Perceived impact of the system on the quality of the
interaction between professionals and patients
45 4.57 .677
6 15 24 3 48
6 The integration of the e-Health system with other
systems and agencies that it must relate to for the
conduct of work
47
4.53 .620
3 16 28 4 51
12 Level of co-operation required by others in using the
system
47 4.53 .620
1 5 15 26 3 50
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 301
301
18 Individuals' perceptions of the efficiency of using the
system
48 4.53 .620
1 5 19 23 2 50
24 How flexibly the system can be used 46 4.53 .620 3 13 30 2 48
30 The commitment of individuals to making the system
work
46 4.53 .620
5 17 24 2 48
2 Allocation of organizational effort to the system 48 4.50 .684 5 14 29 3 51
8 An organizational culture that is supportive of change 48 4.50 .684 3 12 33 2 50
14 Impact of the system on allocation of work between
individuals
44 4.50 .684
1 2 9 14 18 6 50
20 Impact of the system on individuals' beliefs about their
accountability for their work
47 4.50 .684
2 5 16 24 2 49
26 Perceived impact of the system on the amount of time
spent with patients
43 4.50 .684
1 5 16 21 5 48
32 The balance between the needs of individual users,
other users of the system, and the organisation itself
47 4.50 .684
9 14 24 1 48
4 Balance of effort against rewards of using the system 49 4.37 .755 8 15 26 2 51
10 The level of autonomy that the organization has in terms
of commissioning services
45 4.37 .755
8 19 18 5 50
16 Learnabiliity of new skills required to use the system 48 4.37 .755 1 4 17 26 2 50
22 Availability of technical support in using the system 47 4.37 .755 3 11 33 1 48
28 Ease of using the system 47 4.37 .755 1 1 14 31 1 48
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 302
302
5 The supportiveness or otherwise of the broader context
in relation to e-Health
39 4.18 .885
1 9 11 18 11 50
11 Impact of the system on individual's perceptions of
autonomy in their work
44 4.18 .885
1 7 15 21 6 50
17 Individual's own confidence that using the system does
not put patients at risk
47 4.18 .885
1 3 7 36 2 49
23 Availability of evidence about the clinical effectiveness of
the system
47 4.18 .885
1 8 10 28 1 48
29 The existence of a shared understanding of what the
system is for and how it is to be used
45 4.18 .885
7 18 20 2 47
1 Allocation of financial resources to the system 48 4.10 .951 2 7 21 18 3 51
7 How well the e-Health system fits in with priorities and
challenges of the organization
48 4.10 .951
1 5 15 27 3 51
13 Additional workload created by the system 48 4.10 .951 2 2 10 10 24 2 50
19 Impact of the system on the distribution of
responsibilities between individuals
45 4.10 .951
2 15 16 12 5 50
25 Perceived impact of the system on outcomes for patients 45 4.10 .951 1 3 15 26 2 47
31 The existence of ongoing mechanisms for monitoring
and appraising how the system is used
45 4.10 .951
1 5 15 24 3 48
SDO Project (08/1602/135)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 304
Appendix 18 WP3 TARS SPECIFIC (Site 1)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 305
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 306
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 307
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 308
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 309
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 310
Appendix 19 WP3 TARS SPECIFIC (Site 2)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 311
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 312
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 313
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 314
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 315
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 316
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 317
317
Appendix 20 Sample characteristics for TARS Specific (Site 1 and Site 2)
Site 1
% (n)
Site 2
% (n)
Age groups: Age groups:
<25 0 (0) <25 9 (20)
25-34 4 (2) 25-34 20 (47)
35-44 24 (11) 35-44 32 (73)
45-54 59 (27) 45-54 33 (75)
55+ 13 (6) 55+ 7 (15)
Sex Sex
Male 0 (0) Male 14 (32)
Female 100 (46) Female 86 (199)
Working role: Working role:
Community Enrolled Nurse 0 (0) Call handlers 47 (109)
Community Staff Nurse 28 (13) Nurse advisors 24 (56)
District Nursing Sister/Charge Nurse 61 (28) Team leaders 9 (21)
Practice Development Nurse 9 (4) Health information advisors 3 (7)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 318
318
Senior Nurse 2 (1) Other 16 (38)
Time working in role: Time working in role:
<2 years 7 (3) < 1 year 15 (36)
2 to < 5 years 22 (10) 1 year to 23 months 10 (23)
5 to <10 years 28 (13) 2 years to 47 months 20 (45)
10 years plus 30 (14) 4 years to 71 months 16 (36)
Did not specify 13 (6) 6 years + 16 (37)
Did not specify 23 (54)
Time using e-Health system
(Time using not assessed for Site 2)
no months of use 9 (4)
some but <3 mths 20 (9)
4 or 5 mths 9 (4)
6 mths but <12 20 (9)
1 yr but <2 yrs 22 (10)
2 years + 22 (10)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 319
319
Perceived level of routinisation of e-Health Perceived level of routinisation of e-Health
Not at all 0 (0) Not at all 1 (2)
Partly 68 (30) Partly 17 (35)
Completely 32 (14) Completely 83 (174)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 320
320
Appendix 21 WP3 TARS Specific Site 1: TARS items, means, standard deviations and frequencies
WP3 TARS Specific Site 1: TARS items, means, standard deviations and frequencies
Frequencies (not percentages)
N*
Mea
ns Sds
0
1
2
3
4
5
6
DK
N
19 In using the e-Health system, I understand my
accountability for my work 43 4.70 1.15 0 0 2 5 9 15 12 2 45
20 In using the e-Health system, I understand my liability
for my practice 43 4.65 1.11 0 0 1 6 12 12 12 1 44
21 Technical back-up in using the e-Health system is
available if I need it 43 4.21 1.71 1 2 6 5 6 10 13 2 45
14 The skills I have are appropriate for using the e-Health
system 44 4.14 1.37 3 1 9 13 10 8 1 45
11 Using the e-Health system requires co-operation with
other staff 44 3.98 1.99 10 1 6 7 3 17 2 46
5 Government policy initiatives are supportive of this e-
Health system 36 3.97 1.28 1 1 11 12 6 5 8 44
30 There are ongoing mechanisms for monitoring and 29 3.97 1.64 2 2 6 8 5 6 16 45
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 321
321
appraising how this e-Health system is used
3 The e-Health system is a different way of working
compared with other parts of the NHS 38 3.87 1.21 1 1 14 10 9 3 8 46
16 I have confidence that using the e-Health system does
not put patients at risk 41 3.85 1.71 1 5 4 3 11 10 7 3 44
29 The staff here are committed to making the system
work 38 3.82 1.23 2 2 10 15 5 4 5 43
15 The skills needed to use the e-Health system are easily
learned 43 3.79 1.47 1 3 4 8 10 14 3 2 45
8 This organization has a culture that is supportive of
change 43 3.77 1.38 2 5 13 10 7 6
0 43
2 Sufficient organizational effort has gone into supporting
the e-Health system 42 3.64 1.69 2 4 2 13 5 10 6 4 46
13 In using the e-Health system, the allocation of work
between individuals is appropriate 35 3.51 1.27 2 4 13 9 4 3 9 44
28 The staff who work here have a shared understanding of
what the system is for and how it is to be used 41 3.51 1.33 1 1 6 13 11 6 3 3 44
18 In using the e-Health system, responsibilities are divided
between individuals appropriately 37 3.41 1.36 1 3 4 10 11 7 1 6 43
27 The e-Health system is easy to use 41 3.39 1.74 3 4 4 10 8 7 5 2 43
22 I believe there is good evidence about the clinical
effectiveness of using the e-Health system 43 3.37 1.22 1 2 3 21 7 8 1 0 43
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 322
322
23 There is some flexibility in how the e-Health system can
be used 36 3.36 1.42 1 3 5 9 11 5 2 9 45
4 The rewards of using the e-Health system outweighs the
effort 37 3.22 1.65 2 3 8 9 6 5 4 8 45
17 Using the e-Health system is an efficient use of time 42 3.17 1.77 5 3 3 15 5 7 4 1 43
12 The workload involved in using the e-Health system is
manageable 43 3.16 1.60 3 2 11 9 8 7 3 3 46
1 The e-Health system is adequately resourced financially 33 3.15 1.58 2 4 2 13 6 3 3 13 46
7 This e-Health system fits in with the priorities and
challenges of our organization 41 3.15 1.70 4 3 5 13 7 5 4 3 44
24 Using the e-Health system leads to positive outcomes
for patients 40 3.10 1.39 2 4 4 15 9 5 1 5 45
9 There is a culture in this organization of involving staff in
planning and development 44 2.93 1.73 4 6 7 11 9 2 5 0 44
26 In using the e-Health system, the quality of professional
and patient interaction is good 30 2.87 1.46 3 2 4 12 6 2 1 14 44
10 Using the e-Health system makes me feel autonomous
in my work 40 2.83 1.63 4 6 1 20 4 1 4 2 42
25 Using the e-Health system involves the right amount of
time spent with patients 31 2.61 1.36 3 3 8 7 9 1 0 12 43
6 This e-Health system is technically and organizationally
compatible with other systems and agencies that we are
required to work with 39 1.33 1.34 15 8 7 6 3 0 0 6 45
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 323
323
Appendix 22 WP3 TARS Specific Site 2: TARS items, means, standard deviations and frequencies
WP3 TARS Specific Site 2: TARS items, means, standard deviations and frequencies
Frequencies
N*
Mea
ns Sds
0
1
2
3
4
5
6
DK
N
1 The e-Health system is adequately resourced
financially 120 3.84 1.32 2 3 11 34 26 34 10 111 231
2 Sufficient organizational effort has gone into
supporting the e-Health system 182 3.9 1.36 3 6 16 42 49 46 20 49 231
3 The e-Health system is a different way of working
compared with other parts of the NHS 174 4.68 1.31 1 9 29 30 41 64 56 230
4 The rewards of using the e-Health system
outweighs the effort 181 4.15 1.23 2 12 46 49 41 31 44 225
5 Government policy initiatives are supportive of this
e-Health system 123 3.93 1.30 1 4 7 37 30 29 15 106 229
6 This e-Health system is technically and
organizationally compatible with other systems 176 3.35 1.53 10 10 26 50 36 31 13 53 229
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 324
324
and agencies that we are required to work with
7 This e-Health system fits in with the priorities and
challenges of our organization 207 4.07 1.27 7 11 52 60 45 32 21 228
8 This organization has a culture that is supportive
of change 212 4.01 1.49 7 7 11 47 59 41 40 10 222
9 There is a culture in this organization of involving
staff in planning and development 210 2.99 1.78 28 19 28 50 39 30 16 19 229
10 Using the e-Health system makes me feel
autonomous in my work 201 3.55 1.58 11 13 19 54 38 48 18 23 224
11 Using the e-Health system requires co-operation
with other staff 216 3.88 1.38 3 5 21 64 49 42 32 14 230
12 The workload involved in using the e-Health
system is manageable 215 4.08 1.29 2 4 14 52 57 54 32 15 230
13 In using the e-Health system, the allocation of
work between individuals is appropriate 188 3.87 1.36 3 4 19 52 44 43 23 42 230
14 The skills I have are appropriate for using the e-
Health system 219 4.73 1.14 1 6 30 43 74 65 11 230
15 The skills needed to use the e-Health system are
easily learned 221 4.03 1.29 1 6 17 52 60 55 30 9 230
16 I have confidence that using the e-Health system
does not put patients at risk 213 4.22 1.33 2 4 16 39 53 60 39 15 228
17 Using the e-Health system is an efficient use of
time 218 4.21 1.28 1 3 19 42 49 70 34 13 231
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 325
325
18 In using the e-Health system, responsibilities are
divided between individuals appropriately 192 3.77 1.45 8 6 12 56 44 46 20 35 227
19 In using the e-Health system, I understand my
accountability for my work 214 4.84 1.14 1 6 27 32 74 74 8 222
20 In using the e-Health system, I understand my
liability for my practice 204 4.76 1.28 2 2 5 27 34 61 73 19 223
21 Technical back-up in using the e-Health system is
available if I need it 203 3.59 1.50 5 14 27 50 45 41 21 26 229
22 I believe there is good evidence about the clinical
effectiveness of using the e-Health system 229 3.79 1.42 11 5 10 66 63 51 23 229
23 There is some flexibility in how the e-Health
system can be used 203 3.24 1.58 16 11 30 59 39 35 13 28 231
24 Using the e-Health system leads to positive
outcomes for patients 208 4.03 1.19 2 2 13 48 71 49 23 22 230
25 Using the e-Health system involves the right
amount of time spent with patients 201 3.52 1.44 9 8 26 51 53 42 12 28 229
26 In using the e-Health system, the quality of
professional and patient interaction is good 211 3.92 1.19 2 19 60 61 49 20 19 230
27 The e-Health system is easy to use 219 4.1 1.27 1 3 18 49 63 51 34 8 227
28 The staff who work here have a shared
understanding of what the system is for and how it
is to be used 215 4.12 1.27 2 3 17 41 66 54 32 11 226
29 The staff here are committed to making the 209 4.24 1.22 2 13 47 54 56 37 15 224
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 326
326
system work
30 There are ongoing mechanisms for monitoring and
appraising how this e-Health system is used 179 4.36 1.17 1 6 38 51 47 36 50 229
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 327
327
Appendix 23 WP3 Comparison of correlations between TARS Specific samples
WP3 Comparison of correlations between TARS Specific samples
Correlations: Site 1 Site 2
CI items ‘financial resource’ with ‘organizational effort’
‘culture supportive of change’ with both
‘supportive policy initiatives’ and ‘fit with
priorities of organization’
‘priorities’ with ‘reward outweighs effort’
financial resource with organizational effort
SW items Allocation of work with both manageable
workload and existing skills are appropriate
(but autonomy in work and cooperation with
others were low with other SW items)
‘workload is manageable’ with both
‘allocation of work’ & ‘skills I have are
appropriate’
RI items ‘Efficiency’ with ‘confidence in not risking
patients’
‘efficiency’ with both ‘confidence in not
risking patients’ & 'appropriate divisions of
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 328
328
‘appropriate divisions of responsibilities’ &
‘belief in evidence of effectiveness’
‘confidence in not risking patients’ with
‘availability of technical back-up’
‘accountability’ and ‘liability’ very high
responsibilities’
‘accountability’ and ‘liability’ very high
IW items ‘flexibility in use’ with both ‘positive outcomes’ &
‘time spent with patients’
‘time with patients’ with ‘outcomes’ & ‘quality of
interaction’
‘time spent with patients’ with ‘quality of
interaction’
‘outcomes for patients’ with both ‘time with
patients’ and ‘quality of interaction’
‘quality of interaction’ with ‘easy to use’
Extended NPM
items
All three with each other plus:
‘Coherence’ with ‘allocation’ ‘efficiency’, ‘division
of responsibilities’, & ‘easy to use’.
All three with each other plus:
reflexive monitoring with ‘accountability’ &
with ‘liability’
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 329
329
‘Cognitive Participation’ with: ‘skills are
appropriate’; ‘confidence in not risking patients’
& ‘easy to use’
‘Reflexive monitoring’ with: ‘allocation’;
‘confidence’; ‘efficiency’; ‘responsibilities’ &
‘easy to use’
High correlators
across constructs
efficiency
rewards outweigh effort
allocation of work
confidence in not risking patients
efficiency
the workload is manageable
Poor correlators
across constructs
Co-operation with others
Organisational compatibility
Co-operation with others
Different way of working
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 330
330
Appendix 24 WP3 TARS Specific Site 1: Cross tab analysis of agreement with TARS items by perception of level of routineisation
WP3 TARS Specific Site 1: Cross tab analysis of agreement with TARS items by perception of level of routinisation
Partly routine
N (non-agree, agree)
Completely routine
N (non-agree, agree)
χχχχ
1 The e-Health system is adequately resourced financially 19 (11, 8) 12 (8, 4) .239
2 Sufficient organizational effort has gone into supporting the e-Health
system 26 (15, 11) 14 (4, 10) 3.095
3 The e-Health system is a different way of working compared with other
parts of the NHS 26 (11, 15) 10 (5, 5) .031
4 The rewards of using the e-Health system outweighs the effort 23 (17, 6) 14 (5, 9) 5.268*
5 Government policy initiatives are supportive of this e-Health system 26 (13, 13) 11 (0, 11) 8.479**
6 This e-Health system is technically and organizationally compatible with
other systems and agencies that we are required to work with 25 (22, 3) 14 (13, 1) .230
7 This e-Health system fits in with the priorities and challenges of our
organization 27 (21, 6 ) 14 (3, 11) 12.061***
8 This organization has a culture that is supportive of change 30 (18, 12) 14 (1, 13) 10.870***
9 There is a culture in this organization of involving staff in planning and
development 30 (19, 11) 14 (8, 6) .154
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 331
331
10 Using the e-Health system makes me feel autonomous in my work 29 (23, 6) 14 (8, 6) 2.306
11 Using the e-Health system requires co-operation with other staff 28 (11, 17) 14 (6, 8) .049
12 The workload involved in using the e-Health system is manageable 27 (18, 9) 14 (5, 9) 3.586
13 In using the e-Health system, the allocation of work between individuals
is appropriate 21 (13, 8) 14 (5, 9) 2.307
14 The skills I have are appropriate for using the e-Health system 29 (10, 19) 14 (1, 13) 3.707
15 The skills needed to use the e-Health system are easily learned 29 (11, 18) 14 (4, 10) .364
16 I have confidence that using the e-Health system does not put patients
at risk 28 (12, 16) 14 (1, 13) 5.570*
17 Using the e-Health system is an efficient use of time 29 (20, 9) 14 (5, 9) 4.289*
18 In using the e-Health system, responsibilities are divided between
individuals appropriately 26 (15, 11) 13 (4, 9) 2.514
19 In using the e-Health system, I understand my accountability for my
work 29 (7, 22) 14 (0, 14) 4.036*
20 In using the e-Health system, I understand my liability for my practice 29 (6, 23) 14 (1, 13) 1.271
21 Technical back-up in using the e-Health system is available if I need it 28 (11, 17) 14 (1, 13) 4.725*
22 I believe there is good evidence about the clinical effectiveness of using
the e-Health system 30 (21, 9) 14 (7, 7) 1.650
23 There is some flexibility in how the e-Health system can be used 23 (12, 11) 13 (6, 7) .120
24 Using the e-Health system leads to positive outcomes for patients 26 (18, 8) 14 (7, 7) 1.436
25 Using the e-Health system involves the right amount of time spent with 29 (15, 4) 14 (7, 7) 3.039
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 332
332
patients
26 In using the e-Health system, the quality of professional and patient
interaction is good 28 (14, 4) 12 (7, 5) 1.296
27 The e-Health system is easy to use 28 (17, 11) 14 (3, 11) 5.775*
28 The staff who work here have a shared understanding of what the
system is for and how it is to be used 27 (18, 9) 14 (2, 12) 10.124***
29 The staff here are committed to making the system work 25 (12, 13) 14 (2, 12) 4.433*
30 There are ongoing mechanisms for monitoring and appraising how this
e-Health system is used 15 (8, 7) 14 (2, 12) 4.887*
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 333
333
Appendix 25 WP3 TARS Specific Site 2: Significant differences in comparison of Nursing/health staff (NH) with call handlers (CH) on TARS items, specified as percentages of professional group
WP3 TARS Specific Site 2: Significant differences in comparison of Nursing/health staff (NH) with call handlers (CH) on
TARS items, specified as percentages of professional group
Item:
Disagree
N (NH, CH)
Neutral or
some
agreement
N (NH, CH)
Moderate or
strong
agreement
N (NH, CH)
χχχχ
Using the e-Health system makes me feel autonomous in my work 49 (16, 30) 102 (40, 55) 68 (43, 15) 20.458***
The skills needed to use the e-Health system are easily learned 24 (15, 5) 119 (54, 49) 87 (31, 46) 9.509**
In using the e-Health system, I understand my liability for my practice 9 (2, 8) 62 (25, 34) 141 (73, 58) 7.490*
Using the e-Health system involves the right amount of time spent with
patients 43 (25, 17) 104 (56, 47) 54 (19, 36) 7.210*
* denotes significance level of p<0.05; **p<.01; ***p<.001
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 334
334
Appendix 26 WP3 TARS Specific Site 2: Comparison of perception of not/partly routine (NP) with completely routine (C) by TARS items (bracketed figures refer to % within perceived routinisation grouping)
WP3 TARS Specific Site 2: Comparison of perception of not/partly routine (NP) with completely routine (C) by TARS items
(bracketed figures refer to % within perceived routinisation grouping)
Item:
Disagree
N (NP, C)
Neutral or
some
agreement
N (NP, C)
Moderate or
strong
agreement
N (NP, C)
χχχχ
Sufficient organizational effort has gone into supporting the e-Health system 23 (30, 11) 83 (52, 50) 59 (17, 39) 7.757*
The e-Health system is a different way of working compared with other parts of
the NHS 9 (0, 7) 51 (63, 28) 98 (37, 66) 9.818**
This organization has a culture that is supportive of change 24 (9, 13) 98 (72, 47) 70 (19, 40) 6.868*
The skills I have are appropriate for using the e-Health system 7 (10, 2) 69 (55, 31) 123 (35, 67) 12.714**
In using the e-Health system, I understand my accountability for my work 6 (8, 2) 53 (50, 24) 135 (42, 74) 10.918**
In using the e-Health system, I understand my liability for my practice 9 (20, 3) 54 (36, 28) 124 (44, 70) 16.503***
I believe there is good evidence about the clinical effectiveness of using the e-
Health system 25 (24, 9) 119 (54, 58) 65 (22, 33) 7.109*
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 335
335
The staff who work here have a shared understanding of what the system is for
and how it is to be used 19 (22, 8) 98 (52, 50) 78 (26, 42) 6.576*
There are ongoing mechanisms for monitoring and appraising how this e-Health
system is used 6 (5, 4) 79 (74, 45) 78 (21, 51) 6.196*
* denotes significance level of p<0.05; **p<.01; ***p<.001
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 336
Appendix 27 WP4 Interview Schedule for Case Study 1 (Choose and Book)
This schedule was amended for use with Case Studies 2 and 3 by substituting references to Choose and Book with either Picture Archiving and Communication System (CS 2) or the Clinical Nursing Informatics System (CS 3).
Implementation:-
• What do you feel has prompted the implementation of C&B? (Drivers, needs) • Can you describe the implementation process followed? What stage is it now
at? • What has been your role in its implementation?
Within those elements of the implementation process that you have been involved with:-
• What factors do you think have assisted with implementation? Can you tell me a bit more about that? Have any factors that you felt would have helped with implementation been missing?
• What factors have not been so helpful with implementation? Can you tell me a bit more about that? What could have changed this? Are different factors important at different stages in the process?
• With hindsight, has anything that you thought would help (or hinder) the rollout, in reality not made a difference either way? Looking at C&B now that it has “gone live”:-
• What staff groups/units have been involved or affect by the implementation? • What do people who use it report back? • What particular aspects of C&B do you think have been positively received by
staff into their everyday practice? How do you know this? Why do you think this is?
• What has not been received so well into everyday work? How do you know this? Why do you think this is? Is there a difference been units or groups of staff?
• In your experience, which aspects of NHS work has C&B improved? • What dis-benefits have you come across? • Overall how well do you think C&B has been integrated into normal routines?
In this final section I would like to ask you a few more detailed questions on how C&B has impacted in practice on the work that healthcare organizations do. We may have touched on some of these issues earlier - however it would be helpful if you could think about them in more detail here, if you have any knowledge of them.
• What do you think has been the impact of C&B on the clinical encounter? Prompts:
o Changes within consultations and appointments (time, fluency, structure).
o How has it affected doctor-patient interactions? o How do you know this?
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 337
• What has been the impact of C&B on professional relationships between individuals and teams. (Clinicians, AHP and managers/admin staff). Prompt:
o Has C&B affected agreements about levels of authority, competency or performance/quality?
• How the division of labour with the organization been affected by the introduction of C&B? Prompts:
o What changes have occurred? o Was that expected? o What impact have any changes in work made to those groups
affected? o Has this led to any renegotiation of the way people work?
(workloads, wages, rewards or status) • How has introducing C&B impacted on the on the work of the
organization or its delivery of services. Prompts:
o What changes (positive or negative) have there been to policies, processes or budgets?
o Has performance / quality been affected? o What about existing arrangements about the procurement or
allocation of resources? o How successfully has this been managed? o How do you know this?
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 338
338
Appendix 28 Salient Features of Selected Case Studies
Salient Features of Selected Case Studies
Case Study 1 Choose and Book (C&B)
Background The Choose and Book service has been introduced across the NHS in England. CFH deployments began in the Summer of 2004. C&B combines electronic booking and a choice of place, date and time for outpatient appointments. C&B is located across the primary and secondary care interface and requires GPs to launch an electronic referral using the C&B software within the consultation. Via Choose and Book patients can choose their initial hospital appointment from available slots and book it either on the spot (within the consultation) or later on the phone or internet. Consultants log into the system via hospital IT networks to view work-lists and to accept, reject or change priority on referrals, as necessary.
Region London CFH Cluster: The London Cluster was one of the first to launch Choose and Book into Early Adopter Sites (EAS). Many EAS sites implemented Choose & Book onto non LSP*-delivered Patient Administration Systems. The experience of EASs was then used to produce an implementation toolkit for PCTs and acute Trusts across England. However despite such early efforts to facilitate implementation 2006 data4showed that within NPfIT Choose and Book remained the service attracting most criticism from NHS staff especially amongst doctors, practice managers and administrators. *Local Service Provider.
Contexts
• Policy level sponsor National policy level sponsor - part of NPfIT
• Service Sector C&B is located across the primary-secondary care interface via NHS network and IT systems. It also has a web based e-booking service application accessible by patients via the internet
• Proposed Users General Practitioners, Consultants, Patients
• Modality Software package implemented on to GP desktop clinical systems and hospital patient administration systems. Specific service application available on internet.
E-Health fields of activity
• Management systems Provision of an electronic booking system. Promotes an electronic method of sending referral letters and electronically tracking progress of approved appointments.
• Decision support systems
N/A
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 339
339
• Communication systems
The system allows clinicians to communicate information on referral need and priority.
• Information systems Allows both doctors and patients to access information on choice of hospital via NHS IT systems or the internet.
Timeline 2004 – 2007
Case Study 2 Picture Archiving & Communication System (PACS)
Background PACS enables digital images such as x-rays and scans to be stored and viewed electronically, so that doctors and other health care professionals can access the information and compare it with previous images. Files are stored centrally and images travel electronically to doctors and other healthcare professionals. NHS PACS is being delivered by CFH throughout England via five regions or clusters of strategic health authorities. PACS sites are live in all 5 clusters in England. 2006 data has shown that PACS continues to enjoy high favourability and low levels of critical opinion amongst NHS staff.4
Region Southern CFH Cluster: The southern cluster covers Cornwall to Kent and is where the implementation of PACS is most mature. CFH roll out of PACS began in the southern cluster in the spring of 2005 and at the time of the interviews 22 PACS had been implemented in the region.
Contexts
• Policy level sponsor National policy level sponsor - part of NPfIT.
• Service Sector PACS is currently being deployed within NHS secondary care locations, primarily within acute trusts. However in due course it will be implemented in any location where pictures of a medical nature need to be taken, stored or used for the purposes of NHS diagnosis or treatment. This will included some primary care and community settings.
• Proposed Users Staff involved in radiology, radiotherapy, angiography, cardiology, fluoroscopy, CT, MRI, nuclear medicine, ultrasound, dental and symptomatic mammography.
• Modality Digital images with accompanying hardware and software implemented onto hospital IT and network systems.
E-Health fields of activity
• Management systems Allows electronic storage and retrieval of digital images, integrates pictures with electronic patient records.
• Decision support systems N/A
• Communication systems Provides images that can be shared and multiply viewed.
• Information systems N/A
Timeline 2005 - 2008
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 340
340
Case Study 3 Community Nursing Information System
Background A mobile information system for use by community nurses which has been implemented in one Scottish Health Board. The Personal Digital Assistants (PDAs) support patient registration, case activation, clinic and visit scheduling, clinical document generation and information sharing between team members. The PDAs also have a facility for providing electronic access to clinical protocols although this not currently in use in the deployed system.
Region Scotland. The Health Board is the largest of it’s kind in Scotland employing over 800 District Nurses across 10 Community Health Care Partnerships (CHCP’s). The CNIS was first piloted in 2004 and subsequently rolled out across the CHCPs.
Contexts
• Policy level sponsor Local policy level sponsor. Mandate from the Scottish Executive to develop an electronic Single Shared Assessment document.
• Service Sector Primary Care. However the Health Board had integrated Social Services and Primary Care managed via 10 Community Health Care Partnerships with co-terminous boundaries.
• Proposed Users District nurses, staff nurses and nursing assistants
• Modality Web-based system Hardware: Personal Digital Assistants (handheld computers) that can be synchronized with desktop PC’s/wireless laptops.
E-Health fields of activity
• Management systems Helps maintain an electronic patient record. Potential to use the CNIS for caseload profiling, audit and resource management.
• Decision support systems Custom designed Shared Nursing Assessment document containing 10 assessment tools e.g. a nutritional scaling tool; a wound assessment tool; a continence assessment; a leg ulcer assessment tool and the Waterlow Scale. Offers some decision support functions.
• Communication systems Data from the Nursing Shared Assessment can be automatically imported into the Single Shared Assessment (SSA) document used by Social Services. Currently the SSA is sent by post however the intention is to send the document electronically via a multi-agency store. This should promote more efficient inter-agency communication and would enable either DN or SW to initiate an assessment.
• Information systems Some clinical information is available in “help” files.
Timeline 2007-8
SDO Project (08/1602/135)
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 342
Appendix 29 WP4 Print-out of the e-HIT.
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 343
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 344
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 345
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 346
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 347
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 348
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 349
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 350
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 351
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 352
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 353
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 354
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 355
SDO Project (08/1602/135)
Queen's Printer and Controller of HMSO 2007 Page 356
SDO Project (08/1602/135)
Disclaimer This report presents independent research commissioned by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the NIHR SDO programme or the Department of Health. The views and opinions expressed by the interviewees in this publication are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the NIHR SDO programme or the Department of Health Addendum This document was published by the National Coordinating Centre for the Service Delivery and Organisation (NCCSDO) research programme, managed by the London School of Hygiene & Tropical Medicine. The management of the Service Delivery and Organisation (SDO) programme has now transferred to the National Institute for Health Research Evaluations, Trials and Studies Coordinating Centre (NETSCC) based at the University of Southampton. Prior to April 2009, NETSCC had no involvement in the commissioning or production of this document and therefore we may not be able to comment on the background or technical detail of this document. Should you have any queries please contact [email protected].