Embed Size (px)
Citation preview
1
University of Bath
Royal College of Surgeons of Edinburgh
Healthcare Informatics
Supporting Narrative-based medicine in General Practice
computing systems
This project is submitted in accordance with the requirements for the degree of Master of Healthcare
Informatics of the Royal College of Surgeons of Edinburgh / University of Bath 2009.
Dr Ian McNicoll
Supervisor: Robin Beaumont
October 2009
2
Copyright Notice
Attention is drawn to the fact that copyright of this project rests with its author. This copy of the project
has been supplied on condition that anyone who consults it is understood to recognise that its
copyright rests with its author and that no quotation from the project and no information derived from it
may be published without the prior written consent of the author.
Restrictions On Use
This project may be made available for consultation within the University Library and may be
photocopied or lent to other libraries for the purposes of consultation.
Signature …………………
Disclaimer
The opinions expressed in this work are entirely those of the author except where indicated in the
text.
3
Abstract
As part of a redevelopment of a GP electronic patient record, and an attempt to introduce a semi-
structured from of data-entry, termed ‘structured narrative’, the importance of narrative to the GP
consultation was explored via literature review. Particular attention was given to the importance of
clinical cognition and the difficulties of integrating decision support tools into the human problem-
solving process. Alternative approaches to structured narrative were reviewed and a set of potential
generic information architectures were reviewed as possible data representations of structured
narrative.
An evaluation of the proposed structured narrative user-interface was carried out by a combination
of focus-group discussion and a ‘think-aloud’ task analysis of a set of dummy GP consultations,
performed with an early implantation of the interface.
The focus group discussions confirmed a need to allow narrative recording within GP systems. The
‘think-aloud’ evaluation was less successful due to methodological difficulties but did suggest that
the new interface successfully supported ‘structured narrative’ and allowed more naturalistic
interaction between clinician and computer, which may assist integration of decision and data-entry
support.
4
Contents
I Introduction
II Literature Review
1. Narrative and Medicine
Narrative within the clinical consultation
Narrative, cognition and complexity
Narrative within medical informatics
2. Clinical cognition
The ‘bereitschaftpotential’
Introspection
Probalistic decision-making
Naturalistic decision-making
Cognitive style
Knowledge transfer
Clinical problem-solving
3. Clinical cognition and narrative in practice
Medical error
Cognitive aspects of clinical information recording and retrieval
Clinical cognition in decision support
Narrative approaches in existing clinical systems
4. Clinical record architectures
The Terminology model (SNOMED-CT)
The Clinical Document Information model (HL7-CDA)
5
The EHR Information model (openEHR/CEN13606)
A candidate architecture for the current project
6
III Methods /Analysis
1. Focus group requirements gathering
Methods
The Clinical Workspace prototype
Results
Analysis
2. Evaluation of an early Clinical Workspace implementation
Methods
Segmented timings analysis
Qualitative assessment
3. Problems encountered within the evaluation
IV Discussion
Does narrative still have a place in clinical practice?
First impressions?
Decision support and flexible data-entry
Practical evaluation
Technical framework
Next steps
7
8
I. Introduction
‘Use the force, Luke, Let go!’ He turned off his targeting computer – certain folly! But the torpedoes
went down the ventilation shaft and Death Star, the most advanced weapon to threaten the universe
exploded.’
In his short article “May the Force be with you”1, Dr Des Spence, a Glasgow GP, defends the ‘medical
heresy’ of clinical intuition in an era where targets and evidence hold sway.
During my 15 years as a GP principal, as I moved from a traditional medical didactic style of
consulting to a more holistic approach, I became increasingly aware of the frustration in using
clinical software and of a mismatch between the underlying data-entry paradigm and my own
clinical cognition patterns.
In spite of over 30 years of development, uptake of the electronic health record remains far from
universal2. One of the key difficulties identified is that of providing an appropriate user interface
within the context of the clinical encounter to a wide diversity of clinical users3. Many reasons have
been postulated for the difficulty of persuading clinicians to record their findings electronically – lack
of keyboard skills, lack of familiarity with coding systems, lack of appreciation of the value of
structured and coded data, simple techno-phobia, but none of these really applied to my own
situation.
I began to wonder if the discomfort experienced by some (perhaps many) clinicians when
performing structured or coded data-recording, and their preference for narrative, was related to
individual cognitive function and ‘cognitive comfort’ rather than a simple aesthetic preference or
technophobic resistance to change. Perhaps, as described by Don Norman, this is due to “Being
analog in a digital world”4. Similar themes have emerged in other aspects of clinical practice, as the
art and science of UK general practice have changed markedly, evidence-based medicine (EBM)5
establishing itself as a major driving force. This change is due in part due to the development of truly
effective pro-active care, such as in the use of statins in cardiology, but has only been made feasible
by the almost universal uptake of GP computing systems in the UK, as evidenced by the positive
response of most UK general practices to the new GMS contract and Quality Outcomes Framework
(QOF)6.
9
Some clinicians have started to react to this structured style of practice, emphasising the importance
of narrative in medical care7, going beyond a reductionist view of health to a more holistic, patient-
centred approach, echoing the emphasis on communication skills and consultation-analysis which
remain key and established features of general practitioner training8.
I began to investigate the possibility of devising a GP system user-interface termed the Clinical
Workspace which would allow the faithful recording of the narrative of a consultation whilst
integrating other structured, computable items of clinical information such as clinical codes or
prescriptions, termed Clinical Narrative (CN). To assist data–entry it was proposed that, based on
the CN method, clinicians develop personalised ‘Clinical Scenario Templates’ (CST) to allow the rapid
entry of ‘boilerplate’ clinical terms and actions. The effectiveness of this approach would be based
on the ability of experienced clinicians to very rapidly identify the likely clinical context, if not
outcome, at the start of a consultation. Having established a likely scenario such as ‘possible urinary
tract infection’, ‘low back pain’, or ‘possible chest infection’, each clinician tends to have an
individual but consistent pattern of information recording, prescribing, referral etc. It is therefore
possible to ‘pre-load’ the user interface with the correct elements both to assist data-entry and to
facilitate outcome actions. In some respects ‘clinical scenarios’ seem to have much in common with
electronic guidelines, whose implementation difficulties have been well described9. In contrast to
guidelines, clinical scenarios are designed to be personalised and coupling them with the narrative
data-entry paradigm allows a much more interactive approach. If it becomes clear that the initial
scenario selected was incorrect the active scenario can be switched mid-encounter, or indeed
‘dropped’ and ignored, without losing any data already entered.
In addition to investigating these user-interface requirements, a set of data models would be
required as the information components to underpin the Clinical Workspace.
At that time that this interest began, I was involved in a clinical design group tasked at improving the
clinical user interface to the Scottish national GP system GPASS. Demonstration of a prototype of the
Clinical Workspace received a positive response from fellow clinicians and it was agreed to
incorporate some of these into a new version of the application.
10
Aim
The aim of this project was to gain a better understanding of the importance of narrative in
medicine and to clinical cognition, to identify a generic technical framework where the ideas could
be expressed independent of any particular supplier or implementation and to evaluate the Clinical
Workspace as it was commercially developed.
Objectives
A set of questions thus identified themselves as the basis for this project:
1) Is there evidence that narrative remains an important construct to clinicians, independent of
experience with or attitudes to clinical computing?
2) Is there evidence to support the 'first impressions' basis of the CST mechanism i.e. rapid
identification of the likely clinical scenario?
3) What have been the outcomes of previous attempts to integrate narrative, decision or data-entry
support into electronic records?
4) What might be the optimum open standards-based framework in which to develop Clinical
Workspace components?
6) Is there evidence that the proposed Clinical Workspace is better integrated with the pattern of
clinical cognition in a GP consultation?
Methodology
A literature review was conducted to gain insight into the role of narrative within clinical
communication, the significance of the cognitive processes which underlie clinical decision making,
the nature of current clinical data-entry paradigms and the identification of a suitable ‘information
model’ architecture to support structured clinical narrative10. In addition the review sought to
identify the differing styles of electronic clinical guideline authoring, previous similar user-interface
paradigms and the reasons for known difficulties in implementation.
11
As part of the real-world GPASS re-development, focus group-based discussions were conducted to
gain an understanding of the requirement for narrative recording. Finally an evaluation was
conducted of an implementation of the Clinical Workspace using an introspective task-analysis
technique known as ‘Think-aloud’.
12
II Literature Review
1. Narrative and Medicine
Narrative has been defined as
“… a linguistic form [which] has a finite and longitudinal time sequence - that is, it has a beginning, a
series of unfolding events, and (we anticipate) an ending” 7
Clinical practice is suffused with story-telling: the case study presentation, the tales of ‘near-miss’,
the tales of exception, “There was this one patient”11
Medical undergraduate teaching is steadily being transformed away from the traditional rote-
learning of facts dispensed by individual disciplines such as biochemistry, cardiology or therapeutics/
Today’s medical student will be taught using a holistic case-based, topic-centered approach, often
based on the story of a patient’s journey through a personal healthcare episode such as a myocardial
infarction13.
Coupled with this humanistic, problem-based approach, students are taught the discipline of
Evidence-based medicine (EBM)5 as a means of assessing and evaluating the almost limitless
endeavour of basic and applied medical research. Proponents of EBM correctly emphasise the
nuanced application of this approach as it applies to an individual patient:
“Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in
making decisions about the care of individual patients.”14
Nevertheless, the weight of gold-standard clinical trials, meta-analysis and multiplicity of clinical
guidelines can feel like ‘cookbook medicine’ to many practitioners15 and the timely integration of
EBM into clinical practice remains a challenge, a topic which will be discussed further under the
heading of decision support.
The importance of clinical story-telling via the case study or personal experience is now largely
deprecated in favour of statistically-backed ‘evidence’, but it should be remembered that the
substantial progress in patient care over the past 150 years was largely made on the basis of
13
anecdote, or at least, research whose scientific methodology would fall far short of currently
acceptable standards16.
Even in the more ‘pure’ scientific fields such as geology or indeed mathematics, scientific progress
and communication continue to depend on the ability of the researcher to deliver a convincing story
of causality, as well as methodological rigour17.
Stephen Nachmanovitch, a musician and clinician, describes the conflict between the biomedical
model and the need to view the patient as an individual18:
"In real medicine you view the person as unique—in a sense you drop your training. You are
immersed in the case itself, letting your view of it develop in context. You certainly use your training;
you refer to it, understand it, ground yourself in it, but you don’t allow your training to blind you to
the actual person who is sitting in front of you. ... To do anything artistically, you have to acquire
technique, but you create through your technique and not with it."
‘Narrative medicine’ has become a topic of academic interest in its own right, though largely
orientated towards enriching clinical education by introducing the study of humanities into the
medical curriculum. This, it is felt, will enable clinicians to appreciate the more human and aesthetic
aspects of clinical care, such that Haidet talks of “Building rather than Taking”19 a clinical history, a
process of mutual information sharing which tries to capture both patient and clinician perspective;
or a “narrative-based medicine” approach20.
Narrative within the clinical consultation
Alongside these broader narrative approaches to medical education and practice, there has been
considerable interest in the analysis of the clinical consultation, particularly within general practice,
dating from the early 60s and the writings of Balint21. These explored the importance of the doctor-
patient relationship within the setting of a typical GP consultation from a psycho-analytic
perspective and the importance of communication beyond simple diagnosis and “issuing bottles of
medicine”. Offering useful insights such as “The Doctor as Drug” i.e. the placebo effect of a caring
clinician and the value of the “extended consultation” taking place over weeks or years, Balint’s work
has had a profound impact on both undergraduate and postgraduate teaching, where within a
decade the teaching of consultation skills became a cornerstone of UK GP postgraduate medical
education, with a clear emphasis on patient-centeredness8,22 .
14
The importance of consultation skills beyond simply diagnosis and advice was further enforced in the
UK by their formal assessment within the examination for Membership of the Royal College of
General Practitioners and, in some areas, in the summative assessment of GP trainee registrars,
failure of which would place a bar on continuation of a career within UK general practice.
This approach to the consultation, which makes a deliberate effort to explore the patient’s
understanding and goals, contrasts with research undertaken by Patel within the much more
communicatively traditional setting of USA-based secondary care23. This noted the disparity between
the patient’s narrative account of their problem, framed within in their own interpretation of events,
and that of the clinician, who recasts this tale into biomedical dialogue. Although superficially
composed as free text narrative, Hunter notes that this type of clinical record (consultation note,
continuation note, journal) is a highly stylised and structured form of narrative “This is what she has
told me, this is what she looks like; here are my observations and the preliminary sense I have made
of the information I have gathered...”11.
Traditionally, the clinical record has been “the clinician’s story” and though most professionals now
welcome the additional insights to be gained by capturing the patient’s perspective, the key role of
the clinician is to transform the patient’s description of events, to one which is compatible with the
biomedical model, if only to exclude a ‘biomedical approach’ as being inappropriate to the issue
presented. As patients are accorded increasing access to their medical records and the patient held
electronic medical record gains acceptance24 , there will be further debate about who ‘owns’ the
record and ‘whose story is it anyway?’
Patel has described the tendency of electronic representations of clinical records to amplify the risk
of losing the patient’s perspective, primarily through rigid paradigms of data-entry23 . However it
could be argued that this is not a problem inherent to electronic medical records but merely reflects
the attitudes of the clinicians who guided such application development. Indeed, the new paradigm
of ‘Web 2.0’ web-based collaborative applications, described by O’Reilly as the ”architecture of
participation”25, affords the possibility of enabling direct patient contribution and annotation of their
online records26.
The patient narrative and the patient perspective, has a key and increasing role in improving the
delivery of clinical care but this study concentrates on the role of clinically-orientated and derived
narrative within the setting of a traditional clinical consultation; as recording device, cognitive aid
and decision support/learning tool.
15
The similarity of the discourse of a traditional clinical consultation, and its concomitant record, to
story creation is readily apparent; the patient presents with an issue, tells their own story as “a
series of unfolding events”, often prompted by the clinician. The latter, based on knowledge and
expertise, creates their own ‘biomedical’ version of the tale, with a temporally orientated ‘unfolding’
description of events and findings, culminating in a ‘dénouement’ in the form of diagnosis and
suggested actions e.g. in a typical simple concise GP record:
“Symptoms: Gives a 3 week history of pain passing urine with possibly some blood and
feeling generally unwell.
Examination: Slightly tender both loins Pulse 80 BP 120/78
Diagnosis – probable Urinary Tract infection
Treatment: Trimethoprim 200mg twice daily for 3 days
See in 5 days if no better”
Similar sparse but consistent styles of storytelling are also to be found in a variety of clinical ‘genres’
such as the “Case report”, “Discharge letter” or “Clinical handover” . The communication
requirements of the latter have recently attracted considerable research interest, in an attempt to
improve patient safety27, particularly as clinicians’ working hours are shortened, requiring more
‘shift-changes’ and cooperative working.
Narrative, cognition and complexity
The human use of language both to communicate, and to internally represent complex constructs,
would seem to place it very close to the cognitive process but the exact relationship is hotly
contested between those who regard language and cognition as separate but related skills28, and
those who regard language as a direct expression of cognition, and as a consequence, cognition
cannot take place in the absence of language29.
The exact significance of narrative constructs also remains in dispute. Early attempts to marry
psychology, linguistics and computational theory as “cognitive science” focussed on the internal
structure of language30 and regarded attempts to grasp the meta-meaning of narrative as too
complex a challenge. More recently, cognitive scientists have attempted to grasp this nettle,
16
particularly Schank and Abelson31, who made three propositions in postulating a key role of
narrative within human knowledge, memory and decision making:
1. That virtually all human knowledge is based upon stories constructed from past experience.
2. That new experiences are interpreted in the light of old stories.
3. That when stories are retrieved from memory they are variably interpreted according to the
listener and may themselves be revised by this process, updating the person’s “remembered
self”.
They put forward a novel suggestion that narrative and story-telling are at the heart of human
cognition and problem-solving, the story acting as both the prime organisational construct of
memory, and key index mechanism for retrieval and manipulation of memory. Whilst they
acknowledge that this theory currently lacks firm evidential support, it seems to accord well with
emerging paradigms of human cognition based on ‘schema’32and ‘framing’33.
Herman 34 identifies numerous parallels between theories emerging from social science and
‘Narratology’, the study of narrative as a literary construct, and others from the cognitive sciences.
He identifies ‘Story Logic’ as a useful target for future research, attempting to understand the
processes by which story recipients reconstitute a richly contextual but implied ‘storyworld’, evoked
by the cues contained in narrative. As an example, a GP describing an ill patient to a receiving junior
hospital doctor, will communicate significant, objective measures of illness such as pulse, blood
pressure and temperature but will also often emphasise his concern with a more emotionally laden
phrase, such as “I don’t like the look of this at all”. As junior doctors gain experience of taking
admissions from local GPs they learn to judge the significance of such statements, partly
reconstructing the GP’s ‘storyworld’ from their own experience of subjective concern for patients.
The emerging discipline of complexity theory has attracted considerable attention in a range of
disciplines from meteorology to management science35. There are a small number of key attributes
of ‘complex adaptive systems’:
1. They exhibit non-linearity and sensitivity to initial conditions, so that small changes in initial
variables may lead to wildly differing outcomes.
2. Paradoxically, complex systems may also display considerable properties of self-organisation
and be resistant to change.
17
3. The structure of complex systems is often ‘fractal’, with recursive symmetries between scale
levels. The common example given from nature is the tree-branch-leaf-internal leaf
structure, which displays very similar branching structures at every level from macroscopic
though to microscopic.
4. Complex adaptive systems display ‘emergent properties’ which although a consequence of
the existing system, cannot be accurately predicted in advance.
Biological systems and human social structures appear to display many of the properties of complex
adaptive systems, which has led to considerable clinical interest36, particularly in fields such as
general practice, where daily experience can often seem to be ‘on the edge of chaos’ 37. Intriguingly,
in what might be thought of as the more reductionist field of pure biological measurement, there is
some evidence that linearity, the antithesis of complexity, may equate to ill-health, whilst non-
linearity is a marker of a well-functioning biological system. As an example, a ‘normal’ pulse displays
minute, non-linear ’beat to beat’ variation which is reduced in abnormal states of health38,39 .
Heath40 comments upon the symmetries of storytelling, healthcare and complexity,
“Both novelist and general practitioner are committed to keeping the particular alive, resisting
simplification and embracing the complex reality of lived experience.”
Given these shared approaches of those advocating the importance of narrative in human cognition
and healthcare, and those attracted to complexity science, there is a surprising dearth of more
formal academic literature attempting to relate these concepts.
Tsoukas and Hatch41, from a management science perspective, have looked at the capacity of
narrative to ‘tame’ organisational complexity whilst Purves and Robinson42, within the field of
knowledge management for clinicians, tentatively explore the possible relationships between
narrative, complexity and knowledge but otherwise, a possible relationship between complexity,
cognition and narrative seems surprisingly under-researched.
Some new developments in basic human brain research appear to demonstrate that at a
fundamental operational level, as judged from signal and functional MRI evidence, the brain
functions as a complex system “on the edge of chaos”43. Although this level of brain operation is far
removed from cognitive processing, it might be postulated that the brain acts as a ‘complexity
processor’, highly adapted to recognising and exploiting subtle patterns in an infinitely complex
18
world, narrative being an effective and efficient means of communicating the processed
information, and capable of reflecting that complexity.
Narrative within medical informatics
In 1999 Tange44 published a literature review of the handling of medical narrative within electronic
medical records (EMR). A clear theme emerges of the tensions between the requirements of
systems developers and the consumers of data for analysis purposes e.g. public health, who prefer
highly structured computable data, as against a significant number of clinical users who wish to
maintain a narrative style of clinical recording. These tensions were recognised as a significant factor
in preventing the rapid acceptance of EMRs. “All too often, the need for structured data conflicts
with the need for free texts and the power of expression.”45
In discussing, what at that time were newer ‘experimental’ systems, Tange notes that these
applications were increasingly designed to support the capture of medical narrative. Examples are
the development of dynamic structured data entry by systems such as Pen&Pad46and early work on
compositional semantic logic based terminologies, such as GRAIL47, whose influence can be seen
within in SNOMED-CT48.
This approach, in attempting to represent narrative by a highly granular, expressive but controlled
vocabulary was countered by Lincoln and Esson49 who pioneered the use of emerging ‘markup
languages’ to avoid rigid data models which they felt lacked the flexibility required to ‘deal with the
unexpected’ in clinical practice. They argued that SGML, a precursor of XML, allowed clinicians to
gradually structure clinical narrative to whatever level of detail was required, following a document-
centric approach which has now reached maturity in the form of HL7 CDA50 and openEHR51.
Other work from the same period demonstrates a general re-evaluation of the significance of
narrative in medical communication and cognition. A series of articles by Berg52-54, re-examine the
place of traditional paper records, as a reaction to earlier attempts led by Weed55 to banish ‘unruly
narrative’ and promote structured clinical recording. Berg recognises the subtle ‘entanglement’ of
record production, use and retrieval, with the clinical context pertinent at the time of recording, and
with the wider ‘socio-technical’ milieu in which the records are created and understood. Existing
highly-structured or coded records, often failed to recognise these broader contextual factors,
leading to resistance to uptake and a loss of ‘nuance’ in the data collected.
19
Building on the newly emerging themes from cognitive science, narratology and complexity science,
Kay and Purves56 proposed a set of models and metamodels which recognise and promote the use of
narrative within the EMR, claiming that “Medical Informatics are actually in danger of reducing the
semantic richness of, and of degrading the story to limited codes and weakly connected phrases”.
It is probable that some of the renewed interest in narrative was prompted by the emergent and
largely text-based, World Wide Web with the successful development of powerful text indexing and
search algorithms as used by Google57. Text is no longer anathema to computing scientists previously
focussed on semantic purity. Coiera58 reflects this change in mindset within medical informatics in
his paper ‘When Conversation is better than Computation’. He notes that the truly world-changing
technological developments of the late 20th century were largely communicative e.g. mobile phones,
SMS texting, web, and not due to complex computing of semantically precise data sources. Coiera
also points to research which shows that, even in a highly computerised clinical facility, only about
10% of information transactions occurred within the EMR59, the remainder being by personal
contact or communication-assistive technologies such as email or integrated messaging.
In the general IT domain, the development of semantically-rich and precise ‘ontology’ for the web
remains the ‘Holy Grail’ of the Semantic Web project, a goal which has been challenged by Clay
Shirky60
In current clinical systems, released or in development, we start to see the themes identified above
being adopted as standard – SNOMED-CT as a clinical ontology and terminology, HL7-CDA as a
structured, contextual clinical document, and XML-based integrated messaging such as NHS Scotland
SCI-Gateway61.
Summary
Much of the impetus for clinical system design originated with Weed who correctly described the
dangers of wholly subjective clinical narrative and sought to recast the medical history as a scientific
document, a positivist approach paralleled in the trends towards evidence-based medicine and
structured care. Computerisation was, of course, a prime and necessary precursor to this process.
In contrast, many clinicians, whilst broadly supportive of these changes, nevertheless see value in a
less medicalised worldview and the continued expression of aspects of clinical documentation as
narrative.
20
2. Clinical cognition
Clinical cognition may be defined as the study of the mechanisms underlying clinical problem solving
and decision making and is a branch of cognitive science, the study of mind and intelligence, both
human and artificial, often drawn from multiple disciplines such as psychology, social sciences,
linguistics and computer science.
In the early 1950s, research psychology was dominated by the study of Behaviourism following the
lead of Pavlov and Skinner. Proponents of behaviourism maintained that psychology could only be
regarded as an objective science if solely based on observed behaviours and held that descriptions
of mental processes or theories of the mind were opaque to such observation.
The ‘Cognitive revolution’62,63 led, amongst others, by Chomsky, Miller and Bruner was a response to
these ideas, drawing inspiration from the emerging disciplines of computational theory and artificial
intelligence and which regarded the human mind as being amenable to testable ‘reverse-
engineering’ by comparison with successful machine intelligence.
From Miller, The cognitive revolution: a historical perspective63
Pinker64 identifies a series of ideas which characterise the impact of the cognitive revolution, such
that the cognitive approach has become the dominant research activity within applied psychology in
the past 30 years:
The mental world can be grounded in the physical world by the concepts of information,
computation, and feedback.
The mind cannot be a blank slate because blank slates don't do anything.
21
An infinite range of behaviour can be generated by finite combinatorial programs in the
mind.
Universal mental mechanisms can underlie superficial variation across culture.
The mind is a complex system composed of many interacting parts.
Central to the study of human cognition was the early emergence of Information Processing Theory
(IPT) which views the human mind as processing information through the application of logical rules
and strategies. Just as a computer, the human mind appears to have a limited capacity of processing
power and memory capacity. Miller65 ,in ‘The Magical Number Seven, Plus or Minus Two’,
demonstrated that the short-term memory of the human mind could only hold 5-9 ‘chunks’ of
information, a chunk being any ‘meaningful unit’ which may encompass entities of widely varying
scale from a single digit to the details of a person’s face.
The ‘bereitschaftpotential’
Although by the early 1960's, cognitive science was becoming an accepted discipline, it received at
that time a powerful endorsement through the discovery by Deecke and Kornhuber of the
'bereitschaftpotential', a brief cortical electrical signal which precedes any voluntary movement66.
This posted clear evidence in favour of cognitively-mediated volition and against the deterministic
viewpoints of Behaviourism and Freudian psychoanalysis, where human behaviour and affect is seen
as a largely pre-determined and immutable consequence of learned patterns.
Some later fascinating research by Libet cast doubt on such 'cognitive volition'. In his research,
subjects were asked to move a wrist at a time of their choosing, whilst noting the precise time at
which they consciously decided to make the movement. Libet also measured the
Bereitschaftpotential and found that this preceded the subject's awareness of having made the
decision to move by about 350 milliseconds, the decision itself preceding actual movement by about
200 milliseconds. He was able to show that this possibly surprising sequence of events could not be
accounted for by the delay in the subject reporting and noting the decision time.
Libet’s work has been contested both on methodological grounds, e.g. that he had not sufficiently
accounted for recording and noting delay, and on the deeper philosophical implications of such
'unconscious volition' but his findings do not seem wholly at-odds with normal human experience.
Indeed, this seemed blurring of the voluntary and involuntary, resonates with the ability of the brain
22
to store, refine and very rapidly reuse patterns of cognition as part of clinical decision-making, issues
which are discussed in more detail later in this chapter67.
Introspection
One of the principal and long-standing debates between behaviourism and cognitive science was the
question of the validity of ‘introspection’, the self-reporting of a person’s inner thought processes,
experiences or feelings as invalid. Since such self-reporting forms a very significant part of current
approaches to the investigation of clinical cognition, and indeed is part of the methodology of this
study, the debate on introspection deserves some discussion here.
The controversy has its origins in the early 20th century, initially between American introspectionists
who embraced ‘ sensationalism’ , the theory that it is impossible to think without creating some sort
of concrete internal image, and their German equivalents who held that such ‘imageless thought’ at
least for some ideation, was indeed possible. Their prolonged and unresolved dispute, with
seemingly contradictory research findings simply discredited introspection as a valid research
methodology. However a review of both parties’ findings by Monson and Hulbert in 1993 suggested
that their research results were in fact very consistent, and that the originally observed differences
were due to the theoretical perspective from which they were reported, and that both observed
‘vague and elusive processes, which carry… meaning’ – somewhat less concrete as images than
American sensationalism demanded and a little more concrete than the German school of
‘imageless thought’ would have preferred68.
Another criticism of introspection is that subjects are often unable to accurately and reliably relate
their inner thoughts and feelings, and that “Tell me what you are thinking” may be interpreted
variably by different subjects as “What am I thinking?”, “Why am I thinking this way?” or “What am I
feeling?” A 1977 review by Nisbett and Wilson, “Telling more than we can know” is critical of the
reliability of introspection-based research suggesting that “when people attempt to report on their
cognitive processes … they do not do so on the basis of any true introspection”.69 but did recognise
that it was possible for some people to give accurate reports about inner thoughts in some
circumstances. The necessity to do some degree of coaching in introspection is now well recognised,
particularly within cognitive behavioural therapy, where the initial focus is on teaching the patient to
correctly introspect, with the knowledge that not all individuals have the capacity to do this
effectively. In a 2001 review of the introspection debate, Hurlubert, a proponent of an
introspectional methodology known as Descriptive Experience Sampling (DES), acknowledges that
accessing inner experiences is not as simple as “just asking” but claims that the results obtained via
23
DES, after some initial coaching, produce acceptable valid and reliable results68. From his experience
of DES, Hulubert accepts that introspection as “How am I feeling?” is more problematic than
introspection as “What am I thinking?” and that subjects may have to be coached to retrieve the
latter, rather than the former.
Probalistic decision-making
With the rapid advancement and availability of computing power, early research was drawn to the
study of computer-based intelligence which might augment or even replace human decision making.
This research led to a useful understanding of flaws in human decision making, particularly the
effects of systematic bias and ‘framing’, which might lead to inaccurate assessment of risk70. A key
focus was the creation of expert systems, which sought to encapsulate the knowledge and decision
making powers of ‘domain experts’ within computer systems. Much of this work focused on the use
of probabilistic reasoning such as Bayes’ Theorem to enhance decision making in complex
environments where information is incomplete71. Whilst clinical expert systems such as ONOCIN72
could demonstrate considerable accuracy within highly constrained environments, more general
applications proved unpopular, seemingly unable to operate sympathetically with real-world clinical
decision making. It proved particularly challenging to model the complexities of the clinical
environment often lacking a sound evidence base on which to base probabilities, where context is
highly significant and time highly pressured71.
24
Naturalistic decision-making
Other researchers, using direct observational methodologies, began to study ‘real-world’ examples
of human cognition, leading to the theory of Naturalistic Decision making (NDM)70. In 1960 Bruner73
introduced the broad concept of two modes of thought:
Analytic thinking which proceeds step by step and involves careful and deductive reasoning
Intuitive thinking which ‘tends to involve manoeuvres’ based seemingly on an implicit
perception of the total problem’
He later74 reframed ‘analytic thinking’ as ‘logico-scientific mode’ and intuitive thinking as ‘narrative
mode’:
The importance of context was reinforced by Simon and Newell75 who emphasised that human
reasoning is based on the relationship between the human problem solver and the ‘task
environment’ - the goal, problem or task and the context in which these exist.
25
Marshall32 contributed Schema Theory built upon ideas originally developed by Bartlett in the 1930s
which describes how mental models are constructed for problem solving. Marshall regards a schema
as a construct which allows grouping of an individual's similar experiences so that the individual:
can easily recognize additional experiences that are similar, discriminating between these
and ones that are dissimilar;
can access a generic framework that contains the essential elements of all similar
experiences, including verbal and nonverbal components;
can draw inferences, make estimates, create goals, and develop plans using the framework,
and;
can utilize skills, procedures, or rules as needed when faced with a problem for which this
particular framework is relevant
Marshall further describes four styles of knowledge of which the first two are relevant to the clinical
diagnostic process as the precursors to taking appropriate action:
1. Identification knowledge allows rapid pattern recognition of an issue and happens as a
result of the many cognitive processes occurring together; no single stimulus triggers the
recognition.
2. Elaboration knowledge details the key aspects of the issues leading to schema
development. When the general situation has been recognized by identification knowledge,
information about the current experience will be accessed from a previous 'template' about
the situation. Understanding of the issues presented depends on the degree to which these
fit the schema template.
Types of knowledge base (after Marshall)
26
Cognitive style
A somewhat separate area of interest has been ‘Cognitive style’ which refers to consistent aspects of
a person’s manner of cognitive functioning, particularly with respect to acquiring and processing
information. Tennant76 describes cognitive style as “an individual's characteristic and consistent
approach to organising and processing information". This was a burgeoning field from the 1950s
through to late 70s being closely related to efforts within management science and educational
research to match knowledge acquisition and decision making processes to cognitive and personality
traits. A large variety of overlapping and conflicting theories emerged, in spite of regular attempts
to develop a unitary theory77. The most persistent dimensions of Cognitive style have been
summarised78 as:
The Wholist-Analytic Style dimension of whether an individual tends to process information
in wholes or parts.
The Verbal-Imagery Style dimension of whether, during thinking, an individual is inclined to
represent information verbally or in mental pictures.
The wholist-analytical dimension seems to resonate, both with Bruner’s distinction of logico-
hypothetical and narrative modes of thought and with management science research79 which
proposes three broad types of management approach:
The analytical, preferring to solve problems by breaking these into manageable parts using
analytical and quantitative techniques.
The intuitive, relying more on feelings to make decisions, preferring unstructured situations,
and solving problems holistically.
The integrated, using both analytical and intuitive decision making interchangeably as the
situation demands.
Research into cognitive style seems to have been a little neglected in recent times, being regarded as
of secondary importance to the more generic aspects of human cognition but there has been a
resurgence of interest with the development of ‘cognitive ergonomics’ in relation to user-interface
design and the idea of the ‘cognitive discomfort’ generated when individuals are forced to work in a
less personally suited cognitive style. Cegarra80, in a study of non-clinical problem-solvers,
27
demonstrated differences in performance related to cognitive style, and independent of, though
complementary to expertise.
Within a clinical environment, Mandell81 examined the relationship between cognitive style and
resistance to computerisation and claims a correlation between a heuristic/ intuitive decision-
making style and a resistance to computerisation. There are number of difficulties with this study.
The participants, a mixture of nursing and social work staff in a paediatric unit reported attitudes to
computerisation via questionnaire. Cognitive style was assessed via the same questionnaire,
primarily by assessing individual beliefs as to whether clinical data is best seen as quantitative,
rather than qualitative, whether structured or unstructured data is preferable and whether their
work should be regarded as an art or a science. Whilst this seems as reasonable observation of
philosophical/attitudinal preferences, it is arguable whether such self-reports accurately reflect
‘cognitive style’ as might be more appropriately assessed by direct observation of practical
cognition.
Knowledge transfer
A significant separate area of interest, particularly within management science and educational
research, has been ‘knowledge transfer’ within organisations and between individuals, both in
commercial and educational settings. ‘Tacit knowledge’, or “knowledge that usually is not openly
expressed or taught”82, was first described by Polanyi83 who suggested that “we can know more than
we can tell”. In contrast, ‘Explicit knowledge’ describes knowledge acquired by formal methods of
education or instruction. Tacit knowledge is increasingly recognised as a valuable resource within
organisations and a great deal of management research has been directed at improving the transfer
of the tacit knowledge of individual personnel to the organisation as a whole. The foremost
exponents of this kind of knowledge transfer emerged from Japanese companies within the
framework of what has come to be termed as the ‘Lean’ approach84. - “Making personal knowledge
available to others is the central activity of the knowledge-creating company”.
There remains considerable debate as to the best mechanism for affording effective knowledge
transfer, and the place of rich narrative in sharing such expertise. Herschel85 describes a study
comparing the use of structured versus narrative text to impart knowledge via video where he states
that the use of rich narrative is vital to the conversion process, Surprisingly the format used
(Narrative vs. Structured) appeared to have no effect on the effectiveness of transfer itself but the
28
use of structured evaluation to trigger recall was more effective than a free text question such as
“what did you learn from this video”? Interestingly this study, thought not set within a clinical
domain, derived its structuring style from Weed’s SOAP structure for the medical consultation86.
Other researchers, in terms reminiscent of the narrative v. evidence-based medicine debates
challenge the idea that all tacit knowledge can be structured and codified or that codification of
itself should be represents progress87.
Clinical problem-solving
Specific interest in the problem solving skills of clinicians initially emerged to address the needs of
medical educators – how best to impart the expertise of experienced clinicians to both
undergraduate and postgraduate trainees? Elstein88 who, inspired by similar research into the
cognition of chess players, and building on the work of Simon and Newell75 , developed an
observational research methodology known as ‘think-aloud’ where study participants are asked to
solve problems in a real-world setting, and to narrate their thought processes as they go along. This
approach contrasted with earlier decision-making probabilistic research which tended to be lab-
based and other behaviourism-driven research which Elstein describes as a ‘test-and-measure’
approach rather than by direct observation of clinicians actually making decisions.
Elstein’s key finding was that clinicians appeared to make very rapid hypotheses at the start of a
consultation which he described as ‘hypothetico-deductive reasoning and early hypothesis
generation’ and which then guided further data acquisition. He argued that this approach is a
‘psychological necessity’, given the complexity of most clinical scenarios, and limited capacity of
human working memory. He demonstrated that successful diagnosticians did not generate more
hypotheses or hold more hypotheses in working memory but did more successfully interpret the
data available.
He proposed four phases of diagnostic decision-making:
1. cue acquisition
2. hypothesis generation
3. cue interpretation
4. hypothesis evaluation
29
Research by Groen and Patel89 initially challenged this view, showing that experienced secondary-
care clinicians, dealing with routine problems, seemed to move directly to a data acquisition phase,
then to an eventual diagnostic hypothesis. They showed that this ‘forward-reasoning’ process, from
data to hypothesis, correlated well with accurate diagnosis, whilst clinicians who used ‘backward
reasoning’, from hypothesis back to confirmatory data, performed less well. This pattern seemed to
apply equally to clinical novices and to experts working outwith their normal area of expertise90. The
findings seemed to point to a cognitive process more akin to rapid pattern matching, rather than a
deductive process of reasoning.
Barrows91 et al, carrying out similar studies in primary care seemed to support Elstein’s original
finding of early hypothesis generation, with a correlation between the accuracy and promptness of
the early hypothesis and the accuracy of the eventual diagnosis. In a ten-year review of his original
paper, Elstein suggests that the differences between his and Patel’s view are largely of emphasis;
when experienced clinicians are dealing with a familiar or routine scenario, ‘cue acquisition’ so
rapidly and accurately identifies the problem, that any initial hypothesis generation is bypassed or at
least is made opaque by being ‘self-evident’. In other settings, where the case is more complex,
unfamiliar or has confounding elements, as would be more common in a primary care setting, more
evident early hypothesis testing will be employed. Some support for this view is found in a study90
which deliberately placed confounding, conflicting statements within a case scenario. This appeared
to cause a degree of backward reasoning in expert clinicians who otherwise employed purely
forward reasoning to arrive at a diagnosis. Kushniruk92 lends further support, using a concept of
'small worlds' to describe how expert physicians consider small sets of closely related diagnostic
hypotheses. The study showed that non-experts faced with the same scenario, generate a larger
number of less 'connected' diagnoses, often from different disease categories. The problem space of
primary care physicians might be considered to be ‘less small worlds’ which would be consistent
with Barrows’ finding of increased early hypothesis generation in a primary-care setting.91
Crosskerry93 describes a similar mix of cognitive processes, within Emergency care departments,
referring to the pattern recognition of cue acquisition as ‘flesh and blood decision making’. Whilst
recognising the power of such an approach within the highly chaotic and time-pressured
environment of emergency medicine, he exhorts emergency physicians to aim for "cognitive de-
biasing" by being aware of the potential cognitive bias or "cognitive dispositions to respond" in a
rigid, and perhaps unhelpful, way.
30
Very similar themes have emerged from efforts to improve nursing education94. This was particularly
interesting as it coincided with attempts to realign nursing education to adopt a more biomedical
model away from what had been a traditional patient-orientated, care-based approach. Benner95, in
particular, argues for the importance of this traditional intuitive and hermeneutic approach to
problem-solving within nursing. From a cognitive perspective Ofreddy96used a ‘think-aloud’
approach to show that nursing practitioners employ very similar decision-making processes to
general practitioners when faced with the same clinical scenarios. The nurses appeared to select
significantly more cues from the scenarios compared to the GPs which slowed the nurses’ decision-
making process, without any increase in diagnostic accuracy. This was thought to be due to more
efficient organisation of the GPs knowledge into richer ‘chunks’ as a result of having more
appropriate explicit and tacit knowledge.
Summary
The clinical problem solver appears initially to use rapid pattern matching techniques to reduce the
problem space, founded on tacit knowledge and represented within memory as ‘schema’. This
results in the formation of one or more early hypotheses, which are then more formally assessed by
the acquisition of further data, a process known as ‘’backward reasoning’ and thence to final
diagnosis. In some circumstances where the problem space is already narrow e.g. within a speciality,
or where the pattern matching is highly effective, the initial hypothesis generation may be opaque
or hidden and the clinician appears to use ‘forward reasoning’ to progress from acquiring data to
final diagnosis. In these circumstances, ‘backward reasoning’ is only employed where confounding or
confusing data is encountered.
31
3. Clinical cognition and narrative in practice
Patel and colleagues97 have described ‘translational cognition’ as the applied research of human
information storage, processing and retrieval, as used to solve problems and make decisions within a
complex system. They draw attention to the importance of ‘distributed cognition’ where cognitive
activity is distributed across a number of minds external knowledge source. The contexts in which
this takes place is key, being heavily mediated by social, cultural and organisational factors and
consistent with a ‘socio-technical’98 view of the use of IT within clinical settings.
Applied clinical cognitive research is primarily focused on three areas:
1. Medical error
2. Clinical information recording and retrieval
3. Clinical decision support
Medical error
Kohn’s report ‘To Err is Human’99, highlighted the annual toll in the US of almost 100,000
preventable deaths due to medical error, and stimulated interest in the role of human cognitive
processing in relation to error100.
Zhang et al101 developed a cognitive taxonomy of medical errors, based on Reason’s differentiation,
of ‘slips’ due to the incorrect execution of an action sequence and ‘mistakes’ due to the correct
execution of an incorrect action sequence, extending this division to both the evaluation phase of a
medical encounter i.e. diagnosis as well as the execution phase i.e. treatment
32
From Zhang et al
Increased use of IT-mediated decision support is thought to be crucial to implementing safer
working practices via computerised order entry systems (CPOE) and drug prescribing alerting.
Although both have been shown to reduce medical error, CPOE systems can introduce their own
sources of error102 and prescribing alert systems are often poorly targeted, leading to ‘alert fatigue’
and automatic overriding of alerts by clinicians.103
In particular, Ash et al104, highlight two areas of potential failure in the process of entering and
retrieving information via CPOE systems:
1. A human-computer interface unsuited to a highly interruptive use-context, leading to
juxtaposition errors when the wrong option is selected on a screen
2. Cognitive overload due to an overemphasis on highly structured or over-detailed recording:
“Attempting to require professionals to encode data, or enter data in more structured
formats, can be fruitful and is necessary for research or managerial purposes but does not
come without a cost”.
33
Cognitive aspects of clinical information recording and retrieval
A great deal of research is focused on the cognitive aspects of data retrieval, in contrast very little
pertains to the cognitive processes and cost of data recording.
As an exception, the temporary installation and use of a diabetes clinical system (for training
purposes) afforded Patel105 an opportunity to analyse clinical recording styles in 3 phases:
1) written recording prior to system installation (pre-EHR)
2) electronic recording (EHR-phase)
3) written recording after electronic system was withdrawn (post-EHR)
The organisation style and content of the records was segmented using a propositional analysis
technique and the resultant concepts analysed for ‘criticality’ by an independent clinician.
EHR-phase records were found to contain slightly more information critical to the main diagnosis
then pre-EHR records and this was higher still in the post-EHR records, which, though manually
written, were considerably more structured than the pre-EHR equivalents. The key differences were
in increased recording and structuring of past medical history and lifestyle recording e.g. smoking,
alcohol status. One other key effect noted was a diminution of descriptions of time-course and
psychosocial information, during and after the EHR-phase, which would support Patel’s earlier
concern that use of computers tends to reinforce a ‘biomedical’ approach during the consultation.
These findings suggest that use of a computerised system has an ongoing effect on clinical cognition
processes both in the kinds of information sought and recorded, and in the degree and style of
structured recording, though it is not known if this effect is temporary. It has been previously
shown106 that semi-structured narrative appears to be optimum for human clinical retrieval
(especially for novices) in comparison to unstructured narrative or highly structured information.
Studies by Sharda107,108 used propositional analysis to assess the accuracy of comprehension and
degree of inferencing when clinicians read a psychiatric discharge summary. Narrative summaries
were contrasted with semi-structured summaries, and expert clinicians with novices. Although
differences were less clear with experts, the structured summary was shown to induce fewer errors
of comprehension and more successful inferencing when shown to novice clinicians. The study’s
design was based on a theory of natural language processing109 which draws a distinction between
the ‘text base’, a propositional representation of the recorded text itself, and the ‘situation model’
which extends the mental model by including prior knowledge and experience of the reader. This is
34
familiar colloquially as ‘reading between the lines’. The eventual conceptual representation is
derived from the combination of ‘base text’ and ‘situation model’, the latter being highly dependent
on the reader’s expertise and context. This may lead to errors of interpretation if the text is written
by a domain expert but read by a non-expert, owing to failure of the capacity of the reader to apply
an appropriate situation model through lack of expertise. The study attempted to define more
precisely those aspects of the discharge summary which should be made highly structured and
explicit as part of the ‘text base’ to avoid such errors.
Even in the heavily code-biased environment of UK general practice computing systems, it is well
recognised that there are significant barriers to the use of clinical codes, such as the Read code.110,111
De Luisignan112 conducted semi-structured interviews to help ascertain the difficulties perceived in
coding within UK general practice, contrasting the views of clinicians with their practice managers.
Perhaps unsurprisingly the managers perceived more technical barriers such as difficulty in making
the correct choice from picking lists, whilst for clinicians, the coding system was perceived as over-
imposing an inappropriate biomedical model and there were fears of inducing patient distress by the
use of stigmatising diagnostic labels such as 'Having marital problems'. There was particular difficulty
when attempting to record mental health and psychosocial issues or where the diagnosis was
unclear or ‘woolly’.
De Lusignan concludes:
“... coding systems that offer the wrong level of granularity may generate their own barriers to use.
Free text is a vital constituent of the computerised medical record and should not be excluded.”
In a later literature review113, he acknowledges the importance of narrative as being more engaging
and personal, but surprisingly makes no reference to studies of cognition and narrative in
determining ‘barriers to clinical coding’.
One of the difficulties recognised is the balance between granularity and accessibility. For example,
the Read code hierarchy for “Hip replacement” forces the clinician to choose between cemented
and non-cemented replacements, information that is largely irrelevant to the GP and may not be
readily determined from hospital communications.
Zhang114 touches upon the issue of granularity when examining the challenge of developing standard
controlled medical vocabularies from the perspective of basic cognitive research. He shows that the
design and structure of current terminologies may be mismatched with the cognitive structures and
processes of their clinical users. He cites the example of 'Basic level' concepts, part of the
35
Hierarchical model of second level cognition. Basic level concepts are intermediate in any hierarchy,
such as ‘cat’, being neither most abstract, such as ‘animal’, nor most specific, such as ‘Persian blue’.
They appear to be 'cognitively privileged' and fairly universal across a given knowledge domain.
Current terminologies such as Snomed CT do not recognise such a ‘privileged layer’ of hierarchy but
the archetype layer of openEHR115 might appear to fulfil many of the criteria for defining such 'basic
level concepts’.
Recent work by Chisolm116 which compares direct observation of emergency physician activities with
their related recordings in the ‘clinical charts’, demonstrates how clinicians continue to use the
patient record as a ‘cognitive artefact’ rather than as a simple, literal recording of activities. This has
been termed ‘writing as thinking’ and in an impassioned commentary on Chisolm’s research, “The
Chart is Dead—Long Live the Chart”, Wears117 points to the development of use of a ‘shadow chart’
in response to demands from healthcare funding organisations and medicolegal agencies for
increasingly structured, detailed and coded records which conflict with the need for cognitive
assistance with complex clinical work:
“What is useful for supporting distributed, cognitive work in the ED is too messy and situation-
specific for research or management, and what is useful for research and management is too “clean”
and unsuited for use in clinical work. The attempt to accomplish both goals in the same artefact has
ensured that neither can be done well."
These difficulties have prompted interest in using the advances in understanding of clinical cognition
to directly influence user interface and data entry design i.e. software development from a
cognitivist perspective, often in conjunction with emerging theories on ‘cognitive engineering’
principles, primarily Norman’s “Theory of Action”118 and associated evaluation techniques such as
‘cognitive walkthrough’. The think-aloud119 and propositional analysis techniques used to study
clinical cognition are proving helpful observational methods of assessing cognitive ‘fitness’ of user
interfaces.120,121,122 In one example123, the differing use of concepts and propositions by nurses and
doctors were analysed, when reading a series of mock gastro intestinal case reports. This reflects
both differing cognitive approaches and socio-technical demands of the two different professions
and highlights the difficulty of developing common applications to work in a shared information
space.
36
After Johnson123
, showing identification and use of concepts by nurses (RN) and doctors (MD) when reading
the same case reports
Developers are beginning to use these formal ‘cognitive engineering’ methods to inform user
interface design and assess the resultant application. Horsky124 describes a computerised provider
order entry (CPOE) evaluation using distributed resources task analysis which seeks to understand
“what information is required to carry out a task and where should it be located, as an interface
object or as something that is mentally represented to the user.”, in combination with an
observational ‘cognitive walkthrough’ which employed the talk-aloud method. Opportunities for
optimising data entry and potential for error or confusion were identified and noted to guide further
development.
37
Clinical cognition in decision support
Providing timely but appropriate computer-mediated decision support might be considered to be a
variation on the educational and management challenge of fusing tacit and explicit knowledge. In
the educational setting, the problem is normally the translation of tacit knowledge into external
form, to be consumed by domain novices. In the consultation setting, the challenge is effectively
reversed, where external knowledge must be integrated with the clinician’s tacit knowledge and
associated cognitive processes.125 Robinson126 describes this as a requirement to fuse three different
stories, that of the patient, that of the clinician using his tacit knowledge and the explicit knowledge
of medicine, encapsulated in textbooks, research papers, guidelines and terminologies. Using
qualitative interviews based on a consultation transcript he identified repeated examples of the
cognitive dissonance between the clinician’s usual practice and known but ignored evidence-based
guidelines. This has been a constant finding in other similar research and varied clinical settings.127,128
Inaccessibility of paper-based guidelines during consultations was originally thought to be a major
factor in preventing their uptake but whilst the difficulties of assembling, publishing and distributing
clinical guidelines have been largely solved by the use of electronic and web-based media, use of
guidelines within consultations remains problematic.
The most closely reported example is that of the UK-based, GP-targeted PRODIGY125 project which
had at its origins a set of prescribing support guidelines (Release One) but was rapidly expanded to
include support for chronic disease management (Release Two). Initial prospects for PRODIGY were
encouraging, feedback from GPs seemed generally positive, backing was secured from both
government and professional organisations and it was successfully implemented in all English GP
computer systems, closely integrated with the host application. Careful multi-modal evaluation, and
a rapid, iterative development cycle129 allowed responsive and effective adaptation of the system130.
After two pilot phases PRODIGY, as ‘Release One’, was adopted and implemented within the
majority of commercial GP systems and initial feedback from GPs seemed promising, e.g. from a 10%
sample of all English GPs, 84% reported that they would welcome computerised prescribing
guidelines. Video-taped analysis of PRODIGY-enabled consultations suggested that there was no
significant adverse change in doctor behaviour e.g. doctor-patient eye contact remained high and it
was felt that PRODIGY had been well integrated into the host systems and GPs’ consultation
processes129. Off-line assessments of performance suggested that PRODIGY could actually speed up
38
the prescribing process but only 13% of encounters resulted in ‘completion’ of the guideline. In spite
of the seemingly successful roll-out of Release One, there remained disagreement as to whether the
degree of usage reported merited such positive analysis and subsequent action131,132.
PRODIGY (pilot) Phase Three attempted to address earlier criticisms that it was less well adapted to
chronic disease scenarios and adopted a considerably revised internal representation of the
guidelines, using the emerging ontology approach and related tool (Protege)133,134. This, it was
hoped, would result in a much richer and adaptable electronic guideline, capable of smoother
integration with the host clinical system and clinician workflow e.g. it allowed a degree of state-
management where the patient’s status within the guideline pathway was persisted and could be
recalled.
In a “before and after comparison” of PRODIGY Phase Three135 supporting angina and asthma
related encounters in 60 UK general practices, Eccles et al found no evidence of any effect on
consultation rates, process of care measures, including prescribing and investigations, or patient
reported outcomes. It was reported that use of the PRODIGY software was low and there was a
discernible drop-off in usage over the 12 month period of PRODIGY use. This paper provoked an
interesting response where a number of differing views were expressed and the authors robustly
defended their findings136. A significant contribution to these comments from a Phase One and Two
team member and evaluator confirmed anxieties about real-world low usage of PRODIGY, concerns
subsequently confirmed by the public release of a previously internal team evaluation, based on
PRODIGY log file data137. This report showed that, whilst usage per consultation was over 50% in a
few practices, the overall average usage of PRODIGY per consultation reached only 10.2%.
These disappointing findings were, and are, not unique to PRODIGY. A Norwegian study138,
investigating outcome improvements after implementation of a CDSS supporting hypertension
management similarly failed to find any effect. As for PRODIGY, actual usage was low with only 1 in 8
appropriate consultations making use of guideline support. Similarly, a study of a cardiac failure
CDSS in US primary care139 failed to show any effect over a 12 month period and the authors suggest
looking at the possibility of enforcing completion of the guideline rather than ‘pressing escape’ or by
using financial inducements.
39
Interestingly the recent introduction of such financial inducements via target payments for NHS GPs,
for some of the same hypertension indicators, has resulted in a significant improvement in
outcome140, although this may partly reflect previous under-reporting.
Some insight into these difficulties is offered by Gabbay and leMay9 who used an ethnographic
approach to study how primary care clinicians use and manage external knowledge sources within
their normal working practice. Their key finding was that the clinicians only rarely directly used
external research-based sources within patient encounters, preferring to rely on ‘mindlines’
described as “collectively reinforced, internalised, tacit guidelines”. These are informed by personal
reading, the experience of personal and colleagues’ practice, local specialist opinion and other tacit
knowledge sources, finally mediated by organisational and resource constraints. Such ‘communities
of practice’ are thought to construct local, socially constructed “knowledge in practice”. A
comment141 on Gabbay’s paper points to the relevance of understanding complex, non-linear
systems in this context and the ‘mindlines’ described by a startling resemblance to Marshall’s
schema and the remodelling of clinician internal knowledge structures noted by Patel, as clinicians
develop domain expertise.
One possible conclusion of Gabbay’s findings is that more needs to be done to integrate the
implementation of CDSS with professional development, perhaps by allowing guideline overrides to
be logged and used to stimulate and inform ongoing learning. A similar approach has already been
suggested by Robinson142, as part of a review of PRODIGY:
“Learning can take place outside a consultation, within a community of practice, as long as the story
is simulated and related to the consultation. In this situation the patient is represented by an instance
(case memory), and the evidence by an illness story. The learning consists of bringing the two stories
together.”
Almost all current decision support tools are primarily rule-based, probabilistic approaches having
largely fallen out of favour but two differing approaches merit discussion.
A number of clinical decision aids have been built around the PROforma143 guideline modelling
language, including CAPSULE144, a prescribing support tool, akin to PRODIGY Release One and a
breast cancer genetic risk tool145. CAPSULE was received well in a simulation setting but does not
appear to have been subjected to the real-world evaluation which PRODIGY had to endure and does
40
not seem to be used in current GP systems. PROForma is unusual in that it incorporates the notion
of ‘argumentation’146 described as “a general method for assessing the strengths and weaknesses of
alternative solutions to a clinical problem” and which has been suggested as a means of reasoning in
uncertain situations of real-world complexity, not generally well-served by traditional probalisitic
approaches147.
At least for the moment, the conclusion of Eccles et al135 in 2002 seems to remain valid:
“Even if the technical problems of producing a system that fully supports the management of chronic
disease were solved, there remains the challenge of integrating the systems into clinical encounters
where busy practitioners manage patients with complex, multiple conditions.”
Narrative approaches in existing clinical systems
A number of applications and interfaces have been developed which attempt to bridge the gap
between narrative and structure, or at least to enable structured forms of data entry which are more
flexible than rigid forms-based data entry.
The current UK market-leading GP system, EMIS allows the entry of mixed free text and coded items,
employing a degree of text processing where the system offers to encode text as it is entered but
based on known patterns of user-specific data-entry. Similar mechanisms are being investigated in a
variety of academic and commercial research environments such as the NHS/Microsoft CUI148
project, particularly as an adjunct to Snomed-CT encoding.
Pure natural language processing systems10,149,150 (NLP) are particularly applicable to highly
constrained reporting environments such as radiology, where such products are now commonplace.
They have been much less successful in general clinical practice because the domain terminology is
much larger and more complex, whilst and the interactivity required with patients and other staff
places significant demands on the acquisition of good quality auditory input.
The Pen&Pad46 project is of considerable significance, having been developed to take advantage of a
large-scale academic clinical terminology development known as Galen, and whose design
significantly influenced the development of SNOMED-CT and other modern ontologies. The Pen&Pad
user interface allowed users to first select a presenting complaint via a 'Topic Selector' based
primarily on body location. This would then present a list of appropriate terms and phrases which on
41
selection led to an appropriate structured data entry form, which allowed the mouse-click selection
of clinically sensible items based on the Galen inferencing, represented both as a set of Galen terms
and as a narrative-style clinical record. Develop in the mid-1990s, Pen&Pad was designed to
integrate with existing GP systems and was made commercially available but never achieved
popularity. The reasons for this are not documented but personal communication suggests that the
computing requirements both of the user-interface elements and the underlying Galen terminology
server, outstripped the capacity of available technology at that time. In effect, Galen and Pen&Pad
were ahead of their time.
A slightly different approach has been taken by the openSDE151-153 project based primarily in the
Netherlands, which allows highly complex tree-structured data-entry protocols to be authored.
Whilst this is somewhat more flexible then most forms-based data-entry mechanisms it is not truly
narrative-based, although the final structure can be exported to a word processor for final editing
prior to committal of the record.
Various attempts have also been made to integrate pen/stylus and handwriting technologies to
afford a more naturalistic interface, notably the Pen-Ivory154 project. The current Microsoft CUI
Clinical tablet technology155 also makes use of this approach. Reports from early adopters are
promising but no large-scale evaluations have yet taken place but in general hand-writing
recognition has failed to achieve the acceptance once anticipated and in general computing, through
novel devices like the Apple iPhone, the keyboard maintains its dominance as the prime input
device.
Very recently there has been direct interest in 'Structured narrative'156 as a prime data
representation, arising primarily from the use of the Narrative Block capability of the HL7 CDA
document framework. Hyun157 explores these ideas further within the context of a prototype
clinical system for oncology nurses and found high levels of acceptability, the user interface's
capacity to mix structure and free text being regarded as well-matched to existing nursing
documentation requirements.
42
4. Clinical record architectures
In supporting narrative, one the aims was to develop a vendor-neutral and, as far as possible,
interoperable representation of structured narrative, using open standards. The promotion of
narrative, however, should not diminish the role and value of formal, computable representations of
health information. Human beings can mostly accommodate the vagaries and ambiguity of natural
language, but even when narrative is interpreted accurately by NLP (Natural Language Processing),
the coded terms offered will often fail to be sufficiently precise to compute reliably10,149, other than
in tightly controlled domains such as radiological reporting. In addition, the structuring or coding of
clinical entries can be used to enforce particular standards of information recording, whether for
patient safety or operational reasons.
In comparison to other sectors such financial service or other comparable knowledge-driven
industries, the development of interoperability in healthcare has seemed very slow. The SIOp158
report gives an authoritative and realistic overview of the current state of development of
healthcare interoperability and outlines a practical roadmap for ‘next steps’ in the coming years. It
points to the complexity of healthcare in comparison to other sectors and concludes, that at present,
none of the candidate technologies can, by themselves, offer a complete solution.
At present these technologies can be broken down into three groups:
1. Terminologies, such as SNOMED-CT, LOINC and ICD-9/10.
2. Information models, such as HL7-V3, openEHR and EN13606.
3. Inference models for decision support such as PRODIGY and PROForma.
Alan Rector’s group at the University of Manchester have played a significant role in developing a
basic understanding of these concepts, defining a general approach to modelling of the clinical
record159. Rector describes an Information Models as ‘the structure of the information to be stored’,
a Terminology Model as the ‘meaning of what is stored’, whilst an Inference Model holds ‘the
consequences and actions which follow from what is stored’.160
The most extreme proponents of each approach, and indeed those of each individual model,
continue to seek a single unified solution, but most authorities now recognise that successful
interoperability will depend on an appropriate mixture of technologies and models. NHS England is
now exploring such an approach via its Logical Record Architecture161 project which attempts to
blend SNOMED-CT as a terminology model; aspects of openEHR and EN13606 as an information
43
model for clinical content; and HL7-V3 as an information model for clinical messaging, whilst Sweden
intends to use SNOMED-CT and a mixture of openEHR and EN13606 (which are broadly compatible)
to define clinical content and messaging structure162.
The Terminology model (SNOMED-CT)
One of the key features of UK primary care computing is the prominent use of coding systems163,164,
primarily the Read Code165, This is a hierarchical, single-axis clinical coding system, developed
originally by James Read for the Abies GP computer system, but subsequently adopted as a UK
national standard166. It was recognised that the original Read code, with its primary-care focus,
would be inadequate for secondary-care use and work started on the UK 'Clinical Terms' project to
expand its scope accordingly. Recognising that such a huge task required international collaboration,
this was eventually merged with the SNOMED terminology maintained by the College of American
Pathologists, to become SNOMED-CT.
SNOMED-CT is now developed under the aegis of IHTSDO48, International Health Terminology
Standards Development Organisation, a not-for-profit organisation with nine charter member
countries. Whilst a number of other international coding systems and terminologies exist, such as
LOINC167 for laboratory testing and the ICD series (IDC-9, ICD-10) for secondary-uses analysis, there is
widespread expectation that SNOMED-CT will replace, or at least subsume their role, over time.
SNOMED-CT is unique, outside the informatics research community, in meeting some of the
requirements for a healthcare terminology as outlined by James Cimino in his 1998 paper
‘Desiderata for controlled medical vocabularies in the twenty-first century’. It fulfils the role of
‘Model of Meaning’ which Rector ascribes to a terminology i.e. it is primarily a model of the real
biomedical world, attempting to fully describe the relationships between each concept as a series of
“is_a” and “has_a” relationships.
For example;
Blood pressure “is_a” vascular pressure, and “has_a” child concept “Systolic blood pressure”.
One of the challenges of previous coding systems like Read was the ‘combinatorial explosion’168,
where a large number of individual terms had to be created to cope with subtle variations in clinical
circumstances; e.g. Multiple, single codes to cope with every variation of hip replacement, including
laterality, surgical approach, exact procedure, replacement device etc. This is known as ‘pre-
44
coordination’. In contrast SNOMED-CT allows ‘post-coordination’ where a set of more atomic terms
can be bound together to form a single concept from equivalent to ‘Left total hip replacement’.
Whilst this is an elegant solution to the problem faced by pre-coordinated terminologies, ‘Left Total
Hip Replacement’, using SNOMED-CT Compositional Grammar169 is expressed internally as:
243796009|situation with explicit context|: {363589002|associated procedure| =(397956004|prosthetic
arthroplasty of the hip| :363699004|direct device|=304120007|total hip replacement prosthesis|
{363704007|procedure site| =(24136001|hip joint structure|:272741003|laterality|=7771000|left|)
,260686004|method|=257867005|insertion - action|}) ,408730004|procedure context|=385658003|done|
,408731000|temporal context|=410512000|current or specified| ,408732007|subject relationship
context|=410604004|subject of record| }
Even for a relatively simple statement like ‘Left Total Hip Replacement’, it is evident that ‘post-
coordination’ will be a challenge for users170 and that system vendors will have to isolate users from
this complexity.
Much of the additional complexity in the example above arises because SNOMED is modelling
aspects of the ‘context’ of the statement, as evidenced by the elements related to the ‘subject of the
record’ and ‘temporal context’. Rector describes the Information Model as the ‘Model of Use’171,
and it could be argued that this is the more appropriate location to store contextual data though
Cimino argued for a terminological approach in the ‘Desiderata’172. This debate over the boundary
between context and content has been very longstanding173 and largely remains unresolved, in spite
of projects like HL7 Terminfo174, which offer useful pragmatic guidance.
The Clinical Document Information model (HL7-CDA)
Health Level 7 (HL7)175 is a multi-national not-for-profit organisation whose key aim is the
development of interoperability solutions, primarily via the definition of structured messages, with
the vision:
The HL7 Clinical Document Architecture (CDA) grew, initially independently, out of the work of
Lincoln and Esson49 based on a forerunner of XML, SGML. They held the view that considerable
benefit could be accrued by adopting a structured document-orientated approach to recording and
transmitting clinical records, without the necessity to define the contents of the document at the
level of finely-grained semantics. A document-orientated paradigm also has the advantage of being
45
well aligned to pre-existing manual clinical records and documents, such as the encounter, referral
or discharge letter. This approach also acknowledges the importance of basic contextual information
such as the origin and authoring of the document, in the interpretation of a clinical record. This
approach be regarded as being philosophically in tune with Coiera, whose paper, 'When
Conversation is better than Computation'58, holds that the potential health benefits of simple IT-
mediated communications have often been overlooked in the quest for the ‘’Holy Grail’ of
computable semantic interoperability.
Early developments of CDA were integrated into HL7, using HL7v3 to define content and realised as
CDA 1.0176 in 2000, with a subsequent expanded release CDA 2.050 in 2005.
CDA 2.0 currently offers developers three levels of complexity of which only Level 3 is of interest to
this project, supporting semantic interoperability by the addition of structured 'Clinical Statements'
into each CDA Section, alongside the ‘narrative block’ or attachment.
Each Level 3 Section contains an obligatory text or 'narrative block' which may also be accompanied
by any number of Clinical Statements, each of which contains semantically interoperable data, often
coded, for example using SNOMED or LOINC terms. A constraint methodology, known as 'HL7
Templates' may be used to define reusable patterns of Clinical Statement such as 'Family History',
'Specimen' or ‘Glucose Tolerance Test’
The Clinical Statement model allows for a number of relationships to be asserted between
Statements and with the 'Narrative block'. The ability to link structured Clinical Statements and
related narrative is of particular interest to this project.
46
The Narrative Block
The Level 3 narrative block allows its text to be 'marked-up' in a fashion similar to HTML, using
simple formatting commands such as Bold, Italic, numbering etc. This is very similar to XHTML and it
is likely that future versions of CDA will support XHTML. Of more interest is the markup which
potentially links text with individual Clinical Statements. The relevance to this project should appear
evident, as it provides a mechanism to smoothly integrate a body of text with structured content,
whilst maintaining a sense of narrative. Termed 'structured narrative' this has been described by
several investigators, particularly Myestre177 and Johnson156.
Structured content in CDA documents
The development of HL7v2 messages, was quickly recognised as being uncoordinated and
unplanned, resulting in multiple local representations of the same messaging requirement. HL7v3,
developed from 2000 onwards was conceived as addressing this concern, with a better defined
model development framework and an based on a new object-orientated modelling paradigm, the
Reference Information Model (RIM)178 from which all V3 outputs such as message constructs are
derived, through a mixture of intermediate 'constraints’ on the RIM.
47
HL7-RIM class diagram
All HL7-v3 artefacts must be derived from the RIM. The whole of a CDA message, including the
Clinical Statements, and other than non-XML ‘blobs’, is therefore expressed as a set of RIM-derived
objects. Most current CDA implementation use the HL7 Template constraint mechanism to define
their structured content. In some respects these are similar to openEHR archetypes but in contrast
they retain a high degree of technical complexity and comparative paucity of clinical content.
As with all potential solutions, HL7-v3 has suffered from criticism, particularly from an ontological
and philosophical perspective, especially by Smith and Ceusters179 , who regard it as ‘semantically
incoherent’. Schadow180 rebuts this criticism, correctly defending HL7-v3 as an information model
which must reflect the complexities of clinical usage, rather than a pure ‘model of meaning’.
A more practical consideration is the level of abstraction of the core HL7-v3 classes, and their basis
in messaging and events, rather than as direct representations of aspects of a clinical record. This
leads to almost all direct clinical concepts being modelled from a high level of abstraction, and the
layers of sub-classing and control terminology required to model say ‘blood pressure’ leads to highly
technically complex entities. A variety of constraint mechanisms such as RMIMs, CMETS and
Templates have, to date, failed to achieve the level of clinical directness and usability afforded by
openEHR archetypes181 and other related detailed content modelling approaches182.
There is widespread international enthusiasm for CDA Levels 1 and 2, offering as they do, graduated
entry to interoperability and the ‘Narrative block’ is a concept which has very direct relevance to this
project but, in the context of CDA Level 3, there are increasing concerns that HL7v3, as it currently
48
stands, is too difficult and complex to support the level of direct clinical modelling input which will
be required to achieve semantic interoperability161.
The EHR Information model (openEHR/CEN13606)
The openEHR Foundation183 is a not-for-profit company, established in 2003, the founder members
being Ocean Informatics, Australia and University College London. The aim of the foundation is to
produce open, non-proprietary specifications to enable the international collaborative development
of life-long personal health records. The goal is to improve interoperability within and between
systems and reduce in the costly and time-consuming process of embedding clinical knowledge
within systems. The principal deliverable is a fully specified information model for an electronic
heath record based on a “two-layer approach” 184, consistent with modern service orientated
architecture paradigms185. This is coupled with projects to store EHR related clinical knowledge
within a repository of standard “archetypes” 115, the development of tools to support archetype
authoring and governance, and the development of openEHR runtime ‘kernels’ as implementations
of the full specification, allowing clinical applications to manipulate and persist openEHR compliant
data structures 186
openEHR takes an agnostic approach to proprietary use, so that developers are free to use the
specifications within commercial products or via Open Source licensed applications. There are
currently 2 main strands of development, a commercial closed source Microsoft .NET strand
developed by Ocean Informatics and a parallel Open Source JAVA-based development by the
Swedish CAMBIO company. Other open source implementations are being undertaken in Ruby and
Python.
openEHR grew out of a number of prior attempts to develop a standardised unifying information
model for the EHR, such as GEHR187 and the European CEN 13606 standards work.
Elements of openEHR, , have in turn been formally included within the European CEN 13606
standard, in particular archetypes, which are adopted as Part II of CEN13606. International standards
are often criticised for slow pace of development and being divorced from the requirements of
implementation. openEHR differs in its approach by functioning more like an open source software
development project with a small Technical board which reviews change requests and problem
reports, rather than the traditional standards organisation balloting/consensus votes approach. In
49
this manner it hopes to remain both more responsive to user requests for change, and that such
change remains implementable in a timely fashion.
openEHR ‘Three-layer’ modelling
Whilst most openEHR related literature describes it has having a two-layer technical/clinical
approach, it is increasingly recognised that the further division of the clinical layer into a ‘maximal
dataset’ layer and a ‘clinical use-case’ layer will prove particularly powerful both as a technical
approach and as a socio-technical driver of enforceable but organically emergent, interoperability
standards.
50
Technical layer “Reference model”
As with HL7v3, all openEHR artefacts and models are built around a low-level Reference layer. This
defines a number of basic datatypes such as TEXT, CODED_TEXT, QUANTITY, DURATION, low-level
structures such as TREE and TABLE, generic containers such as COMPOSITION and FOLDER and the
key ENTRY type which holds clinical statements, itself containing one or more datatypes or
structures. Almost none of the data types and structures are recognisable as clinical entities, the
only exception being the sub-classing of ENTRY into OBSERVATION, EVALUATION, INSTRUCTION and
ACTION. These reflect a simple ontology based around the notion of a cyclical “Clinical Investigator”
process.
A key feature of the openEHR reference layer is that is almost completely devoid of clinical content-
which is left to the Archetype layer.
UML Diagram of the openEHR Entry class
Maximal clinical dataset layer “Archetype Model”
An openEHR Archetype can be defined as a re-useable, formal model of a clinical concept,
represented not by another layer of a class model, but authored in a constraint language, which
51
‘constrains’ the one of the underlying classes to fit the clinical concept. An archetype represents a
well-recognised clinical concept such as Blood Pressure, Diagnosis, Apgar Score, discharge letter and
using a ‘maximal dataset approach’ attempts to include a description of all the information elements
that a broad, inclusive set of clinicians might wish to record about that concept. All such information
elements must belong to the openEHR reference layer.
The prime motivation for this approach is to isolate the technical requirement to faithfully persist
and retrieve complex, structured data, from the continual clinical demand for new and changing
content to reflect changed clinical practice or local innovation. The technical contract is simply to
optimise the persistence and retrieval of highly static, reference layer objects and is agnostic to the
clinical content expressed within.
The design principle behind each archetype is that if an attribute belongs within the broad scope of
the concept being modelled, it should be included.
The maximal dataset has an important benefit in minimising dispute between clinicians who may
wish to express a particular concept with differing visibility of attribute e.g. within a blood pressure
archetype, a GP will rarely if ever require a Tilt measurement, which is however a valid attribute in a
secondary care setting. These different groups can use ‘templates’ to ‘constrain out’, unwanted data
elements from their particular dataset definitions.
52
Minimal Use-case dataset layer “Template Model”
The openEHR Template layer provides the final level of constraint for actual use-cases such as data-
entry templates, frameworks for messaging or traditional definitions of minimal datasets for analysis
purposes. An openEHR Template consists only of archetypes, constraining out elements of
archetypes, such as Blood pressure-Tilt, which are out of scope for the current use-case and
aggregating different archetypes to form the basis for a data-entry definition or message model e.g.
ENT Discharge, Diabetes review data-entry screen.188
Where an Archetype hopes to express the general, even universal, the Template constrains this to
local and specific. The openEHR Template model is still in development and due for publication in
2009, but practical experience has been gained with some commercial templating tools which have
been fed into the final specification189,190
A candidate architecture for the current project
Investigation suggests that the openEHR architecture offers a suitable basis on which to model both
the constructs of ‘clinical narrative’ and ‘clinical scenario templates’. Designed specifically to model
the contents of an electronic health record openEHR appears better aligned to the objectives of the
study than, for instance, HL7v3, whose original aim was to support message construction and whose
paradigm views the heath domain primarily as a series of actions (Acts).
The OpenEHR Reference model supports a broad range of data types, including simple, coded and
formatted text, whilst the archetype layer has the capacity to support a wide range of structured
clinical concepts, including a ‘Parseable’ datatype, which can represent the marked-up narrative
which will be required. In some respects a Clinical Scenario resembles an openEHR template but
where the data-entry elements are only very loosely coupled to the user-interface.
There are potential drawbacks. In particular, the openEHR approach tends to favour highly
structured records and may be found deficient in a more narrative-based environment. In contrast,
HL7-CDA Level 3 directly supports structured narrative in its Level 3 model156, a feature which is
lacking in the openEHR specifications.
In spite of this advantage, the principle argument against the use of HL7-V3 (or CDA) is that, to date,
detailed clinical content modelling has proven difficult, time-consuming and generally opaque to
53
clinicians. In contrast, the openEHR archetype and template approach appears to promise a truly
workable and scalable approach to this previously difficult problem191,192.
This approach does not preclude the transformation of openEHR defined content, into CDA for
messaging purposes and there is currently no practical alternative to SNOMED-CT as a general
reference terminology, which will be required to enable inferencing, and indeed, just to populate
more clinically generic openEHR concepts such as ‘Diagnosis’ or ‘Procedure’ with use-case
appropriate terms.
54
III Methods / Analysis
The practical aspects of this project were conducted within the setting of a real-world application re-
development. GPASS193 (General Practice Administrative System for Scotland) is an application
currently wholly developed, supported and funded by NHS Scotland for the use of Scottish general
practices. Originally developed in the early 1980’s and comparable to the UK commercial GP clinical
systems, by 2001 it was recognised as functionally having fallen behind those systems, particularly
use during consultations194.
In common with other UK GP systems, GPASS had adopted a primarily Read code-based clinical
recording paradigm and whilst this did support narrative, it had remained somewhat disjointed and
restrictive, in comparison to rival systems, which had been gradually improved over time.
A commercial company was contracted to develop a new version of the application to be known as
‘GPASS Clinical’, taking a fresh look at the user-interface with regard to usability and functionality in
the consulting room.
1. Focus group requirements development
Methods
The development methodology adopted can be best characterised as rapid, iterative prototyping or
‘Rapid Application Development’195, where, in contrast to more formal methodologies such as the
‘Waterfall method’196, user requirements are gathered through the rapid production of a series of
iteratively refined prototype applications or dummy screens.
As part of this process a design focus group (known as the F3 group) was formed from ten existing
GPASS users with support from the project Clinical Director, himself a former GPASS user and a
health informatician (author) to advise the developers on possible approaches to improving the
consultation experience.. Although the composition of the active group altered, there were normally
at least 5 GP users available at each session. The group met approximately monthly but considerable
interim dialogue occurred between group members via email, including the exchange and
55
refinement of candidate screen designs. Focus groups are a well established method of acquiring
and developing software user requirements197, and are particularly helpful in early stages, when
broad parameters need to be established.
The F3 focus group had a wide remit to look at many aspects of user interface design including
easier data-entry, improved adverse reaction recording and support for problem-orientated
summaries but two initial themes emerged; the need for a unified data-entry screen and better
support for narrative.
One of the drawbacks of the existing GPASS system was that it artificially partitioned related
functionality into different sections of the application e.g. recording clinical notes and coding
diagnoses, ordering prescriptions were accessed via separate tabs, requiring users to switch
between them during a consultation. Whilst this arrangement was certainly optimal for viewing the
information, it was cumbersome for data-entry purposes.
“Basing consultation data-entry on a single screen will prevent me having to ‘jump-about’
between screens to perform common, related clinical tasks such as entering Read codes and
prescribing”.
“We need to incorporate as much consultation-related functionality into a single view as
possible – referrals, prescribing, coding and patient leaflets”
The ability to have freedom to record the consultation in a narrative manner also seemed important:
“Although I understand the need to code some parts of the consultation, I find it generally
much easier to record most of the consultation as free text”.
“Much of what I want to record at a consultation does not lend itself to coding”
Some concern was expressed that the use of narrative would impose the need for keyboarding skills
that might be lacking in some users.
“Although I prefer to use the keyboard, rather than a mouse, some of my partners find using
the keyboard very difficult”
Whilst users appreciated the value of data-entry assistance, there was considerable resistance to
forms-based data-entry and formulaic decision support mechanisms. Although none of the GPASS
users had experienced Prodigy, the difficulties encountered with usability were well known. GPASS
56
did have a user-customisable forms ‘engine’ called ‘SPICE screens’, but this was not felt to be
appropriate to generic consultation use.
“The SPICE screens are great for doing chronic disease clinics such as diabetes but they are
just not flexible enough for normal consultations, even when these actually feature a chronic
disease.”
In response to the suggestions and comments made at initial F3 group meetings, a prototype of the
Clinical Workspace approach was introduced with an explanation of the underlying philosophy.
57
The Clinical Workspace prototype
It was explained that the Clinical Workspace interface was intended to act as a hub for all common
consultation-associated activity and comprised two principal elements:
1. The Clinical Narrative pane (CN) , offers a simple text entry area with a word-processor like
paradigm. A set of menu commands and hotkeys allow the insertion of marked-up structured
data within the narrative.
2. The Clinical Scenario Template (CST) pane displays a set of pre-defined data terms, structures
and activities which can be used to populate the Clinical Narrative.
Each CST would reflect the likely content and clinical activity associated with a single commonly
identified scenario from UK general practice such as ‘low back pain’, ‘chest infection’, ‘fever in a
child’ etc. The template would be composed of a range of elements from the following categories:
Default text: plain text terms or phrases commonly used in this scenario but which do not require
coding. Negated terms could be constructed.
Coded text: plain text terms or phrases commonly used in the scenario but which are linked to a
term from a reference terminology, in this case READ codes. ‘Diagnosis’ and ‘Reason for Encounter’
are examples of commonly coded entries. Negated terms could be constructed and linked to an
appropriate negated code.
Structured entries: plain text terms linked to pieces of structured data entry e.g. Blood pressure
which requires a systolic, diastolic values and possibly cuff size.
Prescriptions: pre-formatted drug prescription entries, appropriate to the scenario
Referral links: pre-formatted links to the Scottish SCI-Gateway protocol-based electronic referrals
system, or to other referral pathway mechanisms.
Patient information links: direct access to appropriate patient information for the active scenario.
‘Red-flag’ markers: Any of the above elements could be additionally marked as ‘red-flags’ to remind
the clinician of a particularly important data element or activity.
58
Screenshot of prototype Clinical Workspace
Markup mechanism: Although the final internal format of markup had not been decided, a working
draft was developed based on XHTML with internal UUIDs linking the structured data entries – see
‘Read codeid’ and Drug Id’ below.)
<Consultation>
<PlainText>3 week history of dysuria and frequency. Nil on examination</PlainText>
<ReadCode id = ‘345672’ code= ‘XYZr1’><b>Urinary Tract infection</b></Readcode>
<Drug id =’00123245’>Trimethoprim 200mg bd * 6</Drug>
<PlainText>Review in 1 week</PlainText>
</Consultation>
This would be displayed to the user as:
Clinical Narrative pane
Clinical Scenario pane
Marked up prescription
Triggers ultrasound request
59
3 week history of dysuria and frequency. Nil on examination
Urinary Tract infection
Trimethoprim 200mg bd * 6
Review in 1 week
Clicking on hyperlinked text would allow the user to edit the item or see the full detail of the
structured entry, activating a conventional dialog box.
Template to narrative mechanism: At the start of a consultation, as soon as the clinician had
identified a potential scenario, this would be selected from a searchable drop-down combo control.
The clinician would either type directly into the Clinical Narrative pane, or select elements or
activities from the CST pane (via click or drag-drop) to populate the consultation record. At any time,
the active CST could be switched if, as the consultation progressed, a more appropriate scenario was
identified or could be ignored completely.
In some cases, such as prescription entries, a standard prescribing dialog box would be displayed to
enforce strict, structured entry and allow computerised prescribing safety alerting to take place.
Template Authoring: The original intention was that while a number of CSTs would be supplied as
standard, it should be possible for clinicians to personalise these standard templates and to author
and share their own personal CSTs. The internal structures of the Clinical Narrative and the CSTs
should be aligned as far as possible to allow existing exemplar consultations to be marked up and
saved as a CST in a similar fashion to the way word-processor template file such as MS Word is
generally derived from a working document.
Results
The philosophy and functionality of the Clinical Workspace was discussed at subsequent F3 group
meetings and were generally felt to be compliant with earlier expressed requirements:
“This seems to give a nice balance between being able to use free text but code and prescribe
where needed without having to move between multiple screens”.
“It fits much better with the way I work when I am consulting”
60
The supportive and pre-emptive nature of the template mechanism, in terms of patient safety and
quality management was also appreciated:
“I often have a very good idea of the nature and content of a consultation, soon after the
patient walks in”
“The Red flag items will help me to remember to ask specific questions. I often seem to have
a memory block about particular important questions”
“The templates will make it easier for my partners to enter data quickly and correctly”
“This will make it easy for clinicians to do the right thing” – Clinical Director
However, there were anxieties around the original intention that CSTs should be highly personalised,
based on an individual’s pattern of recording and care-giving:
1. The developers expressed anxieties about being able to implement personalised CSTs within
the available timeframe.
2. The GP users were concerned that the extra workload involved for individual clinicians might
lead to poor uptake amongst less motivated partners.
3. The Clinical Director, representing the principal stakeholder (NHS Scotland) expressed
concern at potential medico-legal implications if individual clinicians failed to adhere to best
evidence-backed practice when self-constructing CSTs , simply reinforcing poor or outmoded
practice.
Analysis
The focus group identified 6 particular requirements:
1. The new interface had to better support clinical narrative if it was to be used more regularly
whilst consulting.
2. The new interface had to capable of allowing a range of consulting-related activities such as
problem coding, prescribing, patient leaflet printing, lab test ordering, and referring, which
61
though previously available, required the user to work across a number of screens and
menus.
3. The level of typing and keyboarding skills in the user community remained variable. The new
interface would have to incorporate as much point-and-click functionality as possible, along
with features such as predictive text, to reduce the burden of typing and keyboarding.
4. Whilst there was general resistance to a decision–support driven interface, such as that
offered by Prodigy, there was a recognition that the system and interface should help users
‘Do the right thing’.
5. The group suggested that advantage could be taken of the repetitive nature of many general
practice consultations, a relatively small number of scenarios making up the vast majority of
encounters. An experienced clinician could often recognise such a common scenario at a
very early stage of the consultation and there was a shared recognition that whilst the terms
used to record the consultation and any actions taken or resources accessed were
somewhat individual to each clinician, they tended to use the same terms, resource and
actions for every similar consultation. Whilst this conformity might seem to lend itself to a
forms-based approach, it was felt that this was too restrictive as although many
consultations may start with a recognised scenario, it is sufficiently common that a
consultation may ‘veer off’ into an alternative scenario or even necessitate ‘free-form’ entry.
6. There were mixed views about the degree to which Clinical Scenario templates should be
locally-authored i.e. by a practice or individual clinician. Some of the group felt that whilst, in
principle, self-authoring would be an ideal approach, that realistically, very few GPs would
have the technical skills or time to do this properly. There were also concerns that poor
authoring might simply perpetuate out-moded or even clinically unsafe practice. Others felt
that it might be possible to use the same ‘markup’ approach to template authoring as for
the narrative itself, and that as such it would be relatively easy for clinicians to create
templates from copies of their own ‘perfect consultations’ where derived from real
consultations or de-novo idealised consultation recordings. It might also be possible to use
previous consultations recorded using marked up narrative, as ‘ad-hoc’ templates in
subsequent consultations.
As a result of the focus group discussions, prior to active development, a number of significant
changes were made from the Clinical Workspace prototype.
62
The original drag and drop mechanism was abandoned in favour of a set of tabbed
checkboxes, as this had already been implemented as part of the commercial product upon
which GPASS Clinical was being developed.
Self-authored templates were also deferred, partly because of technical complexities and
user-training issues but also because of concerns around patient safety. Instead, members of
the focus group constructed 15 CSTs to be released as a standard set, drawing both from
personal expertise and from the material still available from the PRODIGY project.
After some months of development, an interim evaluation of the developing Clinical Workspace was
performed, intended both as part of this research project and to inform further refinement of the
implementation itself.
63
2. Evaluation of an early Clinical Workspace implementation
Methods
A variety of user-interface evaluation techniques were considered, including user feedback
questionnaires 198and other generic evaluations of usability such as Neilsen’s Heuristic Evaluation199
but were rejected in favour of the Think-aloud119 methodology, now established as a powerful user-
interface evaluation tool, and which has the potential to elucidate cognitive aspects of human-
computer interaction of central interest to this research project.
Think-aloud is a technique which requires subjects to talk aloud while solving a particular problem or
task and arose from basic cognitive and educational research but is increasingly established as a
useful tool to guide user-interface development, giving direct insight into the thought process of the
test subject.
It is generally used in conjunction with multi-modal recording techniques, themselves proving useful
in UI analysis, even in the absence of Think-aloud techniques.200,201
Think aloud consists of two elements. In the first, the subject is required to perform a task, typically
a user-interface mediated function and is asked to continually report their direct thoughts as they
tackle the problem. This normally requires a degree of prompting by an observer to the subject to
“…keep talking…”.
In the second element, the data captured by audio recording and any associated media such as
screen or video capture is subsequently analysed to give an understanding of the appropriateness
and/or usability of the user-interface. A variety of techniques may be used in this analysis, including
timed or segmented scoring systems or more qualitative assessment.
In this project, it was intended that a series of such evaluations would take place as the Clinical
Workspace was iteratively evolved, in response to further focus group advice and the results of prior
evaluations. The think-aloud evaluation was recorded using video capture software ‘Camtasia
Studio’ and a microphone, to record both patient-clinician dialogue and the clinicians’ think-aloud
comments. Unfortunately because of constraints of both subject and development time, only a
single cycle of evaluation could be performed within the time-frame of this project.
64
The subjects were four experienced GPASS users who had participated in the F3 group discussions
but had no prior experience of the new Clinical Workspace. Each was given an introductory note
explaining the methodology and asked to comment liberally, both on their diagnostic reasoning and
on their interactions with the software, as they enacted a series of dummy consultations. A fellow
clinician role-played the patient, working from a script which allowed a degree of improvisation, to
reflect the latitude of a normal consultation. Such role-play is an accepted methodology in general
practice training202, and both investigator and subject were familiar and comfortable with this
approach. If the clinician omitted to comment for over 15 seconds, the investigator would prompt
them to ‘keep talking’.
The following three scenarios were employed:
1. A 46 year old woman with a sticky red eye. This was a very simple training scenario to allow the
subjects to familiarise themselves with the think-aloud process and with the Clinical Workspace and
CST.
2. A 48 yr old male smoker with a 2 week history of cough + green sputum and slight haemoptysis.
3. A 49 yr old lady with a history of acid reflux, difficulty swallowing and excessive alcohol intake.
Each consultation was run on two occasions for each clinician. On the first iteration, the Clinical
Workspace mechanism was disabled, and the clinician recorded the consultation via a simple text
entry page using pre-existing, familiar prescribing and clinical coding dialogs. On the second
iteration, the same patient was deemed to be returning with similar symptoms after an interval of
several weeks. Although this prevented exact comparisons of the consultations, it gave a more
realistic setting for the clinicians and helped re-establish their level of uncertainty at the outset of
the second consultation. At that second consultation, the clinician was allowed to select and make
use of an appropriate CST via the Clinical Workspace. The consultations were constructed so that an
appropriate CST was available.
It was made clear to the participating clinicians that there was no intention to assess consultation
quality or outcome and that the consultations selected were not designed to be awkward or
particularly challenging.
After recording, each consultation was assessed by a single experimenter and comments made
against 4 criteria:
1. Data entry speed, ease
65
2. Decision making assistance/ Clinical Knowledge prompts
3. Specific usability issues with the Clinical Workspace
4. Completeness of medical record
Timings were analysed for these different segments of the consultation:
1. Preparatory phase: Locating the patient record and reviewing past history, medication etc
2. Patient presentation: The unprompted history given by the patient
3. Clinical history taking with/without CST
4. Clinician decision-making – diagnosis and prescription decision
5. Management actions e.g. create prescription, make referral, close consultation
6. Post consultation recording (notes + diagnosis coding)
Timings for phases 3, 5 and 6 were totalled and compared as these gave an indication of the
influence of user interface interaction. In contrast, it was felt that phases 1, 2 and 4 were largely
dependent on patient factors, independent of computer use, and that timings in those phases was
unhelpful or misleading in assessing the impact of use of the Workspace.
Analysis
Segmented timings analysis
Scenario 1 - Bronchitis Non-CST Timing (s) n=4 CST Timing (s) n=4
1.Preparation 50 30
2.Patient presentation 15 14
3.Patient History phase 82 191
4.Diagnosis / Management Decision 47 35
5.Management Activity 43 15
66
6.Post-consultation Recording 59 30
Total Recording/Activity (3+5+6) 184 236
Percentage Recording/Activity during
consultation
68% 87%
Scenario 2 - Reflux Oesophagitis
1.Preparation 30 30
2.Patient presentation 20 44
3.Patient history phase 95 294
4.Diagnosis / Management Decision 100 30
5.Management Activity 105 0
6.Post consultation Recording 133 66
Total Recording/Activity (3+5+6) 333 360
Percentage Recording/Activity during
consultation
60% 81%
The numbers of subjects and scenarios was not felt sufficient to be amenable to statistical analysis,
however overall average timings showed that use of the CST appeared to make the process of
history taking, recording and management action slower for both scenarios.
Whilst disappointing, this is perhaps not surprising given the lack of familiarity of the subjects with
the new user interface. It is also possible that there is a cognitive load on the clinician, imposed by
having to choose terms from the CST which are not identical to those normally used. This might be
ameliorated by having user-authored/adapted CSTs as originally envisaged.
Closer examination of the timing breakdown does appear to show an interesting change in clinician
interaction with the computer:
Without the CST active, all four clinicians completed the consultation before recording the
consultation, in keeping with standard GP training and best practice advice, in an effort to maintain
patient-clinician eye-contact and communication.
With the CST activated, the processes of history taking, recording and management action were
more likely to be performed contemporaneously during the consultation. Whilst this raises some
67
interesting issues, it does suggests that the Clinical Workspace does integrate more easily with the
flow of the consultation and the clinical decision making process.
Qualitative assessment
Live observation and subsequent analysis of the screen video capture and audio recording allowed a
degree of qualitative evaluation of the Clinical Workspace:
1. Data entry speed, ease:
In spite of the disappointing timings, there was a strong impression from both observer and
subject that narrative recording and prescription production was considerably faster when using
the CST. This was especially true for one of the clinicians who had poorer keyboarding skills. It is
possible that if these perceived productivity gains are genuine, they were being masked in the
timings by the general unfamiliarity of the subjects with the new interface. A more granular
timings analysis may have been helpful as would repetition of the evaluation after more user
experience had been gained.
2. Decision making assistance/ Clinical knowledge prompts:
The CST did appear to act as a gentle prompt to the clinician to take a better history, particularly
reminding them to cover significant red-flag issues. Both clinicians commented specifically that
they found this helpful. The CST also made it much easier to access appropriate patient guidance
leaflets and referral options.
3. Specific usability issues with the Clinical Workspace:
A key criticism of the user interface was that the method of selecting CST elements (from a table
containing checkboxes) was somewhat cumbersome and unintuitive. This could be verified by
examination of the video capture where subject hesitancy was generally reflected by the mouse
pointer ‘circling’ while the subject worked out how to proceed.
4. Completeness of medical record:
In both scenarios, the medical record produced using the CST was much more complete and
adhered to Weed’s SOAP86 structure. Diagnostic coding was included and accurate, whereas
68
without the CST, clinical codes were not entered at all.
Entering a Diagnostic code via the CST for Bronchitis
3. Problems encountered within the evaluation
A number of problems were encountered in conducting the practical aspects of this project. As a
small part of a large and complex redevelopment of a national GP computer system it was perhaps
inevitable that the resources available, in particular to perform the think-aloud evaluation were
compromised by significant slippages in the development cycle and diminishing commitment to
iterative development of the Clinical Workspace in the face of more fundamental problems within
an increasingly complex product re-development. It was noticeable that the early and successful
rapid application development approach reverted to a more traditional Waterfall methodology as
timescales and contractual issues gained hold. This undoubtedly led to a much less agile
development of the Clinical Workspace than had been originally envisaged, and because of long
delays in development with focus being on other aspects of the application, only a single
opportunity to perform the Think-aloud evaluation within the timeframe of the research project.
69
In addition, the Think-aloud methodology employed, whilst of real practical value, proved
considerably more resource-intensive than had been anticipated, particularly for analysis. It proved
difficult to use as the basis for a quantitative study, given the small numbers of subjects and
scenarios investigated. It is possible that the exact methodology and scoring systems employed
could have been refined with experience, but because of the slow rate of development this could
not be achieved. It might also have been helpful to develop a metric for completeness of the medical
record, rather than a qualitative assessment.
A simpler overall analysis methodology such as a questionnaire would certainly have generated
more quantifiable and statistically amenable results, which would have been more helpful, at least
within the context of the research project.
70
IV Discussion
Does narrative still have a place in clinical practice?
Whilst the focus of current health informatics research remains focussed on the challenges of
semantic interoperability, it does appear for many clinicians, narrative remains an important
construct within the medical record. For some, it is simply a familiar format which does not require
re-training in more structured forms of data acquisition and undoubtedly this remains a significant
factor, though likely to diminish as more clinicians come to understand the value of structured,
coded health records. Paradoxically, the narrative nature of much ‘Web 2.0’ social communication,
such as Facebook or Twitter, may produce a generation of clinicians who are particularly
comfortable with narrative styles whilst an older generation, with rudimentary keyboard skills, may
be more comfortable with structured mouse-driven data-entry.
On the basis of focus group discussions held as part of a real-world GP application re-development,
there seemed to significant demand for a user-interface which supports a narrative style of
recording but which simultaneously allows the seamless recording of structured or coded data
whenever required. This desire was tempered by the knowledge that keyboard use remains difficult
for the current generation of clinical users, which made the use of some sort of data entry support
such as the CST mechanism, a crucial aspect of any implementation.
Experience within the MS/NHS CUI initiative appears to confirm a similar requirement from
secondary care clinicians, this being the driver for the development of a text- based data entry
control, which as the clinician records narrative, will offer equivalent SNOMED-CT terms. It is
understood that this is now being adapted to allow similar parsing to detect and offer appropriate
archetype/template-modelled constructs. For example if a clinician types in “BP”, the control will
display an appropriately structured data-entry control for a blood pressure reading. Similar
functionality already exists in a number of existing systems such as EMIS.
The continued research interest in narrative medicine is largely framed as a bulwark against the
prescriptive and directive paradigms of evidence-based bio-medicine. Complaints of de-
professionalisation and de-humanisation remain common, even by commentators who remain
adherents of appropriately applied evidence-based practice, asserting that narrative remains
important in capturing the richness and humanity of clinical communication, often lost when using
71
more structured forms of data entry. From a technical or engineering perspective this seemingly
aesthetic or philosophical appreciation of narrative can seem perplexing, but it often speaks to the
heart of the 'clinical role' and the growing recognition that the softer 'caring' aspects of clinical
practice may have as much significance in maintaining patients’ health and well-being as those
imparted by the biomedical model.
Narrative medicine, therefore, is currently discussed in the realms of clinical education and practice
but remains relatively unexplored within the fields of clinical cognition and clinical computing. There
is some good evidence that structured clinical content is generally easier to interpret than loose
narrative105 but there has been little or no research into the cognitive difficulty of creating such
structured content.
It is curious too that given the clinical interest in narrative and an equivalent interest in complexity
science, both often fostered by an antithesis to a rigid biomedical model of practice, that there
seems to have been little attempt to research possible relationships between the two fields. Recent
fMRI research points to the importance of complexity in fundamental brain activity43. An intriguing
possibility is that the brain may function primarily as a 'complexity processor', narrative being a
related complexity-aligned communications medium.
First impressions?
The Clinical Scenario Template (CST) design of the Clinical Workspace was based on the assumption
that experienced clinicians appear to make extremely rapid accurate decisions about the likely
content of a consultation on the basis of 'first impressions'.
Although the exact mechanisms remain disputed, there does appear to be consistent evidence that
the human brain gradually amalgamates received explicit knowledge, in the form of taught material,
text books, guidelines, and personal experiences, into rapidly accessed and deployed tacit
knowledge. This process of tacit knowledge acquisition has been heavily researched by the
educational and management science communities for whom it is seen a key factor in the transition
of an individual from novice to expert.
Cognitive science identifies 'frames' and 'schema' which allow increasingly complex concepts to be
'chunked' into rapidly retrievable and useable knowledge. Differences can be seen between novice
ands expert users in their relative use of such schema, reflecting their differential. Novices adopt a
72
pure hypothetico-deductive model, whilst experts tend to arrive rapidly at a semi-concrete
conclusion which is then disproved by exclusion of known indicators.
Decision support and flexible data-entry
Although the CST mechanism was designed to offer data-entry support, rather than decision
support, there were elements of crossover and the history of attempts to integrate explicit
knowledge sources into clinical systems was instructive. Similarly there have been prior attempts to
construct more flexible and narrative-aligned methods of data-entry.
From early attempts to develop Artificial Intelligence systems through probalistic techniques, to
cognitively based decision support such as Prodigy, the successful implementation of decision
support within clinical systems remains uncommon. Targeted decision support such as prescribing
and adverse reaction warnings have proved more successful, albeit with a recognition that the
avoidance of ‘alert fatigue’ requires significant tailoring to specific contexts of usage. In other
settings where
Gabbay and LeMay’s9 work highlights the difficulties of adapting such support to sensitively match
the user’s cognitive requirements. These are uniquely individual, based on a mixture of explicit
knowledge, including ‘off-line’ perusal of formal guidelines but crucially entwined with knowledge
gained from personal experience, that of close colleagues and awareness of local resources which
may not meet those specified in formal guidelines.
It would seem possible that there is an element of personal preference between narrative and
structure which is independent of other factors and while individual ‘cognitive style’ was a subject
which attracted some research interest, there is little good evidence to confirm that this might play a
part in such preference.
Attempts to develop flexible data-entry methodologies have also proved difficult. Preference for
structured as against narrative forms of data-entry is highly dependent on clinical role, experience
and the nature of the clinical context e.g. a routine chronic care review will generally favour a
structured approach, in contrast to a routine ‘unscheduled HP consultation. The Pen&Pad project
represented a bold attempt to fuse the developing understanding of inferential health ontology with
a naturalistic point and click interface. It remains unclear why this was ultimately unsuccessful but it
73
seems likely that, given the available technology, poor system responsiveness may have been a
significant factor.
In part through the popularity of CDA and development of the Narrative Block , ‘Structured
narrative’ is receiving growing attention as a possible solution to allow a naturalistic fusion of free
text, code and structure within the electronic health record. The research available remains in the
realms of prototyping and implementations which might be subject to proper evaluation are only
just being released, the GPASS 2007 application, which was the subject of the practical aspects of
this project being one example.
Practical evaluation
Whilst it was encouraging to see the ideas behind the Clinical Workspace being implemented and
seemingly well-received, the practical evaluation aspect of this research project was in retrospect
unsuccessful.
Although regarded as a useful exercise from the perspective of both users and system developers,
the Think-aloud evaluation performed was insufficiently robust to determine whether the Clinical
Workspace was indeed better integrated with the pattern of clinical cognition in a GP consultation.
Although qualitative feedback from users was very positive and there were indications from direct
observation and segmented timings that the use of the computer during the consultation may have
been more naturally aligned to the clinical decision–making process, considerably more detailed
research would have been needed to confirm such impressions in a quantifiable manner. A more
rigorous methodology, such as that used in Prodigy evaluations, coupled with a much larger sample
size, may have delivered some quantifiable results, but given the context of the project and the
pressures on the wider product delivery, this may have been unachievable.
Nevertheless, there were sufficient qualitative pointers to the new interface having a positive effect
on the interaction of the clinician with the computer. Since more interactivity is a necessary pre-
requisite for more carefully attuned data-entry and decision support, this should regarded as a
potentially positive development but if this effect is confirmed, i.e. that clinicians more easily and
readily interact with the computer throughout a consultation, care must be taken that this is not at
the expense of impairing communication with the patient e.g. by loss of eye-contact. Development
of UI techniques which minimise screen-focussing, such as use of keyboard shortcuts will be vital.
74
Pen/stylus interfaces which are less demanding of hand-eye-screen coordination than mouse-driven
alternatives, may be helpful, even if handwriting recognition is not used.
Technical framework
Whilst the developed and evaluated version of the Clinical Workspace was within a commercial
application, the intention was that the underlying information representations of both Clinical
Narrative and Scenarios, should be developed within an open standards-based architectural
framework.
HL7 CDA has many attractive features which would make it a strong candidate. Many jurisdictions
and vendors have been attracted by its layered approach to interoperability which can allow the
gradual development of semantics but, in addition, the CDA specification offers direct support for
structured narrative via the Narrative Block concept, employing a form of XML markup which allows
structured data entries to be referenced within blocks of narrative text – the exact requirement for
Clinical Workspace components. The major drawback of HL7 CDA is the difficulty of specifying
structured content, which arises because the underlying HL7v3 formalism lacks a clinically accessible
constraint layer. The HL7 Template mechanism attempts to simplify this aspect of modelling but
remains too technically complex and obscure to allow broad-based clinical input to content
definition.
In contrast, the openEHR framework was felt to have a more robust approach to allowing the
clinically-led specification of structured content, mediated via the use of archetypes, which
represent single clinical concepts modelled as ‘maximal datasets’, and the openEHR template layer
which allows further constraint to suit local use cases and contexts. Whilst it lacks a specific
equivalent to the CDA ‘Narrative Block’, openEHR has a sufficiently rich reference model to allow the
representation of structured narrative, using the DV_Parsable datatype to store XML markup, either
identical in format to that used within CDA, or by adopting a more appropriate openEHR-aligned
equivalent.
Next steps
In the period taken to conduct this project, there has been in significant shift away from the view
that interoperability issues would be solved almost solely by the adoption of large scale ontologies
75
such as SNOMED-CT. It is now generally accepted that, at least in the near future, scalable semantic
interoperability will only be achieved by the interaction of terminologies with structural information
models.
Specific modelling constructs such as ‘Structured narrative’ are being positively reported in the
literature but have not yet been followed up by concomitant evaluations of actual implementations.
This is clearly an important avenue of further research, which must include studies of any possible
impact on patient-clinician communication.
At a more fundamental level, the relationship between human cognition, narrative and approaches
to clinical decision-making remain ill-defined. fMRI studies are starting to become commonplace203
and may offer increasingly concrete answers, previously only hinted at by introspectional techniques
such as ‘Think-aloud’204. Whilst the intrusive nature of fMRI currently precludes its use in natural
settings such as a clinical consultation, there are some novel approaches e.g. using game-based
simulation205 which might allow aspects of clinical decision-making to be analysed using this
technique.
Such basic and applied research will be crucial if electronic health records are to fulfil their potential
to improve the quality and efficiency of patient care.
76
Acknowledgements
This project arose from personal experience and the germ of an idea. Through a series of
professional and family circumstances (mostly positive), it took considerably longer to reach a
conclusion than was ever anticipated.
In part this was because it has, for the most part, been a labour of love; the pursuance of ideas and
the opportunity to delve into knowledge previously denied by the pragmatic realities of a primary
medical degree.
Thanks must go to Robin Beaumont, my project tutor, for helping me re-float what was at one stage
a rather grounded and leaky boat!
Thanks too to my family who have (mostly) put up with the regular crises along the way.
Ian McNicoll
October 2009
77
References
1. Spence. May the force be with you. 2007. Available at:
http://www.bmj.com/cgi/content/extract/335/7630/1158 [Accessed June 24, 2008].
2. Schade C, Sullivan F, de Lusignan S, Madeley J. e-Prescribing, Efficiency, Quality:
Lessons from the Computerization of UK Family Practice. 2006. Available at:
http://www.jamia.org/cgi/content/abstract/13/5/470.
3. Walsh S. The clinician's perspective on electronic health records and how they can affect
patient care. BMJ. 2004;328(7449):1184-1187. Available at: http://bmj.com.
4. Don Norman's jnd.org / Being Analog. Available at:
http://www.jnd.org/dn.mss/being_analog.html [Accessed March 22, 2007].
5. Greenhalgh T, Books B. How to Read a Paper: The Basics of Evidence Based Medicine.;
2001.
6. So how was it for you? A year of the GMS Contract. Br J Gen Pract. 2005;55(514):396.
Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1463169 [Accessed
October 8, 2009].
7. Greenhalgh T, Hurwitz B. Narrative based medicine: Why study narrative? BMJ.
1999;318(7175):48-50. Available at: http://www.bmj.com [Accessed August 18, 2007].
8. Neighbour, R. The Inner Consultation. Lancaster: MTP; 1987.
9. Gabbay J, le May A. Evidence based guidelines or collectively constructed "mindlines?"
Ethnographic study of knowledge management in primary care. BMJ. 2004;329(7473):1013.
Available at: http://bmj.com/cgi/content/abstract/329/7473/1013.
78
10. Meystre S, Haug PJ. Natural language processing to extract medical problems from
electronic clinical documents: Performance evaluation. Journal of Biomedical Informatics.
2006;39(6):589-599. Available at: http://www.sciencedirect.com/science/article/B6WHD-
4HR71JP-1/1/14bc8dc9fbfb1358203b13e50397ef6c [Accessed June 23, 2008].
11. Hunter K. Doctors' Stories. Princeton: Princeton University Press
12. Weyden MBVD, Armstrong RM, Gregory AT. The 2005 Nobel Prize in Physiology Or
Medicine. The Medical Journal of Australia. 2005;183(11):612-614. Available at:
http://www.mja.com.au/public/issues/183_11_051205/van11000_fm.html#0_i1091639
[Accessed June 24, 2008].
13. Morrison J, Watt G. New Century, New Challenges for Community Based Medical
Education. Medical Education. 2003;37(1):2-3. Available at: http://www.blackwell-
synergy.com/doi/abs/10.1046/j.1365-2923.2003.01428.x [Accessed June 24, 2008].
14. Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence based
medicine: what it is and what it isn't. BMJ. 1996;312(7023):71-72. Available at:
http://www.bmj.com [Accessed June 24, 2008].
15. Timmermans S, Mauck A. The Promises And Pitfalls Of Evidence-Based Medicine.
Health Aff. 2005;24(1):18-28. Available at:
http://content.healthaffairs.org/cgi/content/abstract/24/1/18 [Accessed June 24, 2008].
16. Bryson B. A short history of nearly everything. Broadway Books New York; 2003.
17. This Dynamic Earth--Contents [USGS]. Available at:
http://pubs.usgs.gov/gip/dynamic/dynamic.html [Accessed July 20, 2008].
18. Nachmanovitch S. Free Play: Improvisation in Life and Art.. New York, NY:
Tarcher/Putnam; 1990.
19. Haidet P, Paterniti D. "Building" a History Rather Than "Taking" One: A Perspective on
79
Information Sharing During the Medical Interview. Arch Intern Med. 2003;163(10):1134-
1140. Available at: http://archinte.ama-assn.org/cgi/content/abstract/163/10/1134.
20. Launer J. Narrative-based Primary Care: A Practical Guide. Radcliffe Publishing; 2002.
21. Balint M. The Doctor, His Patient and the Illness. Churchill Livingstone; 2000.
22. Pendleton D. The Consultation: an approach to learning and teaching. Oxford
[Oxfordshire]; New York: Oxford University Press; 1984.
23. Patel V, Arocha J, Kushniruk A. Patients' and physicians' understanding of health and
biomedical concepts: relationship to the design of EMR systems. J Biomed Inform.
2002;35(1):8-16. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=12415722.
24. Cimino JJ, Li J, Mendonca EA, et al. An evaluation of patient access to their electronic
medical records via the World Wide Web. Proc AMIA Symp. 2000;7:151-155.
25. Oreilly T. What is Web 2.0: Design Patterns and Business Models for the Next
Generation of Software. Communications & Strategies, No. 1, p. 17, First Quarter 2007.
2007. Available at: http://ssrn.com/paper=1008839 [Accessed July 30, 2009].
26. Boulos MN, Maramba I, Wheeler S. Wikis, blogs and podcasts: a new generation of
Web-based tools for virtual collaborative clinical practice and education. BMC Medical
Education. 2006;6(41):1472-6920.
27. Wong MC, Turner P, Yee KC. Involving clinicians in the development of an electronic
clinical handover system-thinking systems not just technology. Stud Health Technol Inform.
2008;136:490-5.
28. Pinker S. Language as an adaptation to the cognitive niche. In: In M. Christiansen & S.
Kirby, ed. Language evolution: States of the Art. New York: Oxford University Press; 2003.
80
Available at: http://pinker.wjh.harvard.edu/articles/papers/Language_Evolution.pdf
[Accessed July 15, 2008].
29. Bickerton D. Language evolution: A brief guide for linguists. Lingua. 2007;117(3):510-
526. Available at: http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V6H-
4H3JHK6-
3&_user=10&_coverDate=03%2F31%2F2007&_rdoc=2&_fmt=high&_orig=browse&_srch
=doc-
info(%23toc%235815%232007%23998829996%23638943%23FLP%23display%23Volume)
&_cdi=5815&_sort=d&_docanchor=&_ct=5&_acct=C000050221&_version=1&_urlVersion
=0&_userid=10&md5=dc6b3d8207e2918ac7c5d98acb3b0d9f [Accessed July 16, 2008].
30. Hauser MD, Chomsky N, Fitch WT. The Faculty of Language: What Is It, Who Has It,
and How Did It Evolve? Science. 2002;298(5598):1569-1579. Available at:
http://www.sciencemag.org/cgi/content/abstract/298/5598/1569 [Accessed July 16, 2008].
31. Schank RC, Abelson RP. Knowledge and Memory: The Real Story. Knowledge and
Memory: The Real Story. 1995;8:1-86.
32. Marshall SP. Schemas in problem solving. Cambridge UniversityPress; 1995.
33. Patel VL, Arocha JF, Kaufman DR. A Primer on Aspects of Cognition for Medical
Informatics. J Am Med Inform Assoc. 2001;8(4):324-343. Available at:
http://www.jamia.org/cgi/content/abstract/8/4/324 [Accessed August 10, 2007].
34. Herman D. Narratology as a cognitive science. Image [&] Narrative (Online Magazine of
the Visual Narrative, KU Leuven). 2000;1.
35. Chapman J. System failure. Demos, London; 2002.
36. Plsek PE, Greenhalgh T. Complexity science: The challenge of complexity in health care.
BMJ. 2001;323(7313):625-628. Available at: http://www.bmj.com [Accessed July 30, 2008].
81
37. Burton C. Postcards from the 21st century. Complexity. Br J Gen Pract. 2001;(51):866-
867.
38. Nonlinear Dynamics for Clinicians. Available at: http://www.physionet.org/tutorials/ndc/
[Accessed July 20, 2008].
39. Goldberger AL, Amaral LAN, Glass L, et al. PhysioBank, PhysioToolkit, and PhysioNet :
Components of a New Research Resource for Complex Physiologic Signals. Circulation.
2000;101(23):e215-220. Available at:
http://circ.ahajournals.org/cgi/content/abstract/101/23/e215 [Accessed July 20, 2008].
40. Heath I. Complexity and uncertainty as the links between science and the humanities in
general practice. Journal of New Zealand Family Practice. 2004;31(3).
41. Tsoukas H, Hatch MJ. Complex thinking, complex practice: The case for a narrative
approach to organizational complexity. Human Relations. 2001;54(8):979-1013. Available at:
http://hum.sagepub.com/cgi/content/abstract/54/8/979 [Accessed June 23, 2008].
42. Purves I, Robinson P. "Knowledge management" for health: what "tools" can improve the
performance of workgroups, clinicians and patients? Medinfo. 2004;11(Pt 1):678-82.
Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=15360899.
43. Robson D. Disorderly genius: How chaos drives the brain . New Scientist. 2009;(2714).
Available at: http://www.newscientist.com/article/mg20227141.200-disorderly-genius-how-
chaos-drives-the-brain.html?full=true [Accessed July 30, 2009].
44. Tange HJ, Hasman A, de Vries Robbe PF, Schouten HC. Medical narratives in electronic
medical records. International Journal of Medical Informatics. 1997;46(1):7-29. Available at:
http://www.sciencedirect.com/science/article/B6T7S-3W0FW2T-
2/2/b5411f096e2c908ce62e3f55f0469bda [Accessed August 5, 2007].
82
45. Lovis C, Baud RH, Planche P. Power of expression in the electronic patient record:
structured data or narrative text? International Journal of Medical Informatics. 2000;58-
59:101-10. Available at: http://www.ncbi.nlm.nih.gov/pubmed/10978913 [Accessed August
2, 2008].
46. Kirby J, Rector A. The PEN&PAD data entry system: from prototype to practical system.
1996.
47. Hardiker NR, Rector AL. Modeling Nursing Terminology Using the GRAIL
Representation Language. J Am Med Inform Assoc. 1998;5(1):120-128. Available at:
http://www.jamia.org/cgi/content/abstract/5/1/120 [Accessed August 2, 2008].
48. IHTSDO: SNOMED CT. Available at: http://www.ihtsdo.org/snomed-ct/ [Accessed
August 2, 2008].
49. Lincoln TL, Essin DJ. A document processing architecture for electronic medical records.
Medinfo. 1995;8 Pt 1:227-30. Available at: http://www.ncbi.nlm.nih.gov/pubmed/8591159
[Accessed August 2, 2008].
50. Dolin RH, Alschuler L, Boyer S, et al. HL7 Clinical Document Architecture, Release 2. J
Am Med Inform Assoc. 2005:M1888. Available at:
http://www.jamia.org/cgi/content/abstract/M1888v1.
51. openEHR Primer - openEHR :: future proof and flexible EHR specifications. Available
at: http://www.openehr.org/shared-resources/openehr_primer.html [Accessed July 29, 2008].
52. Berg M, Goorman E. The contextual nature of medical information. International Journal
of Medical Informatics. 1999;56(1-3):51-60. Available at:
http://www.sciencedirect.com.ezp1.bath.ac.uk/science/article/B6T7S-3Y0R8VY-
8/2/b9ed89de88df78e90e60cb126994a4ba [Accessed August 6, 2007].
53. Berg M, Toussaint P. The mantra of modeling and the forgotten powers of paper: a
sociotechnical view on the development of process-oriented ICT in health care. International
83
Journal of Medical Informatics. 2003;69(2-3):223-234. Available at:
http://www.sciencedirect.com/science/article/B6T7S-47RBSTF-
4/1/1603efed945b55e684905d0035352efa [Accessed July 17, 2008].
54. Berg M. Patient care information systems and health care work: a sociotechnical
approach. International Journal of Medical Informatics. 1999;55(2):87-101. Available at:
http://www.sciencedirect.com/science/article/B6T7S-3X9420M-
1/1/2b5dd2a50c378e40d301b611051face7 [Accessed July 17, 2008].
55. A new look at the Problem Oriented Medical Record. Available at:
http://www.phcsg.org/main/pastconf/camb96/mikey.htm [Accessed October 21, 2008].
56. Kay S, Purves I. Medical records and other stories: a narratological framework. Methods
Inf Med. 1996;35(2):72-87.
57. Index (search engine) - Wikipedia, the free encyclopedia. Available at:
http://en.wikipedia.org/wiki/Search_engine_indexing [Accessed July 30, 2009].
58. Coiera E. When Conversation Is Better Than Computation. J Am Med Inform Assoc.
2000;7(3):277-286. Available at: http://www.jamia.org/cgi/content/abstract/7/3/277
[Accessed July 30, 2008].
59. Safran C, Sands DZ, Rind DM. Online medical records: a decade of experience. Methods
of Information in Medicine. 1999;38(4-5):308-12. Available at:
http://www.ncbi.nlm.nih.gov/pubmed/10805019 [Accessed August 2, 2008].
60. Shirky: The Semantic Web, Syllogism, and Worldview. Available at:
http://www.shirky.com/writings/semantic_syllogism.html [Accessed August 2, 2008].
61. Bunduchi R, Williams R, Graham I, Smart A. XML-based clinical data standardisation in
the National Health Service Scotland. Informatics in Primary Care. 2006;14(4):227.
62. Gardner H. The mind's new science: a history of the cognitive revolution. 1985.
84
63. Miller GA. The cognitive revolution: a historical perspective. Trends in Cognitive
Sciences. 2003;7(3):141-144.
64. Pinker S. The Blank Slate: The Modern Denial of Human Nature. Penguin Books; 2003.
65. Miller GA. The Magical Number Seven, Plus or Minus Two: Some Limits on our
Capacity for Processing Information.
66. Deecke L, Kornhuber HH. Human freedom, reasoned will, and the brain: The
Bereitschaftspotential story. In: The Bereitschaftspotential: movement-related cortical
potentials. Springer; 2003:283.
67. Benjamin Libet - a short delay. Available at: http://www.consciousentities.com/libet.htm
[Accessed July 30, 2009].
68. Hurlburt RT, Heavey CL. Telling what we know: describing inner experience. Trends in
Cognitive Sciences. 2001;5(9):400-403. Available at:
http://www.sciencedirect.com.ezp2.bath.ac.uk/science/article/B6VH9-43VBW64-
K/2/0efcf3756da780f9b32d16c63487e831 [Accessed July 30, 2009].
69. Nisbett RE, Wilson TD. Telling more than we can know: Verbal reports on mental
processes. Psychological Review. 1977;84(3):231-259. Available at:
http://www.sciencedirect.com.ezp2.bath.ac.uk/science/article/B6X04-4NN6WFG-
1/2/e1fea65d24be9c301e9ff302653e71ef [Accessed July 30, 2009].
70. Patel V, Kaufman D, Arocha J. Emerging paradigms of cognition in medical decision-
making. J Biomed Inform. 2002;35(1):52-75. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=12415726.
71. Taylor P. From Patient Data to Medical Knowledge: The Principles and Practice of
Health Informatics. Wiley; 2006.
85
72. Shortliffe EH. Medical expert systems-Knowledge tools for physicians. West J Med.
1986;145(6):830-839.
73. Bruner JS. The Process of Education. Harvard Univ Pr; 1977.
74. Bruner JS. Actual Minds, Possible Worlds. Harvard University Press; 1986.
75. Newell A, Simon HA. Human problem solving. Englewood Cliffs, NJ: Prentice-Hall;
1972.
76. Tennant M. Psychology and Adult Learning. Routledge; 1997.
77. Kozhevnikov M. Cognitive Styles in the Context of Modern Psychology: Toward an
Integrated Framework of Cognitive Style. Psychological Bulletin. 2007;133(3):18.
78. Riding RJ. On the Nature of Cognitive Style. Educational Psychology. 1997;17(1):29.
Available at: http://www.informaworld.com.ezp1.bath.ac.uk/10.1080/0144341970170102
[Accessed August 6, 2008].
79. Agor WH. Using intuition to manage organizations in the future. Business Horizons.
1984;27(4):49-54. Available at: http://www.sciencedirect.com/science/article/B6W45-
4D5P86R-4T/2/b812685029f2eaf1dee2116ba41d5053 [Accessed August 11, 2008].
80. Cegarra J, Hoc J. Cognitive styles as an explanation of experts' individual differences: A
case study in computer-assisted troubleshooting diagnosis. International Journal of Human-
Computer Studies. 2006;64(2):123-136. Available at:
http://www.sciencedirect.com.ezp2.bath.ac.uk/science/article/B6WGR-4GWBF6G-
4/1/82a500c12d10e43dd91e42862210e50f [Accessed July 30, 2008].
81. Mandell SF. Resistance to computerization. Journal of Medical Systems. 1987;11(4):311-
318. Available at: http://dx.doi.org/10.1007/BF00994015 [Accessed August 5, 2008].
86
82. WAGNER R, STERNBERG R. Practical intelligence in real-world pursuits: the role of
tacit knowledge. Journal of personality and social psychology. 1985;49(2):436-458.
83. Polanyi M. The Tacit Dimension. In: Knowledge in Organisations.
84. Nonaka I. The knowledge-creating company. Harvard business review. 2007;85(7-8).
85. Herschel RT, Nemati H, Steiger D. Tacit to explicit knowledge conversion: knowledge
exchange protocols. Journal of Knowledge Management. 2001;5(1):107 - 116. Available at:
http://www.emeraldinsight.com/10.1108/13673270110384455 [Accessed June 23, 2008].
86. Weed LL. Medical Records that guide and teach. New Engl. Med. 278
(1968)(11+12):593-600+652-597.
87. Johnson B, Lorenz E, Lundvall B. Why all this fuss about codified and tacit knowledge?
Ind Corp Change. 2002;11(2):245-262. Available at:
http://icc.oxfordjournals.org/cgi/content/abstract/11/2/245 [Accessed August 11, 2008].
88. Elstein AS, Shulman LS, Sprafka SA. Medical Problem Solving: A Ten-Year
Retrospective. Evaluation & the Health Professions. 1990;13(1):5.
89. Patel VL, Groen GJ. Knowledge based solution strategies in medical reasoning. Cognitive
Science: A Multidisciplinary Journal. 1986;10(1):91. Available at:
http://www.informaworld.com/10.1207/s15516709cog1001_4 [Accessed August 18, 2008].
90. Patel VL, Groen GJ, Arocha JF. Medical expertise as a function of task difficulty. Mem
Cognit. 1990;18(4):394-406. Available at:
http://www.psychonomic.org/search/view.cgi?id=6937.
91. Barrows HS, Norman GR, Neufeld VR, Feightner JW. The clinical reasoning of
randomly selected physicians in general medical practice. Clinical and Investigative
Medicine. Médecine Clinique Et Experimentale. 1982;5(1):49-55. Available at:
http://www.ncbi.nlm.nih.gov/pubmed/7116714 [Accessed August 18, 2008].
87
92. Kushniruk AW, Patel VL, Marley AA. Small worlds and medical expertise: implications
for medical cognition and knowledge engineering. International Journal of Medical
Informatics. 1998;49(3):255-271. Available at:
http://www.sciencedirect.com.ezp2.bath.ac.uk/science/article/B6T7S-3T2H1D3-
1/1/f45f38c81d502d27e72ebc55ced773f8 [Accessed June 23, 2008].
93. Croskerry P. Achieving Quality in Clinical Decision Making: Cognitive Strategies and
Detection of Bias. Academic Emergency Medicine. 2002;9(11):1184-1204. Available at:
http://dx.doi.org/10.1197/aemj.9.11.1184 [Accessed August 25, 2008].
94. Ritter BJ. An Analysis of Expert Nurse Practitioners' Diagnostic Reasoning. Journal of
the American Academy of Nurse Practitioners. 2003;15(3):137-141. Available at:
http://dx.doi.org/10.1111/j.1745-7599.2003.tb00270.x [Accessed August 5, 2008].
95. Benner P, Tanner C. Clinical judgment: how expert nurses use intuition. American
Journal of Nursing. 1987;87(1):23-31.
96. Offredy M. Decision-making in primary care: outcomes from a study using patient
scenarios. Journal of Advanced Nursing. 2002;40(5):532-541. Available at:
http://dx.doi.org/10.1046/j.1365-2648.2002.02410.x [Accessed August 5, 2008].
97. Patel VL, Zhang J, Yoskowitz NA, Green R, Sayan OR. Translational cognition for
decision support in critical care environments: A review. Journal of Biomedical Informatics.
2008;41(3):413-431. Available at: http://www.sciencedirect.com/science/article/B6WHD-
4RTM31C-1/1/b61255b6761a917e53f70bcca15ef70d [Accessed June 23, 2008].
98. Coiera E. Four rules for the reinvention of health care. BMJ. 2004;328(7449):1197–1199.
Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=411109 [Accessed
August 25, 2008].
99. Kohn LT, Corrigan JM, Donaldson MS, McKay T, Pike KC. To err is human. National
Acad. Press; 2000.
88
100. Zhang J, Patel V, Johnson T. Medical error: is the solution medical or cognitive? J Am
Med Inform Assoc. 2002;9(6 Suppl):S75-7. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=12386188.
101. Zhang J, Patel V, Johnson T, Shortliffe E. A cognitive taxonomy of medical errors. J
Biomed Inform. 2004;37(3):193-204. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=15196483.
102. Wachter RM. The End Of The Beginning: Patient Safety Five Years After ‘To Err Is
Human’. Health Affairs. 2004;4:534.
103. Shah NR, Seger AC, Seger DL, et al. Improving Acceptance of Computerized
Prescribing Alerts in Ambulatory Care. J Am Med Inform Assoc. 2006;13(1):5-11. Available
at: http://www.jamia.org/cgi/content/abstract/13/1/5 [Accessed July 25, 2008].
104. Ash JS, Berg M, Coiera E. Some Unintended Consequences of Information Technology
in Health Care: The Nature of Patient Care Information System-related Errors. J Am Med
Inform Assoc. 2004;11(2):104-112. Available at:
http://www.jamia.org/cgi/content/abstract/11/2/104 [Accessed August 25, 2008].
105. Patel V, Kushniruk A, Yang S, Yale J. Impact of a computer-based patient record
system on data collection, knowledge organization, and reasoning. J Am Med Inform Assoc.
2000;7(6):569-85. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=11062231.
106. Tange HJ, Schouten HC, Kester ADM, Hasman A. The Granularity of Medical
Narratives and Its Effect on the Speed and Completeness of Information Retrieval. J Am Med
Inform Assoc. 1998;5(6):571–582. Available at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=61337 [Accessed August 13,
89
2008].
107. Sharda P, Das AK, Cohen TA, Patel V. Customizing clinical narratives for the electronic
medical record interface using cognitive methods. International Journal of Medical
Informatics. 2006;75(5):346-368. Available at:
http://www.sciencedirect.com/science/article/B6T7S-4GY8B1M-
5/2/bb3f5606399de9db2dfbeff52eeaeb4f [Accessed August 26, 2008].
108. Sharda P, Das AK, Patel V. Specifying Design Criteria for Electronic Medical Record
Interface Using Cognitive Framework. AMIA Annu Symp Proc. 2003;2003:594–598.
Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1480192 [Accessed
August 26, 2008].
109. Patel V, Arocha J. Cognitive models of clinical reasoning and conceptual representation.
Methods of information in medicine. 1995;34(1-2):47-56.
110. Jordan K, Porcheret M, Croft P. Quality of morbidity coding in general practice
computerized medical records: a systematic review. Fam. Pract. 2004;21(4):396-412.
Available at: http://fampra.oxfordjournals.org/cgi/content/abstract/21/4/396 [Accessed
August 26, 2008].
111. de Lusignan S, Valentin T, Chan T, et al. Problems with primary care data quality:
osteoporosis as an exemplar. Informatics in Primary Care. 2004;12:147-156. Available at:
http://www.ingentaconnect.com/content/rmp/ipc/2004/00000012/00000003/art00004
[Accessed August 27, 2008].
112. de Lusignan S, Wells SE, Hague NJ, Thiru K. Managers see the problems associated
with coding clinical data as a technical issue whilst clinicians also see cultural barriers.
Methods Inf Med. 2003;42(4):416-22. Available at:
http://www.schattauer.de.ezp1.bath.ac.uk/index.php?id=744&no_cache=1&L=1&tx_ivabonn
enten_pi5[download]=1&tx_ivabonnenten_pi5[artikel]=475.
113. de Lusignan S. The barriers to clinical coding in general practice: A literature review.
90
Informatics for Health and Social Care. 2005;30(2):89. Available at:
http://www.informaworld.com.ezp2.bath.ac.uk/10.1080/14639230500298651 [Accessed
August 26, 2008].
114. Zhang J. Representations of health concepts: a cognitive perspective. Journal of
Biomedical Informatics. 2002;35(1):17-24. Available at:
http://www.sciencedirect.com/science/article/B6WHD-46V4S1Y-
4/1/b2ab9950449cdbf062866a56d1ea8c97 [Accessed June 23, 2008].
115. Beale T. OpenEHR Archetype Tutorial. 2006. Available at:
http://www.oceaninformatics.biz/publications/openEHR-Design-1.ppt.
116. Chisholm CD, Weaver CS, Whenmouth LF, Giles B, Brizendine EJ. A Comparison of
Observed Versus Documented Physician Assessment and Treatment of Pain: The Physician
Record Does Not Reflect the Reality. Annals of Emergency Medicine. In Press, Corrected
Proof. Available at: http://www.sciencedirect.com.ezp1.bath.ac.uk/science/article/B6WB0-
4S26JPN-5/2/0d6fd4ce56e1395676146ae0fdc8c364 [Accessed August 13, 2008].
117. Wears RL. The Chart is Dead--Long Live the Chart. Annals of Emergency Medicine. In
Press, Corrected Proof. Available at:
http://www.sciencedirect.com.ezp1.bath.ac.uk/science/article/B6WB0-4SDFS18-
3/2/7ff103c069d7d9af899bd2810037cbcd [Accessed August 13, 2008].
118. Kushniruk A, Patel V. Cognitive and usability engineering methods for the evaluation of
clinical information systems. J Biomed Inform. 2004;37(1):56-76. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=15016386.
119. Jaspers MWM, Steen T, Bos C, Geenen M. The think aloud method: a guide to user
interface design. International Journal of Medical Informatics. 2004;73(11-12):781-795.
120. Weir CR, Nebeker JJ, Hicken BL, et al. A Cognitive Task Analysis of Information
Management Strategies in a Computerized Provider Order Entry Environment. J Am Med
91
Inform Assoc. 2007;14(1):65-75. Available at:
http://www.jamia.org/cgi/content/abstract/14/1/65 [Accessed July 20, 2008].
121. Cohen T, Kaufman D, White T, et al. Cognitive evaluation of an innovative psychiatric
clinical knowledge enhancement system. Medinfo. 2004;11(Pt 2):1295-9. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=15361023.
122. Cimino JJ, Patel VL, Kushniruk AW. Studying the Human– Computer–Terminology
Interface. J Am Med Inform Assoc. 2001;8(2):163–173. Available at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=134555 [Accessed August 13,
2008].
123. Johnson CM, Turley JP. The significance of cognitive modeling in building healthcare
interfaces. International Journal of Medical Informatics. 2006;75(2):163-172. Available at:
http://www.sciencedirect.com.ezp2.bath.ac.uk/science/article/B6T7S-4GTW002-
1/1/e5c45b700232327bc10d0c15814bb836 [Accessed June 23, 2008].
124. Horsky J, Kaufman D, Patel V. The cognitive complexity of a provider order entry
interface. AMIA Annu Symp Proc. 2003:294-8. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=14728181.
125. Purves I. Facing future challenges in general practice: a clinical method with computer
support. Fam Pract. 1996;13(6):536-43. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=9023531.
126. Robinson P, Heywood P. What do GPs need to know? The use of knowledge in general
practice consultations. Br J Gen Pract. 2000;50(450):56–59.
127. Backlund L, Skånér Y, Montgomery H, Bring J, Strender L. GPs' decisions on drug
treatment for patients with high cholesterol values: A think-aloud study. BMC Med Inform
92
Decis Mak. 2004;4:23. Available at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=539306 [Accessed July 20,
2008].
128. Greenes R, Peleg M, Boxwala A, et al. Sharable computer-based clinical practice
guidelines: rationale, obstacles, approaches, and prospects. Medinfo. 2001;10(Pt 1):201-5.
Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=11604733.
129. Purves IN, Sugden B, Booth N, Sowerby M. The PRODIGY project - –the iterative
development of the release one model. Proceedings of the AMIA Annual Symposium.
1999:359-363.
130. Rogers J, Jain NL, Hayes GM. Evaluation of an implementation of PRODIGY phase
two. Proc AMIA Symp. 1999;604:8.
131. Buchan IE, Hanka R, Pencheon D, Bundred P. Introduction of the computer assisted
prescribing scheme Prodigy was premature. BMJ. 1996;313(7064):1083. Available at:
http://www.bmj.com [Accessed August 28, 2008].
132. Purves I. Prodigy, a computer assisted prescribing scheme. BMJ.
1996;313(7071):1549b. Available at: http://www.bmj.com [Accessed August 28, 2008].
133. Hall R., Jones N., Purves I. Acquiring, encoding, and re-using clinical knowledge in
PRODIGY. Informatics in Primary Care. 2002;10:211-216. Available at:
http://www.ingentaconnect.com/content/rmp/ipc/2002/00000010/00000004/art00005
[Accessed August 27, 2008].
134. Johnson P, Tu S, Booth N, Sugden B, Purves I. Using scenarios in chronic disease
management guidelines for primary care. Proc AMIA Symp. 2000:389-93. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=11079911.
93
135. Eccles M, McColl E, Steen N, et al. Effect of computerised evidence based guidelines
on management of asthma and angina in adults in primary care: cluster randomised controlled
trial. Bmj. 2002;325(7370):941. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=12399345.
136. bmj.com Rapid Responses for Eccles et al., 325 (7370) 941. Available at:
http://www.bmj.com/cgi/eletters/325/7370/941#26964 [Accessed August 27, 2008].
137. Beaumont R, Doughty J, Steen N. Prodigy System Generated Data ('Log File Data')
Report for Phase Two. Available at:
http://www.fhi.rcsed.ac.uk/rbeaumont/virtualclassroom/chap13/report1.pdf [Accessed July
30, 2009].
138. Hetlevik I, Holmen J, Krüger Ø. Implementing clinical guidelines in the treatment of
hypertension in general practice: Evaluation of patient outcome related to implementation of
a computer-based clinical decision support system. Scandinavian Journal of Primary Health
Care. 1999;17(1):35. Available at:
http://www.informaworld.com.ezp1.bath.ac.uk/10.1080/028134399750002872 [Accessed
August 28, 2008].
139. Tierney WM, Overhage JM, Murray MD, et al. Effects of Computerized Guidelines for
Managing Heart Disease in Primary Care. Journal of General Internal Medicine.
2003;18(12):967-976. Available at: http://dx.doi.org/10.1111/j.1525-1497.2003.30635.x
[Accessed August 28, 2008].
140. Steel N, Maisey S, Clark A, Fleetcroft R, Howe A. Quality of clinical primary care and
targeted incentive payments: an observational study. Br J Gen Pract. 2007;57(539):449–454.
Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2078183 [Accessed
August 28, 2008].
141. Kernick DP. Evidence based medicine: does it make a difference?: Management of
94
complex systems needs new approaches. BMJ. 2005;330(7482):92-b-93. Available at:
http://www.bmj.com [Accessed August 28, 2008].
142. Robinson P, Purves I, Wilson R. Learning support for the consultation: information
support and decision support should be placed in an educational framework. Medical
Education. 2003;37(5):429-433. Available at: http://dx.doi.org/10.1046/j.1365-
2923.2003.01498.x [Accessed August 25, 2008].
143. Sutton DR, Fox J. The syntax and semantics of the PROforma guideline modeling
language. Journal of the American Medical Informatics Association. 2003;10(5):433-443.
144. Walton RT, Gierl C, Yudkin P, et al. Evaluation of computer support for prescribing
(CAPSULE) using simulated cases. BMJ. 1997;315(7111):791-5.
145. Emery J, Walton R, Murphy M, et al. Computer support for interpreting family histories
of breast and ovarian cancer in primary care: comparative study with simulated cases. BMJ.
2000;321(7252):28-32. Available at: http://www.bmj.com/cgi/content/abstract/321/7252/28
[Accessed August 28, 2008].
146. Fox J, Johns N, Rahmanzadeh A. Disseminating medical knowledge: the PROforma
approach. Artificial Intelligence In Medicine. 1998;14(1-2):157-182.
147. Fox J. Probability, logic and the cognitive foundations of rational belief. Journal of
Applied Logic. 2003;1(3-4):197-224.
148. Microsoft Health CUI - Home. Available at: http://www.mscui.net/ [Accessed October
7, 2009].
149. Jagannathan V, Mullett CJ, Arbogast JG, et al. Assessment of commercial NLP engines
for medication information extraction from dictated clinical notes. International Journal of
Medical Informatics. 2009;78(4):284-291. Available at:
http://www.sciencedirect.com.ezp2.bath.ac.uk/science/article/B6T7S-4TKXDFB-
1/2/f8adf72e00b6596c6773e19817c9ca9a [Accessed August 8, 2009].
95
150. Elkin PL, Brown SH, Bauer BA, et al. A controlled trial of automated classification of
negation from clinical notes. BMC Med Inform Decis Mak. 2005;5:13. Available at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1142321 [Accessed July 20,
2008].
151. Los R, van Ginneken A, de Wilde M, van der Lei J. OpenSDE: Row Modeling Applied
to Generic Structured Data Entry. 2004. Available at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=353023.
152. Venema AC, Ginneken AMV, Wilde MD, Bogers AJ. Is OpenSDE an alternative for
dedicated medical research databases? An example in coronary surgery. BMC Med Inform
Decis Mak. 2007;7:31. Available at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2173886 [Accessed July 20,
2008].
153. Bleeker S, Derksen-Lubsen G, van Ginneken A, van der Lei J, Moll H. Structured data
entry for narrative data in a broad specialty: patient history and physical examination in
pediatrics. BMC Medical Informatics and Decision Making. 2006;6(1):29. Available at:
http://www.biomedcentral.com/1472-6947/6/29 [Accessed July 20, 2008].
154. Poon AD, Fagan LM, Shortliffe EH. The PEN-Ivory project: exploring user-interface
design for the selection of items from large controlled vocabularies of medicine. J Am Med
Inform Assoc. 1996;3(2):168–183. Available at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=116299&rendertype=abstrac
t [Accessed August 13, 2008].
155. News - The new Panasonic Toughbook CF-H1 - UK & Ireland. Available at:
http://www.panasonic.co.uk/html/en_GB/1838907/index.html [Accessed October 7, 2009].
156. Johnson SB, Bakken S, Dine D, et al. An Electronic Health Record Based on Structured
Narrative. J Am Med Inform Assoc. 2008;15(1):54-64. Available at:
http://www.jamia.org/cgi/content/abstract/15/1/54 [Accessed July 20, 2008].
96
157. Hyun S, Johnson SB, Stetson PD, Bakken S. Development and evaluation of nursing
user interface screens using multiple methods. Journal of Biomedical Informatics. 2009.
158. Semantic Interoperability for Better Health and Safer Healthcare. Available at:
http://ec.europa.eu/information_society/activities/health/docs/publications/2009/2009semanti
c-health-report.pdf [Accessed August 2, 2009].
159. Rector AL, Nowlan WA, Kay S. Foundations for an electronic medical record. Methods
Inf Med. 1991;30(3):179-86.
160. Rector A. The Interface between Information, Terminology and Inference Models.
MEDINFO. 2001:246-50.
161. Logical Record Architecture for Health and Social Care — NHS Connecting for Health.
Available at: http://www.connectingforhealth.nhs.uk/systemsandservices/data/lra [Accessed
August 2, 2009].
162. Chen R, Klein G, Sundvall E, Karlsson D, Ahlfeldt H. Archetype-based conversion of
EHR content models: pilot experience with a regional EHR system. BMC Medical
Informatics and Decision Making. 2009;9(1):33. Available at:
http://www.biomedcentral.com/1472-6947/9/33 [Accessed August 3, 2009].
163. Benson T. Why general practitioners use computers and hospital doctors do not---Part 1:
incentives. BMJ. 2002;325(7372):1086-1089. Available at:
http://www.bmj.com.ezp1.bath.ac.uk/ [Accessed August 2, 2009].
164. Benson T. Why general practitioners use computers and hospital doctors do not—Part 2:
scalability. BMJ. 2002;325(7372):1090–1093. Available at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=131195 [Accessed August 2,
2009].
165. 33. Read Codes. Available at: http://www.nasgp.org.uk/z_old_handbook/33.htm
97
[Accessed August 2, 2009].
166. Buckland R. The language of health. BMJ. 1993;306(6873):287-288. Available at:
http://www.ncbi.nlm.nih.gov.ezp1.bath.ac.uk/pubmed/8461644 [Accessed August 3, 2009].
167. Logical Observation Identifiers Names and Codes (LOINC). 2004. Available at:
http://www.nlm.nih.gov/research/umls/loinc_main.html [Accessed August 3, 2009].
168. Coiera E. Medical Informatics, the Internet, and Telemedicine. London: Arnold. 1997.
169. SNOMED CT Technical_Implementation Guide 2009. Available at:
http://www.ihtsdo.org/fileadmin/user_upload/Docs_01/SNOMED_CT_Publications/SNOME
D_CT_Technical_Implementation_Guide_20090131.pdf [Accessed August 14, 2009].
170. Andrews JE, Richesson RL, Krischer J. Variation of SNOMED CT Coding of Clinical
Research Concepts among Coding Experts. J Am Med Inform Assoc. 2007;14(4):497-506.
Available at: http://www.jamia.org/cgi/content/abstract/14/4/497.
171. Rector AL. Models of Meaning and Models of Use: Binding Terminology to the EHR–
An Approach using OWL.
172. Cimino J. Desiderata for controlled medical vocabularies in the twenty-first century.
Methods Inf Med. 1998;37(4):394–403.
173. Haimowitz IJ, Patil RS, Szolovits P. Representing medical knowledge in a
terminological language is difficult. In: Symposium on Computer Applications in Medical
Care.; 1988:101-105.
174. Terminfo H. Meaning, Use & Implementation Meaning, Use & Implementation.
175. Health Level Seven. HL7. Available at: http://www.hl7.org/about/index.cfm [Accessed
August 9, 2009].
98
176. Dolin RH, Alschuler L, Beebe C, et al. The HL7 clinical document architecture. Am
Med Inform Assoc; 2001.
177. Meystre S, Haug PJ. Medical Problem and Document Model for Natural Language
Understanding. AMIA Annu Symp Proc. 2003;2003:455–459. Available at:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1479970 [Accessed July 20,
2008].
178. Beeler GW. HL7 Version 3--An object-oriented methodology for collaborative
standards development. International Journal of Medical Informatics. 1998;48(1-3):151-161.
Available at: http://www.sciencedirect.com.ezp2.bath.ac.uk/science/article/B6T7S-
3VXY3SW-R/2/5b06ea6b45772b8bc09d23702ff6d6cc [Accessed August 14, 2009].
179. Smith B, Ceusters W. HL7 RIM: An incoherent standard. Studies in Health Technology
and Informatics. 2006;124:133.
180. Schadow G, Mead CN, Walker DM. The HL7 Reference Information Model Under
Scrutiny. Studies in health technology and informatics. 2006;124:151.
181. Garde S, Knaup P, Hovenga E. openEHR Archetypes in Electronic Health Records: the
Path to Semantic Interoperability?
182. Huff S. Detailed clinical models as the basis for medical decision support. In: IHC;
2004. Available at: http://www.sageproject.net/links/detailedclinicalmodels.ppt
file://C:%5CDocuments%20and%20Settings%5CAll%20Users.WINDOWS%5CDocuments
%5CMSC%5CProject%5Cdetailedclinicalmodels.ppt.
183. Kalra D, Beale T, Heard S. The openEHR Foundation. Studies in health technology and
informatics. 2005;115:153-73. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=16160223.
184. Beale T. Archetypes: Constraint-based Domain Models for Future-proof Information
99
Systems. OOPSLA 2002 workshop on behavioural semantics. 2002.
185. Elements of Service-Oriented Analysis and Design. Available at:
http://www.ibm.com/developerworks/webservices/library/ws-soad1/ [Accessed July 20,
2008].
186. Patrick J, Ly R, Truran D. Evaluation of a Persistent Store for openEHR. In: HIC 2006
and HINZ 2006: Proceedings;. 2006. Health Informatics Society of Australia; :83-89.
Available at:
<http://search.informit.com.au/documentSummary;dn=950988993823515;res=E-LIBRARY>
ISBN: 0975101374.
187. Ingram D. The good European health record. Health in the New Communications Age:
Health Care Telematics for the 21st Century. 1995:66.
188. Leslie H. International developments in openEHR archetypes and templates. HIM J.
2008;37(1):38-9.
189. Buck J, Garde S, Kohl CD, Knaup-Gregori P. Towards a comprehensive electronic
patient record to support an innovative individual care concept for premature infants using
the openEHR approach. International Journal of Medical Informatics. 2009;78(8):521-531.
Available at: http://www.sciencedirect.com/science/article/B6T7S-4W1BFVH-
1/2/0681f94906864c61d98305226a29af58 [Accessed July 30, 2009].
190. Leslie H. International developments in openEHR archetypes and templates. The HIM
journal. 2008;37(1):38.
191. Hovenga EJS, Garde S, Carr T, Hullin CM. Innovative approaches and processes for
capturing expert aged care knowledge for multiple purposes. electronic Journal of Health
Informatics. 2007;2(1).
192. Bernstein K, Bruun-Rasmussen M, Vingtoft S, Andersen SK, Nøhr C. Modelling and
implementing electronic health records in Denmark. International Journal of Medical
100
Informatics. 2005;74(2-4):213-220.
193. NHS Scotland. GPASS - Clinical Software. Available at: http://www.gpass.scot.nhs.uk/
[Accessed August 15, 2009].
194. Pringle M, et al. Independent External Assessment of GPASS: Looking to the Future of
Health Informatics in Scotland 2002. NHS Scotland; 2002. Available at:
http://www.show.scot.nhs.uk/gpassslg/GPASSPringleReview.pdf [Accessed June 9, 2006].
195. Beynon-Davies P, Carne C, Mackay H, Tudhope D. Rapid application development
(RAD): an empirical review. European Journal of Information Systems. 1999;8:211-223.
196. DeBellis M, Haapala C. User Centric Software Engineering. IEEE Expert. 1995:34-41.
197. Paetsch F, Eberlein A, Maurer F. Requirements engineering and agile software
development. In: Proceedings of the Twelfth International Workshop on Enabling
Technologies: Infrastructure for Collaborative Enterprises. IEEE Computer Society
Washington, DC, USA; 2003:308.
198. Perlman G. User Interface Usability Evaluation with Web-Based Questionnaires. 2009.
Available at: http://oldwww.acm.org/perlman/question.html [Accessed October 4, 2009].
199. Nielsen J. Ten usability heuristics. Retrieved October. 2001;1:2001.
200. Theadom A, de Lusignan S, Wilson E, Chan T. Using three-channel video to evaluate
the impact of the use of the computer on the patient-centredness of the general practice
consultation. Informatics in primary care. 2003;11(3):149-56. Available at:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=14680538.
201. Gibson M, Jenkings K, Wilson R, Purves I. Multi-tasking in practice: coordinated
activities in the computer supported doctor-patient consultation. Int J Med Inform.
2005;74(6):425-36. Available at:
101
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation
&list_uids=15914080
202. Mansfield F. Supervised role-play in the teaching of the process of consultation. Medical
Education. 1991;25(6):485-490.
203. Monti MM, Parsons LM, Osherson DN. The boundaries of language and thought in
deductive inference. Proceedings of the National Academy of Sciences. 2009;106(30):12554-
12559. Available at: http://www.pnas.org.ezp1.bath.ac.uk/content/106/30/12554.abstract
[Accessed August 13, 2009].
204. Fangmeier T, Knauff M, Ruff CC, Sloutsky V. fMRI Evidence for a Three-Stage Model
of Deductive Reasoning. Journal of Cognitive Neuroscience. 2006;18(3):320-334. Available
at: http://dx.doi.org.ezp2.bath.ac.uk/10.1162/jocn.2006.18.3.320 [Accessed August 13, 2009].
205. Klasen M, Zvyagintsev M, Weber R, Mathiak K, Mathiak K. Think Aloud during fMRI:
Neuronal Correlates of Subjective Experience in Video Games. In: Fun and Games.;
2008:132-138. Available at: http://dx.doi.org.ezp2.bath.ac.uk/10.1007/978-3-540-88322-
7_13 [Accessed September 20, 2009].
