View
3
Download
0
Category
Preview:
Citation preview
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
1 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
Maged N. Kamel Boulos1, Ann C. Brewer2, Chante Karimkhani3, David B. Buller4, Robert P.
Dellavalle5
1Faculty of Health & Human Sciences, University of Plymouth, Drake Circus, Plymouth, Devon PL4 8AA, UK 2
Dermatology Residency Program, Mayo Clinic in Arizona, Scottsdale, AZ 85259, USA 3
Columbia University College of Physicians and Surgeons, New York, NY 10032, USA 4
Klein Buendel, Inc, Golden, CO 80401, USA 5Dermatology Service, Denver VA Medical Center, Denver, CO 80220, USA
Abstract
This paper examines the state of the art in mobile clinical and health-related apps. A 2012 estimate puts the number of health-related apps at no fewer than 40,000, as healthcare professionals and consumers continue to express concerns about the quality of many apps, calling for some form of app regulatory control or certification to be put in place. We describe the range of apps on offer as of 2013, and then present a brief survey of evaluation studies of medical and health-related apps that have been conducted to date, covering a range of clinical disciplines and topics. Our survey includes studies that highlighted risks, negative issues and worrying deficiencies in existing apps. We discuss the concept of ‘apps as a medical device’ and the relevant regulatory controls that apply in USA and Europe, offering examples of apps that have been formally approved using these mechanisms. We describe the online Health Apps Library run by the National Health Service in England and the calls for a vetted medical and health app store. We discuss the ingredients for successful apps beyond the rather narrow definition of ‘apps as a medical device’. These ingredients cover app content quality, usability, the need to match apps to consumers’ general and health literacy levels, device connectivity standards (for apps that connect to glucometers, blood pressure monitors, etc.), as well as app security and user privacy. ‘Happtique Health App Certification Program’ (HACP), a voluntary app certification scheme, successfully captures most of these desiderata, but is solely focused on apps targeting the US market. HACP, while very welcome, is in ways reminiscent of the early days of the Web, when many “similar” quality benchmarking tools and codes of conduct for information publishers were proposed to appraise and rate online medical and health information. It is probably impossible to rate and police every app on offer today, much like in those early days of the Web, when people quickly realised the same regarding informational Web pages. The best first line of defence was, is, and will always be to educate consumers regarding the potentially harmful content of (some) apps.
Keywords: mobile apps, text messaging, smartphones, mobile tablet computers, mobile health (mHealth), telemedicine, healthcare, evaluation, regulation and certification, quality
Correspondence: mnkamelboulos@plymouth.ac.uk
DOI: 10.5210/ojphi.v5i3.4814
Copyright ©2014 the author(s)
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
2 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
Background
Smartphones, the most common “personal computer” today, have revolutionised the
communication landscape. Almost ‘always on’ and highly portable (carried by their users
everywhere they go), smartphones provide real-time, on-demand communication, while their
rich multimedia touch-displays operate with increasing speeds, delivering data services and
computing power to document and improve the networked lives of their owners [1,2].
Communication via smartphones is personalised: smartphones store and exchange large
amounts of personal information and users are able to customise their phones to suit their
personal preferences and needs. A smartphone can record a large number of details about its
user’s current status and whereabouts. It can relay appropriate social support and enable real-
time and asynchronous exchanges with other users via social networks and other forms of
mobile communications. The latter include text messaging (Short Message Service—SMS),
photography (still and video), location and other sensors (global positioning system [GPS],
accelerometers, ambient light sensor, etc.), built-in applications or apps (e-mail, contacts,
calendar, document readers and video players, etc.) and wireless data service [2]. (‘App’,
short for ‘application (program)’, refers to a self-contained piece of software coded for a
specific purpose and usually optimised to run on a mobile device.)
Smartphones out-shipped feature phones worldwide for the first time in Q1 2013 [3,4]. One of
the main differences between smartphones and feature phones is that the latter, besides being
less expensive than the former, offer very limited or no support for third-party, full-fledged
apps. According to the ‘Mobile Health 2012’ report published by Pew Research Centre’s
Internet & American Life Project, 85% of US adults own a cell phone; of them, 53% own
smartphones. Half of smartphone owners use their devices to get health information. One-
fifth of smartphone owners have health apps on their devices [5].
The mobile revolution is offering an unprecedented opportunity to provide medical support
when and where people need it. Large numbers and varieties of medical and health-related
apps exist on the market today. A 2012 estimate puts the number of health-related apps at no
fewer than 40,000 [6]. From basic apps composed of text message reminders to apply
sunscreen, to sophisticated apps that coordinate the management of diabetes, apps play a
multitude of functions in health and healthcare.
Mobile technology has several potential advantages for providing actionable medical advice,
but also has its own limitations and potential problems associated with it. These aspects of
mobile technology will be the focus of the rest of this paper.
Range of mobile applications
Apps for medical providers
Many apps are developed for a target audience of healthcare workers, including physicians,
nurses and assistants. These apps are generally more sophisticated, with medical terminology
and functions, and not easily navigable by non-health professionals. In a study published in
This is an Open Access article. Authors own copyright of their articles appearing in the Online Journal of Public Health Informatics. Readers may copy articles without permission of the copyright owner(s), as long as the author and OJPHI are acknowledged in the copy and the copy is used for educational, not-for-profit purposes.
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
3 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
2012, a group of surveyed healthcare workers indicated the most popular categories of mobile
applications functions include drug-referencing tools, clinical decision-support tools,
communication, electronic health-record system access and medical education materials [7].
The top apps were drug-reference guides such as ‘Epocrates’ (Epocrates, Inc.) [8] and
‘Lexicomp’ (Wolters Kluwer) [9], as well as clinical decision-support reference tools such as
‘UpToDate’ (Wolters Kluwer) [10] and ‘Medscape’ (WebMD LLC) [11].
Specialty or disease-specific apps
Though some apps cover a broad spectrum of general medical knowledge, others may be
tailored to specific specialties such as colorectal disease-themed apps in gastroenterology or
goniometer apps in orthopaedics [12,13]. The ‘Eye Handbook’ (developed at the University of
Missouri Kansas City, USA) is a free ophthalmology-themed app offering mobile diagnostic
visual tests, a user directory of eye-care professionals, ophthalmology specific calculations,
ICD-9 codes and an atlas of common ophthalmic conditions [14,15]. The field of infectious
disease may particularly benefit from the rapid updates of apps which provide news and
updates in real time. ‘Outbreaks Near Me’ (developed at Boston Children’s Hospital, USA)
utilises news media reports, medical e-mail list services and alerts from official national and
international organisations to monitor global infectious diseases via the ‘HealthMap’ database
[16,17].
Medical education and teaching
The younger generation of technologically capable medical professionals in training, such as
students and residents, harness the power of innovative apps to improve learning. The
mobility of a smartphone or tablet allows students to carry a plethora of clinical resources in a
convenient and searchable package. The flexibility of the mobile platform provides
interactivity in teaching and a more personalised education. Given the high rate of mobile
technology adoption among young clinicians, mobile applications display great potential to
augment traditional training. A survey published in 2012 of ACGME (US Accreditation
Council for Graduate Medical Education) programmes demonstrated over 85% of
respondents used a smartphone and over half of respondents used mobile applications on a
daily basis, most commonly drug guides, medical calculators, coding/billing apps and
pregnancy wheels [18]. ‘Bump’ (Bump Technologies, Inc.), a mobile app that transfers
information between two mobile devices when put in very close proximity together,
demonstrated enhancement of pharmacy student learning and communication in patient
simulation scenarios [19]. For new trainee doctors, a mobile app, which functioned as a
portable electronic library, provided a wealth of information when senior or attending
physicians were not available and thus enhanced patient care [20]. German physicians
adapted the German College of General Practitioners and Family Physicians practice
guidelines into an easily accessible app for physicians and medical students [21]. Flashcard
and subject review applications, such as ‘Microbiology and Immunology Wiz’ (Current
Clinical Strategies Publishing) [22] or ‘Lange Microbiology & Infectious Diseases Flash
Cards’ (Modality Inc.) [23], allow for portable and customisable education for medical
students [17]. When provided with access to smartphones, resident physicians in Botswana
effectively utilised several point-of-care mobile applications in delivering healthcare in a
resource-limited setting [24].
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
4 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
Apps for patients and the general public (including health and fitness apps)
On the other end of the spectrum, there are patient-centred apps capable of performing an
equally wide array of functions. Current apps aid patients in managing chronic disease,
lifestyle management, smoking cessation and even self-diagnosis.
The diabetes mellitus epidemic is reflected in the number of apps geared towards diabetic
patients. On the Android platform alone, over 80 diabetes apps offer a variety of functions,
including self-monitoring blood glucose recording, medication or insulin logs, and prandial
insulin dose calculators [25]. Another diabetes intervention app integrated communication
between patients and a healthcare provider. The patient would log fasting blood sugars, daily
eating behaviours, medication compliance, physical activity and emotions into a mobile
online diary. A remote therapist with access to these diaries would then formulate
personalised feedback to the patient [26].
The most number of apps belong to the exercise and weight loss category. The built-in
camera, standard in smartphones today, allows users to record a photo diary of daily food and
drink. These photos may be transferred to a server, which identifies and quantifies the food
portion [27]. A weight loss trial utilised a mobile app to monitor dietary intake, body weight
and objectively-measured physical activity (obtained from a Bluetooth-enabled
accelerometer) of its participants [28].
Kamel Boulos and Yang surveyed dozens of mobile, location-based (outdoor) exergaming
apps that harness the power of sharing through online social networks and gamification
principles on GPS-enabled smartphones. In these apps, the real world becomes the ‘game
map’ or playground, and players can even discover and learn about new places and their
geographies while burning calories and keeping fit [29].
Apps are also developed for smoking cessation and alcohol addiction. At least 47 iPhone
apps for smoking cessation are available [30]. ‘A-CHESS’ (Alcohol Comprehensive Health
Enhancement Support System), a smartphone-based intervention for preventing relapse in
alcoholic dependency harnesses mobile technology to improve treatment and motivation [31].
In patients with chronic disease characterised by life-threatening flares, apps may allow them
to track and even report symptoms. The app ‘m.Carat’ was developed at Faculdade de
Medicina da Universidade do Porto, Portugal, for asthmatic and allergic rhinitis patients to
record their exacerbations, triggers, symptoms, medications, lung function tests and visits to
the doctor or the hospital [32]. Users can also receive disease education, medication
information, task notifications, and synchronise records with an online database to better
control their symptoms [33].
Psychiatric patients benefit from ambulatory monitoring through an app that randomly
prompts the patient to self-report psychotic symptoms multiple times throughout the day [34].
Another app for sickle cell disease patients allows them to access an online diary to record
pain and other symptoms [35]. Monitoring symptoms in patients with COPD (chronic
obstructive pulmonary disease) through a mobile app alerts patients and providers to
suspected disease exacerbations, thereby facilitating prompt intervention [36].
An app developed for patients with dementia, ‘iWander’, assists patients with daily living by
providing audible prompts offering to direct the patient home, sending notifications and GPS
coordinates to caretakers, or by calling local 911 (US emergency) services [37].
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
5 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
Patients may even use apps to attempt self-diagnosis without a medical visit. Patients with a
camera-enabled smartphone can use apps to take photographs of skin lesions and send these
to a remote server for computer analysis and/or review by a board certified dermatologist
[38,39]. Such apps are not without their pitfalls and this will be discussed in detail later in this
paper.
Apps may empower non-medical professionals to provide basic triage at the scene of trauma
such as on the sidelines of a sporting event. An app development team with a neurosurgeon
among its members created ‘Concussion Test’, which follows the standardised and validated
Sport Concussion Assessment Tool 2 (SCAT2). Five similar concussion apps exist for
purchase as well [40].
Mobile app technology has far-reaching potential in the public health domain as well. Apps
may be used to contribute to the care and prevention of sexually transmitted disease (STD). A
study of available apps demonstrated 55 unique mobile apps for HIV (human
immunodeficiency virus) and STD education, prevention, testing and resources [41].
Text messaging
Text messaging, or Short Message Service (SMS), dates back to the early days of mobile
phones, and most applications worked equally well on the simplest mobile phones, the more
advanced feature phones, as well as full-fledged smartphones. Text messaging reminder
applications are easy to implement with tools such as InSTEDD’s ‘Remindem’ (free and
open source) [42]. Due to the pervasive nature of the mobile phone, text-messaging
applications have the unique opportunity to alert patients directly regardless of location (or
availability to take a voice call).
Evaluation of text messaging shows promising results for assisting with clinical monitoring
and counselling, keeping medical appointments, smoking cessation, weight loss, chronic
disease management (e.g., diabetes care monitoring), or reminding people to use sun
protection to prevent sunburns in the near term and skin cancer in the long term [43-46].
Adherence to therapy and disease control may be improved in psoriasis patients who receive
daily educational and motivational text messages [47]. Text message reminders to patients
may also improve antibiotic compliance [48] or even prevent recurrent cardiovascular events
[49].
On the one hand, text messaging provides nearly universal user access, particularly in low-
resource settings, with decreasing costs, simple interventions to develop and use,
customisable content and schedule, and a push-mode delivery that prompts users to read and
possibly respond. On the other hand, text messaging offers limited interaction, with often
only passive engagement, and does not leverage the latest smartphone computing power.
Electronic health records
Mobile apps may also provide access to electronic health records and patient information.
Kharrazi et al. mention 19 apps that allow patients to store personal health records on their
mobile devices [50]. ‘HealthVault’ (Microsoft) acts as a platform for personal health data,
including data from personal health monitoring and fitness devices, putting consumers in
control of their health information, with the ability to securely share it with clinicians,
caregivers, family members, or others, as needed [51]. ‘DocbookMD’, an app meeting
HIPAA (US Health Insurance Portability and Accountability Act) encryption and security
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
6 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
requirements [52], allows physicians to easily transmit text messages and images to one
another [53].
Telemedicine and telehealthcare
The implementation of telemedicine and telehealthcare (including clinical telemonitoring of
patients) through the application of mobile devices is clearly a practical and potentially low-
cost choice in the delivery of healthcare, as seen, for example, in the mobile component of
the CAALYX/eCAALYX (Complete Ambient Assisted Living Experiment/Enhanced
Complete Ambient Assisted Living Experiment) prototype systems [2,54].
Several applications have been described that harness mobile devices and apps to increase
efficiency and access to care, particularly in emergency situations [55]. When time is of the
essence, apps can increase speed and accessibility to critical specialist care in real time, e.g.,
in stroke or acute trauma.
Acute stroke care is made portable and accessible to non-urban centres via real-time video on
smartphones [56]. The ‘i-Stroke’ system was developed to transfer clinical data, computed
tomography (CT), magnetic resonance imaging (MRI), angiographic and intraoperative
images, as well as expert opinion, all in real time [57]. ‘ResolutionMD’ (Calgary Scientific)
[58] is an FDA (US Food and Drug Administration)-cleared teleradiology app incorporated
into a telestroke network, which provides remote vascular neurologists with radiographic
images [59,60].
Acute trauma patients also benefit from timely and efficient management. An iPhone-based
teleradiology program was used for the diagnosis of acute cervical trauma, examining CT
scans to evaluate for the presence of fractures or displacements [61].
Resource-limited settings and remote locations (e.g., distant rural areas and desert
settlements) may benefit from access to specialist care and teleconsultations through mobile
technology, particularly in disciplines with no locally-residing specialists, such as
ophthalmology or dermatology. In one study, the iPhone was used to send fundoscopic
images to board certified ophthalmologists for review to detect diabetic retinopathy [62].
Mobile phone multimedia messaging allowed general practitioners to send teledermatology
referrals in the form of photos and relevant clinical information to specialist dermatologists
for consultation [63].
In some instances, mobile apps may allow telemedicine to replace time-consuming office
visits altogether. This modality may benefit specialties that require frequent follow-up care or
monitoring, such as rehabilitation or post-operative care of patients. A physical therapy app
provided virtual-reality-based balance exercises through a mobile device. Remote
physiotherapists with access to the results could adjust the level of exercises accordingly [64].
Surgeons utilised remote real-time monitoring of free flaps via smartphone photography to
replace in-person examination [65].
Smartphone attachments
Several applications work in conjunction with some specialised phone attachment piece or
wireless sensors in order to perform specialised activities that are not part of a phone’s
standard functions, e.g., the CAALYX/eCAALYX system described in [2,54].
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
7 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
For example, an adaptor with electrocardiogram (ECG) electrodes may transmit electrical
data to detect abnormal heart rhythms in a non-hospital setting [66]. Patients with diabetes
may synchronize a glucometer attachment to their mobile device to track blood glucose and
share the data through an internet connection [67,68]. The next generation of smartphone app
technology may even enable users to perform routine blood testing [69]. Using a Windows
Phone with a micro lens mounted over the camera, the ‘Lifelens’ app captures high resolution
images of cells in a drop of blood and then analyses them to detect to existence of malaria
[70].
Evaluation studies of mobile medical and health-related apps
Ever since the first cellular phone call was placed on 3 April 1973, the mobile age has grown
and continues to do so at an exponential rate, particularly during the 21st century [71]. The
vast array of smartphones, mobile tablets and mobile medical and health-related apps on offer
today (see the app examples presented earlier in this article) provides consumers with an
unprecedented opportunity to achieve their health and healthcare goals, and overcome many
obstacles along the way. However, with such a booming industry also come concerns, risk
and potential dangers. It is perhaps surprising that relatively very little research has been
undertaken so far (as of 2013) to investigate the validity and efficacy of these devices and
apps in the contexts of health and healthcare. Here, we will briefly present several evaluation
and validation studies exploring the diversity of health and healthcare-related apps. The
majority of these studies have been conducted on small patient populations to compare the
efficacy of a smartphone/tablet app versus conventional resources.
Diabetes management
One of the most well studied areas is that of diabetes management. A review conducted by
Demidowich et al. [25] investigated 42 Android apps for diabetes self-management. The
mean composite usability score which evaluated six standard features per app was 11.3 out of
a possible 30, and no apps offered direct data input from glucometers. The study concluded
that surprisingly few apps provided a comprehensive method of diabetes self-management,
but did mention ‘Glucool Diabetes’ (3qubits) [72], ‘OnTrack Diabetes’ (GExperts Inc.) [73],
‘Dbees’ (Freshware.pl) [74] and ‘Track3 Diabetes Planner’ (Coheso, Inc.) [75] as
recommended apps [25].
Medical imaging
Another fascinating area of investigation is that of tablet (iPad [Apple, Inc.]) clinical imaging
apps used to view medical and diagnostic images. Such mobile apps pose particular
advantages in emergency settings. Several studies have been conducted to compare the
efficacy of the iPad to diagnose pulmonary embolism and intracranial haemorrhage versus
conventional Picture Archiving and Communications System (PACS) or liquid-crystal
display (LCD) monitor systems. The studies have found the iPad to be equivalent to
conventional methods, but express the need for conducting further research to examine minor
discrepancies [76-78]. In addition, the iPad is being explored as an aid in laparoscopic training
for residents and for percutaneous kidney access [79,80]. Much of this exploration is described
as ‘case report’ experimental studies rather than full evaluation reports.
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
8 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
Global health and infectious diseases
The need for affordable, reliable and prompt diagnostic and therapeutic measures is
especially evident in the global health infectious disease arena. The iPad has been shown to
be comparable to conventional PACS LCD viewing in the diagnosis of tuberculosis [81]. A
study performed in rural Bangladesh demonstrated that basic mobile phone technology is
both efficient and effective in improving case detection and management of malaria [82].
Mobile phone messaging has been proposed and investigated as a method to improve
medication adherence and communication in HIV management, but a systematic review
published in 2013 by van Veltoven et al. examined 21 studies and determined that there is
limited evidence that mobile phones are efficacious in HIV care [83]. Further studies are
needed to adequately assess this topic.
Pain management using smartphone-based diaries
An innovative and novel use of smartphone technology is in the realm of pain management.
As quoted in a 2011 New York Times interview, Dr Sean Mackey, chief of pain management
at Stanford School of Medicine, USA, explains that “before we did not have good data on
what is the burden of pain in our society…the number of people is more than diabetes, heart
disease, and cancer combined” [84]. Pain is an incredibly diverse and prevalent state that it is
often hard for patients to describe, which makes it even harder for caregivers to diagnose and
treat. Smartphone technology has the potential to revolutionise real-time pain reporting.
In a usability testing study published in 2012, a smartphone-based e-diary was successfully
used by children and adolescents with sickle cell disease to report pain symptoms [34]. A
randomised clinical trial has shown women with chronic widespread pain experience fewer
catastrophising events (rumination, expecting the worst, and feeling helpless) when using
smartphone-based diaries with immediate therapist feedback [85]. Similar studies have shown
successful usability of smartphone pain assessment in wheelchair users and adolescents with
cancer [86,87].
Dermatology
The field of dermatology is also taking advantage of the technological smartphone revolution.
One randomised community trial provided text message reminders to use sunscreen daily as
the intervention (for sun protection, to prevent sunburns in the near term and skin cancer in
the long term), and monitored dispensed sunscreen. Text messages increased sunscreen use,
with greater daily adherence [46].
Text messaging has also been investigated as a tool for improving motivation and treatment
adherence in patients with psoriasis. Following 12 weeks of daily text message
reminders/educational tools, patients demonstrated significantly better improvement of
disease severity and quality of life, with superior adherence to therapy and optimised patient-
physician communication compared to a control group with no text message intervention
[47].
Multimedia (text and photos) messaging (Multimedia Messaging Service[MMS]) has proven
to be a promising tool in teledermatology, where it has been used to send digital photographs
of skin conditions to specialist dermatologists for diagnosis. One study compared MMS
photographs sent to dermatologists at a university hospital versus separate face-to-face visits
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
9 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
in 40 patients. The diagnosis based on multimedia message referral was correct for 78% and
provided management recommendations for 98% of patients. This study employed two
dermatologists for the multimedia message diagnosis and two separate dermatologists for the
face-to-face visits. Interestingly, there was a lower concordance (68%) between the two
dermatologists on the MMS arm of the study compared to 88% between the face-to-face
dermatologists [63].
Another potential use of smartphones in dermatology involves apps targeted for patients. In
particular, apps have been designed to aid patients with suspicious skin lesions to determine
if their lesion is benign or malignant. A study published in 2013 in JAMA Dermatology
analysed four of these apps and discovered that the ability of apps to assess melanoma risk is
highly variable. In fact, three of the four studied apps incorrectly classified 30% or more of
melanomas as ‘unconcerning’. A major conclusion of this study was that extreme caution
should be exercised when consumers use apps to assess their medical risks, since many apps
are subjected to very little or absolutely no regulatory oversight [39]. Such apps might still be
useful, as long as there is an appropriate in-app disclaimer warning users in a clear and
simple language about the app’s diagnostic limitations, the possibility and implications of
‘false negatives’, and that the app should not be taken as a substitute for proper professional
clinician’s evaluation and advice.
Regulatory control and certification of medical and health apps
Studies expressing concerns about existing apps for colorectal diseases, microbiology,
dermatology, asthma, diabetes and opioid converters
A number of recent articles and studies have investigated the potential dangers and safety of
some clinical and health apps aimed at healthcare professionals (but available to all) or aimed
at the general public, and whether (and most importantly how) they should be assessed and
controlled by the US Food and Drug Administration (FDA) and/or other relevant and
corresponding entities in other countries such as the Medicines and Healthcare Products
Regulatory Agency (MHRA) in England [88-91].
For example, O'Neill and Brady [12] recently questioned the reliability of unregulated
medical apps specifically applied towards colorectal diseases. Of a total of 68 individual
colorectal themed apps they surveyed in their study (amongst which five were duplicates),
only 29% had had user satisfaction ratings and 32% had named medical professional
involvement in their development or content. With such a little medical professional
involvement in the design of the majority of these apps, increased regulation of some kind is
definitely required if we were to improve accountability for app content [12].
Similarly, in another study by Visvanathan et al. [92], the accuracy and reliability of the
content of apps used in diagnosis and patient management were called into question. Of 94
microbiology-themed apps they surveyed, only 34% had stated medical professional
involvement. The lack of medical professional involvement in the design of the majority of
these apps again undermines users' ability to be informed regarding app content quality.
Visvanathan et al. conclude by proposing that increased regulatory measures be introduced to
safeguard patient welfare [92].
Ferrero and colleagues [38] investigated the potential danger of clinical dermatology apps
targeting patients, calling for FDA regulation of such apps. They tested ‘Skin Scan’, an app
created to help with the identification and management of skin cancer, against 93 clinical
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
10 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
images from the National Cancer Institute and Fitzpatrick’s Dermatology in General
Medicine, and found only 10.8% (10/93) were rated as high risk melanomas. ‘Skin Scan’ has
since then been renamed to ‘SkinVision’ (SkinVision BV, Netherlands) [93] by its
developers, who have also announced they are planning a clinical trial in Europe to compare
the effectiveness of their app against traditional diagnostic tools [94].
Likewise, Robson et al. [95] and Wolf et al. [39] urged caution when using melanoma risk
analysis and detection apps due to their diagnostic inaccuracy. Huckvale et al. [96] and
McKinstry [97] expressed their concerns about worrying deficiencies in existing asthma self-
management apps, concluding that none combine reliable information and appropriate
supportive tools, and that some are even unsafe. Demidowich et al. [25] described
deficiencies and other issues hindering usability in their reviewed sample of diabetes self-
management apps for Android smartphones. O'Neill and Brady [98] pointed to apps where
inaccuracy or inconsistency could potentially cost lives, citing their experience with opioid
conversion calculators. They examined 23 different opioid conversion medical apps and
found alarming inconsistencies in their outputs.
Apps as a ‘medical device’: regulation in USA and Europe
Through guidance first released in draft form in July 2011, the US FDA defined “a small
subset of mobile medical apps that may impact on the performance or functionality of
currently regulated medical devices and as such, will require FDA oversight” [99,100]. Such
medical apps could present a real risk to patients if the apps do not work as intended. The
FDA has already cleared a handful of mobile medical apps [101] that are either used as an
accessory to an FDA-regulated medical device or transform a mobile platform into a
regulated medical device (e.g., an app that turns a smartphone into an ECG machine).
FDA-approved apps include ‘Mobile MIM’ for iPhone and iPad, a diagnostic radiology app
by Cleveland-based MIM Software Inc. that enables a healthcare professional to view
medical images on an iPad and make a diagnosis [102]. More recently, Intuitive Medical
Technologies in Shreveport, Louisiana, received 510(k) FDA clearance for their ‘iExaminer’
adapter and companion app for the iPhone. The ‘iExaminer’ attachment connects ‘Welch
Allyn’s PanOptic Ophthalmoscope’ to an iPhone, aligning the eye piece of the
ophthalmoscope to the phone’s camera. The combined system and app allows clinicians to
image the eye (fundus exams) and save the images for later review or sharing with colleagues
[103].
In a must-read IEEE Spectrum article published in September 2012, Strickland presented a
list of top-five requests to the FDA that came out of a survey of key industry figures on what
they would like to see in FDA rules for medical apps [104]. The requests deal essentially with
the need to crisply define the boundaries between apps that have to be regulated by the FDA
and apps that do not have to go through the certification process. The surveyed industry
experts called for clarifying the difference between a medical app and a wellness app, as well
as the difference between diagnosing and monitoring; establishing the risk-level threshold for
FDA enforcement; defining the limits of the FDA’s rule on apps that serve as device
accessories; and making a plan for how to handle “modular” apps, i.e., multiple apps
designed to work together (if one of those apps is regulated by the FDA, must all the others
be as well?) [104].
In Europe, ‘ONCOassist’ [105], an Irish app for the iPhone and iPad that contains prognostic
tools and useful calculators for oncologists at the point-of-care (and as such falls within the
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
11 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
definition of a medical device), has received CE certification or Conformité Européenne, a
key mark of a product’s compliance with relevant EU [European Union] legislation [106] in
2013, and displays the ‘CE mark’ on its welcome screen [107].
Indeed, the European Medical Device Directive MDD 93/42/EEC explicitly mentions
‘software’ in its definition of ‘medical device’ [108]. But many apps with dosage calculator
functions currently marketed in the UK still do not carry the CE mark to show that they have
been registered as class I medical devices with the MHRA in England or one of the
corresponding regulatory bodies in other EU countries [109]. The likelihood of an app being
treated as a medical device by the MHRA depends on what the app does and the
corresponding level of patient risk associated with it. High risk apps, e.g., those performing
complex calculations using patient data to aid diagnosis or treatment decisions, can be safely
classified as ‘medical devices’.
According to MHRA, if an app is purely a record archiving and retrieval system (electronic
health records), it is unlikely to be considered a medical device; however, if it includes a
module that interprets data or performs some calculation, then it is likely that this particular
component may be considered a medical device. Decision-support apps are also generally not
considered a medical device by MHRA if they just provide existing (i.e., reference/evidence-
based) information to enable a healthcare professional to make a clinical decision. However,
if the apps perform a calculation or interpret or interpolate data and the clinician does not
review the raw data, then such apps may be considered medical devices for MHRA purposes.
Nevertheless, an app performing simple and straightforward calculations such as BMI (body
mass index) should not be treated as a medical device, but a dosage calculator that
recommends a dose based on individual patient details should be [109,110].
Developers and publishers wanting to have the ‘CE mark’ on their apps need to notify the
MHRA in the UK (or a corresponding agency in other European countries), producing a
‘declaration of conformity’ that includes detailed technical documentation of how their app
design conforms to the Medical Device Directive MDD 93/42/EEC. As part of the technical
documentation, app developers will also need to have undertaken a controlled test and risk
assessment to demonstrate that their app supports and improves upon any existing process
used to present the same information or function. Once all the registration documentation is
ready, developers and/or publishers should submit it to the MHRA with the appropriate
registration fee (£70.00 GBP at time of writing) [109].
NHS (England) online Health Apps Library: a full-fledged ‘vetted app store’ is still a
far distance away
The National Health Service (NHS) in England runs an online Health Apps Library [111],
where it lists and recommends some carefully selected apps such as ‘iBreastCheck’
(Breakthrough Breast Cancer) [112] and the NHS’ own ‘Health Choices’ app [113] (not all of
the listed apps can be considered as medical devices). The NHS Health Apps Library only
provides links to third-party stores hosting the actual apps. Developers can submit their apps
for review and possible listing in the Library.
Visitors can also review and rate apps in the NHS Health Apps Library. Many other app
directory sites (and the app stores run by Amazon, Apple, Google, Microsoft and Research In
Motion [RIM/BlackBerry]) allow users, including healthcare professionals, to review and rate
apps, e.g., the ‘Medical App Journal’ directory [22].
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
12 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
It has been proposed that the NHS should provide its own full-fledged ‘vetted app store’
[114,115]. A proper app store (cf. Apple iTunes Store or Google Play) would accept app
submissions from developers and also deliver those apps (if approved) to users’ devices,
while handling any payments that might be involved (in the case of paid apps). For the NHS,
there would be the additional task of assessing and ensuring the quality of these apps from a
medical/health point of view, e.g., ‘is the medical content or advice offered by an app sound,
safe and up-to-date?’ This is in addition to the technical assessment of submitted apps, to
determine if they behave as intended, without crashing the devices running them, and where
applicable, if they are secure and protect user’s privacy. These are not trivial tasks, and could
prove very demanding and well beyond the remits of the NHS, when one considers the
thousands of medical and health apps that a specialised health apps store would have to deal
with.
Beyond ‘apps as a medical device’: desiderata for successful clinical and health-related
apps
Going outside the rather rigid and narrow limits of the definitions by the FDA and MHRA of
‘software (apps) as a medical device’ and the associated risks to patient safety, there are other
aspects of clinical and health-related apps that deserve much attention. These aspects or
factors are also equally important in the case of health-related and medical apps that are
excluded from conventional (software as a) medical device regulations. These factors affect
an app’s overall fitness for purpose, effectiveness and value-for-money (for paid apps), and
can thus be seen as ‘desiderata or ingredients for app success’.
Medical and health app reviewers, such as the curators of the above mentioned NHS Health
Apps Library, should cover (and perhaps rate in some way) these aspects in any app review
or assessment they undertake (where applicable), to allow end users to make informed
comparisons and decisions about which apps to download and use. Furthermore, we
recommend that these factors should be routinely considered by app developers and
publishers, perhaps in the form of a checklist to be added to their existing quality assurance
(QA) procedures (see also our discussion of Happtique Health App Certification Standards
below), as a kind of industry self-regulation and/or voluntary certification.
These app aspects cover content quality (for health education and reference apps), usability
issues (including ‘cognitive accessibility’ and associated consumers’ general and health
literacy issues), device connectivity standards, as well as a number of other issues that are
fully covered in Happtique Health App Certification Standards [116,117], such as app security
and user privacy.
Regarding content quality, apps need to: (1) provide authorship information, including
detailed information about authors’ affiliations and credentials and about any medical
professional involvement in content preparation; (2) list all references or sources of content
(attribution); (3) fully disclose any app sponsorship or other commercial funding
arrangements, and any potential conflicts of interest; and (4) ensure a balanced, non-biased
coverage of facts and information currency (up-to-datedness). These are the same essential
criteria governing the quality benchmarking of online medical/health-related information
resources and Web sites in general (prior to the apps era) [118,119]. For apps serving medical
images, evaluators additionally need to establish that all the relevant ethical issues, such as
obtaining informed consent from patients to publish their images in a smartphone app, were
duly considered in the design of these apps [120].
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
13 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
But high-quality, evidence-based content alone is of limited value, if presented in a way that
does not adequately match and address the usability, accessibility, readability (reading with
understanding) and health literacy needs of target audiences. An app that is perfectly usable
by a younger person might be very difficult to manipulate by an older or disabled person with
different and unique usability needs related to ageing and/or physical and cognitive
impairment [2,121-126].
App designers and content developers tend to focus on the “more obvious” types of usability
and accessibility (i.e., visual, auditory and motor), and often overlook or give little attention
to the cognitive accessibility aspects of their content and user interfaces, which have to do
with users’ average reading age [127] and general, digital and health literacy levels [128].
Almost half of all Europeans show limited health literacy, according to the European Health
Literacy Survey results published in 2011 [129], and this should be taken into consideration
when designing and evaluating apps intended for consumption by the general public.
Presenting correct, unbiased information but in a way that is hard to understand by the
intended audience not only renders this information useless, but also makes misunderstanding
a likely possibility, which can have serious negative health consequences [128].
For apps that connect to devices, such as glucometers, heart rate and blood pressure monitors,
integration with Continua Health Alliance’s [130] home and mobile telehealthcare ecosystem
of devices from different manufacturers and suppliers would be highly desirable and future-
proof. Frohner et al. [131] describe one such an app, and Microsoft ‘HealthVault’ apps for
Windows Phone and Windows 8/RT [132] already support Continua certified devices [133].
Biometric fingerprint identification is now available on some smartphones and can help
verifying and ensuring the identity of the person or patient using the device or a particular
app running on it. Some apps allow patients to manually log and edit their health and lifestyle
data before submitting these details electronically to their treating clinicians. Problems may
arise in relation to selective and subjective reporting of data by patients and/or patient
compliance in maintaining such manual logs, resulting in incomplete data being submitted to
clinicians. Sensors can automate the logging of some clinical/health data and partially solve
issues related to patient compliance and reporting, but patients may still forget or choose not
to wear the corresponding sensor devices, particularly when perceived as intrusive,
cumbersome and/or threatening their individual privacy. ‘Patient-centred design’ and
appropriate involvement of end users in app and device design, which are now mandatory
requirements explicitly stated in FDA regulation 21 CFR 820.30, can help mitigate these
latter issues.
Happtique Health App Certification
Happtique [134] is a US mobile health (mHealth) solutions company aimed at integrating
mHealth into patient care and daily life. The company released their ‘Happtique Health App
Certification Program’ (HACP) in an attempt to address many of the above discussed app
evaluation criteria, as well as some additional relevant and equally important aspects,
beginning from where FDA (and MHRA) usually leave off. In other words, Happtique’s
scope includes, but also goes beyond, the ‘software as a medical device’ definition used by
FDA and MHRA/‘CE mark’.
HACP is meant to assist healthcare providers and consumers in identifying medical, health
and fitness apps that “deliver credible content, contain safeguards for user data, and function
as described”. The final HACP Certification Standards and Associated Performance
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
14 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
Requirements were released on 27 February 2013. They assess operability, privacy, security
(collectively referred to as the ‘Technical Standards’) and content (‘Content Standards’), and
are available at [116,117,135].
App Operability (OP) Standards (OP1 to OP9) cover issues such as ensuring the app installs,
launches, and runs consistently on target device(s) and operating system(s). HACP has some
very comprehensive privacy requisites, App Privacy (P) Standards (P1 to P6), requiring, for
example, the app to disclose to its users the type(s) and full details of all data it (or any in-app
advertiser) collects or accesses on user devices, either pertaining to the usage of the app
and/or to the specific user, including user-generated data and data that are collected
automatically about the user, and how and by whom all that data are used [135].
App Security (S) Standards (S1 to S7) deal with issues such as verifying that the app,
including without limitation, any advertisement displayed or supported through it, is free of
malware. App Content (C) Standards (C1 to C11) require (among other things) that content
of apps “be written and presented in a manner that is appropriate for the intended audience”.
Content Standards also cover in-app advertisements, demanding that “an app that contains
advertisements clearly identifies the advertising and complies with any and all applicable
regulatory requirements, particularly advertisements that involve or relate to products or
services that are clinical or related to health” [135].
Happtique has partnered with third party organisations to serve as HACP Partners for the
evaluation of apps against the Certification Standards, based upon each organisation’s area of
expertise. The testing of the Technical Standards was assigned to Intertek, a multinational
inspection, product testing and certification company headquartered in London, United
Kingdom [136]. Apps that meet all of the Technical Standards are then evaluated for the
Content Standards by the Association of American Medical Colleges (AAMC) [137] and
CGFNS (Commission on Graduates of Foreign Nursing Schools) International [138], with the
help of clinical specialists selected based on the app’s specific subject matter [117].
HACP remains a voluntary programme. In order to qualify for submission, an app’s content
must be written in English, run natively on iOS (Apple), Android, Blackberry or Windows
devices, be intended for sale or use within the US (Happtique is currently solely focused on
the US market), and pertain to at least one of the following areas: provision of healthcare,
health monitoring and management, and/or advancement of healthcare or medical knowledge.
Apps requiring any public or private certification, registration, clearance or similar approval
(e.g., by FDA) must obtain those approvals prior to submitting an application for Happtique
Certification [117].
Apps successfully meeting all of HACP’s Standards and Associated Performance
Requirements are granted the Happtique Certification and Seal, valid for a two-year period
and specifically associated with the app version that was submitted for evaluation. The app
developer may then use the Seal for promotional, advertising or marketing purposes, as long
as the Certification is valid and such use is in compliance with Happtique’s guidelines. All
active Seals contain a link back to the ‘Happtique Certified App Directory’ that includes
relevant information about the app and its certification history [117] (cf. Health On the Net
Foundation’s HONcode certificate and seal [139].
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
15 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
Conclusions
Apps may have deficiencies and limits. App development, support, maintenance and regular
updating may entail significant costs. Interactions may require substantial effort. Advice may
not align with users’ expectations or life activities. Not all content may benefit all users, and
getting users to download and engage with mobile apps is an art [1]. Health disparities and
low health literacy and numeracy may negatively affect use [128]. Furthermore, many
smartphone apps are not based on behavioural change theories or guidelines (where these
could have made a significant, positive difference, if implemented) [30,44].
Relatively few studies exist on the effectiveness (clinically and cost-wise) of mobile
smartphone apps and more research is needed to properly address this issue. Assessment of
some aspects of specific apps or types of apps may require a full blown clinical trial or
evaluation study and the necessary resources to conduct it, which is well beyond what can be
evaluated by a single person or a few people using a checklist of criteria to look for.
Software applications (apps) and ‘software as a medical device’ are not new concepts. But
the mobile Social Web is now enabling millions of people to more easily share, rate,
recommend, and find software applications about almost any topic under the sun (‘there is an
app for that’). Before the advent of smartphones, small-form-factor tablets and the latest
generations of mobile operating systems and Web browsers that support the concept of apps
and associated ‘app stores’, downloading and installing software (including shareware [cf.
trial versions of paid apps today] and freeware [cf. free apps]) was always possible, but not as
easy or as popular (among average Internet users) as it is today.
However, this ubiquity (ease-of-installation and popularity) of today’s apps is also bringing
in additional risks to less experienced users who might find themselves tricked to download
apps that contain malware, or violate their online privacy, or offer them dubious medical
information and advice, hence the need to educate users, particularly the general public, and
raise their awareness about the potential negative aspects of mobile apps and how to appraise
the quality of an app before installing it and granting it permissions on their devices. Hogan
and Kerin [140] consider it crucial to educate patients regarding the “unregulated and
potentially harmful content of (some) apps”.
Voluntary app certification schemes such as HACP, while very welcome (as standard setters
in the domain), are reminiscent of the early days of the Web, when many “similar” quality
benchmarking tools and codes of conduct for information publishers were proposed and
developed to appraise and rate online medical and health information [141]. Indeed, many
apps on offer today are nothing more than custom informational Web sites displayed in their
own dedicated app screens (custom Web browser screens). It is probably impossible to rate
and police every app on offer today, much like in those early days of the Web, when people
quickly reached the same conclusion regarding Web pages offering medical and health
information. The best first line of defence was, is, and will always be to educate consumers.
Empowered patient groups and consumer leaders can play an important role in this respect, as
seen, for example, in the recent release of the ‘European Directory of Health Apps 2012-
2013’ [142].
Postscript
A new class of wearable smartphone-connected devices is emerging, namely smart watches
(worn like a standard watch around the wrist) and optical head-mounted displays such as
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
16 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
Google Glass (worn like eyeglasses). These devices offer new, unique form factors and
affordances, requiring dedicated apps to properly address the associated usability
opportunities and limitations [143,144]. Smart watches are being used as physical activity
trackers [143], while Google Glass has been experimentally deployed as ‘surgical assistant’
and for medical education purposes [144,145]. As these wearable devices and their
corresponding apps continue to develop and mature, dedicated research and expert reviews
will soon become necessary to investigate and document their various potential and current
clinical and health applications.
Authors' contributions
MNKB conceived and planned the study, conducted the initial PubMed literature survey, and
drafted the manuscript with contributions (mainly to the section entitled ‘Range of mobile
applications’) from ACB, CK, DBB and RPD. All authors have read and approved the final
manuscript.
Competing interests
The authors declare that they have no competing interests. Any reference herein to any
specific commercial apps, devices, other products, or services by trade name, trademark,
manufacturer, provider or otherwise does not necessarily constitute or imply its endorsement,
recommendation or favouring by the authors.
Acknowledgements
Commercial products, services and company/brand names mentioned in this paper are
trademarks and/or registered trademarks of their respective owners.
References
1. Heron KE, Smyth JM. 2010. Ecological momentary interventions: incorporating mobile
technology into psychosocial and health behaviour treatments. Br J Health Psychol.
15(1), 1-39. doi:http://dx.doi.org/10.1348/135910709X466063. PubMed
2. Boulos MN, Wheeler S, Tavares C, Jones R. 2011. How smartphones are changing the
face of mobile and participatory healthcare: an overview, with example from
eCAALYX. Biomed Eng Online. 10, 24. doi:http://dx.doi.org/10.1186/1475-925X-10-
24. PubMed
3. More Smartphones Were Shipped in Q1 2013 Than Feature Phones, An Industry First
According to IDC (Press Release, 25 Apr 2013) -
http://www.idc.com/getdoc.jsp?containerId=prUS24085413
4. Millar M. Market for feature phone apps is low-tech goldmine (BBC News, 7 May 2013)
- http://www.bbc.co.uk/news/business-22390444
5. Fox S, Duggan M. Mobile Health 2012. Washington, DC, Pew Research Center’s
Internet & American Life Project 2012. Available at
http://pewinternet.org/~/media//Files/Reports/2012/PIP_MobileHealth2012_FINAL.pdf
6. Pelletier SG. Explosive Growth in Health Care Apps Raises Oversight Questions. AAMC
Reporter 2012, Oct. Available at
https://www.aamc.org/newsroom/reporter/october2012/308516/health-care-apps.html
7. Conn J. 2012. Most-healthful apps. Mod Healthc. 42(50), 30-32. PubMed
8. Epocrates - http://www.epocrates.com/
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
17 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
9. Lexicomp - http://www.lexi.com/
10. UpToDate - http://www.uptodate.com/
11. Medscape - http://www.medscape.com/public/mobileapp
12. O'Neill S, Brady RR. 2012. Colorectal smartphone apps: opportunities and risks.
Colorectal Dis. 14(9), e530-34. PubMed http://dx.doi.org/10.1111/j.1463-
1318.2012.03088.x
13. Jenny JY. 2013. Measurement of the Knee Flexion Angle with a Smartphone-
Application is Precise and Accurate. J Arthroplasty. 28(5), 784-87.
doi:http://dx.doi.org/10.1016/j.arth.2012.11.013. PubMed
14. Eye Handbook - http://www.eyehandbook.com/
15. Lord K, Shah VA, Krishna R. 2013. The Eye Handbook: a mobile app in ophthalmic
medicine. Mo Med. 110(1), 49-51. PubMed
16. Outbreaks Near Me - http://www.healthmap.org/outbreaksnearme/
17. Oehler RL, Smith K, Toney JF. 2010. Infectious diseases resources for the iPhone. Clin
Infect Dis. 50(9), 1268-74. PubMed http://dx.doi.org/10.1086/651602
18. Franko OI, Tirrell TF. 2012. Smartphone app use among medical providers in ACGME
training programs. J Med Syst. 36(5), 3135-39. PubMed
http://dx.doi.org/10.1007/s10916-011-9798-7
19. Kane-Gill SL, Post JP, Smithburger PL, Seybert AL. 2012. "Bump": using a mobile app
to enhance learning in simulation scenarios. Simul Healthc. 7(5), 326-27. PubMed
http://dx.doi.org/10.1097/SIH.0b013e31825e8bcf
20. Hardyman W, Bullock A, Brown A, Carter-Ingram S, Stacey M. 2013. Mobile
technology supporting trainee doctors' workplace learning and patient care: an
evaluation. BMC Med Educ. 13, 6. PubMed http://dx.doi.org/10.1186/1472-6920-13-6
21. Waldmann UM, Weckbecker K: Smartphone Application of Primary Care Guidelines
used in Education of Medical Students. GMS Zeitschrift fur medizinische Ausbildung
2013, 30(1):Doc6.
22. Microbiology and Immunology Wiz -
http://www.medicalappjournal.com/app_details.php?appsname=Microbiology+and+Imm
unology+Wiz+2013
23. Lange Microbiology & Infectious Diseases Flash Cards, 2nd Edition, by Modality Inc.
(iTunes store) - https://itunes.apple.com/gb/app/lange-microbiology-
infectious/id496299826?mt=8
24. Chang AY, Ghose S, Littman-Quinn R, Anolik RB, Kyer A, et al. 2012. Use of mobile
learning by resident physicians in Botswana. Telemed J E Health. 18(1), 11-13. PubMed
http://dx.doi.org/10.1089/tmj.2011.0050
25. Demidowich AP, Lu K, Tamler R, Bloomgarden Z. 2012. An evaluation of diabetes self-
management applications for Android smartphones. J Telemed Telecare. 18(4), 235-38.
PubMed http://dx.doi.org/10.1258/jtt.2012.111002
26. Nes AA, van Dulmen S, Eide E, Finset A, Kristjansdottir OB, et al. 2012. The
development and feasibility of a web-based intervention with diaries and situational
feedback via smartphone to support self-management in patients with diabetes type 2.
Diabetes Res Clin Pract. 97(3), 385-93. PubMed
http://dx.doi.org/10.1016/j.diabres.2012.04.019
27. Schap TE, Zhu F, Delp EJ, Boushey CJ. 2013. Merging dietary assessment with the
adolescent lifestyle. J Hum Nutr Diet. doi:10.1111/jhn.12071. PubMed
28. Pellegrini CA, Duncan JM, Moller AC, Buscemi J, Sularz A, et al. 2012. Study protocol:
A smartphone-supported weight loss program: design of the ENGAGED randomized
controlled trial. BMC Public Health. 12, 1041. PubMed http://dx.doi.org/10.1186/1471-
2458-12-1041
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
18 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
29. Boulos MN, Yang SP. 2013. Exergames for health and fitness: the roles of GPS and
geosocial apps. Int J Health Geogr. 12, 18. doi:http://dx.doi.org/10.1186/1476-072X-12-
18. PubMed
30. Abroms LC, Padmanabhan N, Thaweethai L, Phillips T. 2011. iPhone apps for smoking
cessation: a content analysis. Am J Prev Med. 40(3), 279-85 PubMed
http://dx.doi.org/10.1016/j.amepre.2010.10.032
31. McTavish FM, Chih MY, Shah D, Gustafson DH. 2012. How Patients Recovering From
Alcoholism Use a Smartphone Intervention. J Dual Diagn. 8(4), 294-304. PubMed
http://dx.doi.org/10.1080/15504263.2012.723312
32. m.Carat -
http://www.caratnetwork.org/index.php?option=com_content&view=article&id=67&lan
g=en
33. Burnay E, Cruz-Correia R, Jacinto T, Sousa AS, Fonseca J. 2013. Challenges of a mobile
application for asthma and allergic rhinitis patient enablement-interface and
synchronization. Telemed J E Health. 19(1), 13-18. PubMed
http://dx.doi.org/10.1089/tmj.2012.0020
34. Palmier-Claus JE, Ainsworth J, Machin M, Barrowclough C, Dunn G, et al. 2012. The
feasibility and validity of ambulatory self-report of psychotic symptoms using a
smartphone software application. BMC Psychiatry. 12, 172. PubMed
http://dx.doi.org/10.1186/1471-244X-12-172
35. Jacob E, Stinson J, Duran J, Gupta A, Gerla M, et al. 2012. Usability testing of a
Smartphone for accessing a web-based e-diary for self-monitoring of pain and symptoms
in sickle cell disease. J Pediatr Hematol Oncol. 34(5), 326-35. PubMed
http://dx.doi.org/10.1097/MPH.0b013e318257a13c
36. Johnston NW, Lambert K, Hussack P, Gerhardsson de Verdier M, Higenbottam T, et al.
2013. Detection of COPD Exacerbations and Compliance with Patient Reported Daily
Symptom Diaries Using a BlackBerry-Based Information System. Chest.
doi:http://dx.doi.org/10.1378/chest.12-2308.
37. Sposaro F, Danielson J, Tyson G: iWander: An Android application for dementia
patients. Conf Proc IEEE Eng Med Biol Soc 2010, 2010:3875-3878.
38. Ferrero NA, Morrell DS, Burkhart CN. 2013. Skin scan: a demonstration of the need for
FDA regulation of medical apps on iPhone. J Am Acad Dermatol. 68(3), 515-16.
PubMed http://dx.doi.org/10.1016/j.jaad.2012.10.045
39. Wolf JA, Moreau J, Akilov O, Patton T, English JC, et al. 2013. Diagnostic Inaccuracy
of Smartphone Applications for Melanoma Detection. JAMA Dermatol. 149(4), 422-26.
PubMed http://dx.doi.org/10.1001/jamadermatol.2013.2382
40. Curaudeau GA, Sharma N, Rovin RA. 2011. Development of an iPhone application for
sideline concussion testing. Neurosurg Focus. 31(5), E4. PubMed
http://dx.doi.org/10.3171/2011.8.FOCUS11186
41. Muessig KE, Pike EC, Legrand S, Hightow-Weidman LB. 2013. Mobile phone
applications for the care and prevention of HIV and other sexually transmitted diseases: a
review. J Med Internet Res. 15(1), e1. PubMed http://dx.doi.org/10.2196/jmir.2301
42. InSTEDD Remindem - http://remindem.instedd.org/en
43. Fjeldsoe BS, Marshall AL, Miller YD. 2009. Behavior change interventions delivered by
mobile telephone short-message service. Am J Prev Med. 36(2), 165-73.
doi:http://dx.doi.org/10.1016/j.amepre.2008.09.040. PubMed
44. Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, et al. 2011. Health behavior
models in the age of mobile interventions: are our theories up to the task? Transl Behav
Med. 1(1), 53-71. PubMed http://dx.doi.org/10.1007/s13142-011-0021-7
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
19 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
45. Klasnja P, Pratt W. 2012. Healthcare in the pocket: mapping the space of mobile-phone
health interventions. J Biomed Inform. 45(1), 184-98.
doi:http://dx.doi.org/10.1016/j.jbi.2011.08.017. PubMed
46. Armstrong AW, Watson AJ, Makredes M, Frangos JE, Kimball AB, et al. 2009. Text-
message reminders to improve sunscreen use: a randomized, controlled trial using
electronic monitoring. Arch Dermatol. 145(11), 1230-36.
doi:http://dx.doi.org/10.1001/archdermatol.2009.269. PubMed
47. Balato N, Megna M, Di Costanzo L, Balato A, Ayala F. 2013. Educational and
motivational support service: a pilot study for mobile-phone-based interventions in
patients with psoriasis. Br J Dermatol. 168(1), 201-05. PubMed
http://dx.doi.org/10.1111/j.1365-2133.2012.11205.x
48. Suffoletto B, Calabria J, Ross A, Callaway C, Yealy DM. 2012. A mobile phone text
message program to measure oral antibiotic use and provide feedback on adherence to
patients discharged from the emergency department. Acad Emerg Med. 19(8), 949-58.
PubMed http://dx.doi.org/10.1111/j.1553-2712.2012.01411.x
49. Redfern J, Thiagalingam A, Jan S, Whittaker R, Hackett M, Mooney J, De Keizer L,
Hillis G, Chow C: Development of a set of mobile phone text messages designed for
prevention of recurrent cardiovascular events. Eur J Prev Cardiol 2012, May.
doi:10.1177/2047487312449416.
50. Kharrazi H, Chisholm R, VanNasdale D, Thompson B. 2012. Mobile personal health
records: an evaluation of features and functionality. Int J Med Inform. 81(9), 579-93.
PubMed http://dx.doi.org/10.1016/j.ijmedinf.2012.04.007
51. Microsoft HealthVault - https://www.healthvault.com/gb/en
52. DocbookMD - http://docbookmd.com/
53. Conde C. 2012. An apt app. Tex Med. 108(11), 53-56. PubMed
54. Rocha A, Martins A, Freire JC. 2011. Junior, Kamel Boulos MN, Vicente ME, Feld R,
van de Ven P, Nelson J, Bourke A, Olaighin G, Sdogati C, Jobes A, Narvaiza L,
Rodríguez-Molinero A: Innovations in health care services: The CAALYX system. Int J
Med Inform. doi:10.1016/j.ijmedinf.2011.03.003.
55. Kamel Boulos MN. Quo vadis smartphones and apps in emergency medicine?
International Hospital and Equipment (IHE) 2012, 38:14-15. http://www.ihe-
online.com/index.php?id=3315
56. Demaerschalk BM, Vegunta S, Vargas BB, Wu Q, Channer DD, et al. 2012. Reliability
of real-time video smartphone for assessing National Institutes of Health Stroke Scale
scores in acute stroke patients. Stroke. 43(12), 3271-77. PubMed
http://dx.doi.org/10.1161/STROKEAHA.112.669150
57. Takao H, Murayama Y, Ishibashi T, Karagiozov KL, Abe T. 2012. A new support
system using a mobile device (smartphone) for diagnostic image display and treatment of
stroke. Stroke. 43(1), 236-39. PubMed
http://dx.doi.org/10.1161/STROKEAHA.111.627943
58. ResolutionMD - http://www.calgaryscientific.com/resolutionmd/mobile-resmd/
59. Demaerschalk BM, Vargas JE, Channer DD, Noble BN, Kiernan TE, et al. 2012.
Smartphone teleradiology application is successfully incorporated into a telestroke
network environment. Stroke. 43(11), 3098-101. PubMed
http://dx.doi.org/10.1161/STROKEAHA.112.669325
60. Mitchell JR, Sharma P, Modi J, Simpson M, Thomas M, et al. 2011. A smartphone
client-server teleradiology system for primary diagnosis of acute stroke. J Med Internet
Res. 13(2), e31. PubMed http://dx.doi.org/10.2196/jmir.1732
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
20 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
61. Modi J, Sharma P, Earl A, Simpson M, Mitchell JR, et al. 2010. iPhone-based
teleradiology for the diagnosis of acute cervico-dorsal spine trauma. Can J Neurol Sci.
37(6), 849-54. PubMed
62. Kumar S, Wang EH, Pokabla MJ, Noecker RJ. 2012. Teleophthalmology assessment of
diabetic retinopathy fundus images: smartphone versus standard office computer
workstation. Telemed J E Health. 18(2), 158-62. PubMed
http://dx.doi.org/10.1089/tmj.2011.0089
63. Börve A, Holst A, Gente-Lidholm A, Molina-Martinez R, Paoli J. 2012. Use of the
mobile phone multimedia messaging service for teledermatology. J Telemed Telecare.
18(5), 292-96. PubMed http://dx.doi.org/10.1258/jtt.2012.120206
64. Krpič A, Savanovic A, Cikajlo I. 2013. Telerehabilitation: remote multimedia-supported
assistance and mobile monitoring of balance training outcomes can facilitate the clinical
staff's effort. Int J Rehabil Res. 36(2), 162-71.
doi:http://dx.doi.org/10.1097/MRR.0b013e32835dd63b. PubMed
65. Engel H, Huang JJ, Tsao CK, Lin CY, Chou PY, et al. 2011. Remote real-time
monitoring of free flaps via smartphone photography and 3G wireless Internet: a
prospective study evidencing diagnostic accuracy. Microsurgery. 31(8), 589-95. PubMed
http://dx.doi.org/10.1002/micr.20921
66. Lau JK, Lowres N, Neubeck L, Brieger DB, Sy RW, et al. 2013. iPhone ECG application
for community screening to detect silent atrial fibrillation: A novel technology to prevent
stroke. Int J Cardiol. 165(1), 193-94. doi:http://dx.doi.org/10.1016/j.ijcard.2013.01.220.
PubMed
67. Kamel Boulos MN. Smartphones in diabetes care and management. CyberTherapy &
Rehabilitation (C&R) Magazine 2011, 4(3):20-21.
http://www.cybertherapyandrehabilitation.com/past-issues/2011/issue-3-
2011/smartphones-in-diabetes-care-and-management/
68. Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. 2012. Design of an
mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J Med
Internet Res. 14(3), e70. PubMed http://dx.doi.org/10.2196/jmir.2058
69. Potticary J. 2012. Smartphone apps set to provide next generation of blood-testing
technology. Bioanalysis. 4(19), 2326. PubMed
70. Clayton S. Lifelens is testing for Malaria – on a Windows Phone (Next at Microsoft, 2
Aug 2011) - http://blogs.technet.com/b/next/archive/2011/08/02/lifelens-is-testing-for-
malaria-on-a-windows-phone.aspx
71. Ozdalga E, Ozdalga A, Ahuja N. 2012. The smartphone in medicine: a review of current
and potential use among physicians and students. J Med Internet Res. 14(5), e128.
PubMed http://dx.doi.org/10.2196/jmir.1994
72. Glucool - Diabetes management for Android - http://www.glucool.com/
73. OnTrack Diabetes - Android Apps on Google Play -
https://play.google.com/store/apps/details?id=com.gexperts.ontrack&hl=en
74. Diabetes under control - dbees - http://dbees.com/
75. Track3 Diabetes Planner - http://www.track3.com/android_diabetes_logbook/
76. Johnson PT, Zimmerman SL, Heath D, Eng J, Horton KM, et al. 2012. The iPad as a
mobile device for CT display and interpretation: diagnostic accuracy for identification of
pulmonary embolism. Emerg Radiol. 19(4), 323-27.
doi:http://dx.doi.org/10.1007/s10140-012-1037-0. PubMed
77. Park JB, Choi HJ, Lee JH, Kang BS. 2013. An Assessment of the iPad 2 as a CT
Teleradiology Tool Using Brain CT with Subtle Intracranial Hemorrhage Under
Conventional Illumination. J Digit Imaging. doi:http://dx.doi.org/10.1007/s10278-013-
9580-0. PubMed
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
21 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
78. Mc Laughlin P, Neill SO, Fanning N, Mc Garrigle AM, Connor OJ, et al. 2012.
Emergency CT brain: preliminary interpretation with a tablet device: image quality and
diagnostic performance of the Apple iPad. Emerg Radiol. 19(2), 127-33.
doi:http://dx.doi.org/10.1007/s10140-011-1011-2. PubMed
79. Bahsoun AN, Malik MM, Ahmed K, El-Hage O, Jaye P, et al. 2013. Tablet based
simulation provides a new solution to accessing laparoscopic skills training. J Surg Educ.
70(1), 161-63. doi:http://dx.doi.org/10.1016/j.jsurg.2012.08.008. PubMed
80. Rassweiler JJ, Müller M, Fangerau M, Klein J, Goezen AS, et al. 2012. iPad-assisted
percutaneous access to the kidney using marker-based navigation: initial clinical
experience. Eur Urol. 61(3), 628-31. doi:http://dx.doi.org/10.1016/j.eururo.2011.12.024.
PubMed
81. Abboud S, Weiss F, Siegel E, Jeudy J. 2013. TB or Not TB: interreader and intrareader
variability in screening diagnosis on an iPad versus a traditional display. J Am Coll
Radiol. 10(1), 42-44. doi:http://dx.doi.org/10.1016/j.jacr.2012.07.019. PubMed
82. Prue CS, Shannon KL, Khyang J, Edwards LJ, Ahmed S, et al. 2013. Mobile phones
improve case detection and management of malaria in rural Bangladesh. Malar J. 12, 48.
doi:http://dx.doi.org/10.1186/1475-2875-12-48. PubMed
83. Velthoven MH, Brusamento S, Majeed A, Car J. 2013. Scope and effectiveness of
mobile phone messaging for HIV/AIDS care: a systematic review. Psychol Health Med.
18(2), 182-202. doi:http://dx.doi.org/10.1080/13548506.2012.701310. PubMed
84. Parker-Pope T. Giving Chronic Pain a Medical Platform of Its Own (18 Jul 2011) -
http://well.blogs.nytimes.com/2011/07/18/giving-chronic-pain-a-medical-platform-of-its-
own
85. Kristjánsdóttir OB, Fors EA, Eide E, Finset A, Stensrud TL, et al. 2013. A smartphone-
based intervention with diaries and therapist-feedback to reduce catastrophizing and
increase functioning in women with chronic widespread pain: randomized controlled
trial. J Med Internet Res. 15(1), e5. doi:http://dx.doi.org/10.2196/jmir.2249. PubMed
86. Spyridonis F, Gronli TM, Hansen J, Ghinea G: Evaluating the usability of a virtual
reality-based Android application in managing the pain experience of wheelchair users.
Conf Proc IEEE Eng Med Biol Soc 2012, 2012:2460-3.
doi:10.1109/EMBC.2012.6346462.
87. Stinson JN, Jibb LA, Nguyen C, Nathan PC, Maloney AM, et al. 2013. Development and
testing of a multidimensional iPhone pain assessment application for adolescents with
cancer. J Med Internet Res. 15(3), e51. doi:http://dx.doi.org/10.2196/jmir.2350. PubMed
88. Anonymous. 2012. Health apps and safety: views from recent sources. Health Devices.
41(10), 330-31. PubMed
89. Buijink AW, Visser BJ, Marshall L: Medical apps for smartphones: lack of evidence
undermines quality and safety. Evid Based Med 2012, ebmed-2012-100885 (Published
Online First: 25 Aug 2012). doi:10.1136/eb-2012-100885.
90. Cummings E, Borycki EM, Roehrer E. 2013. Issues and considerations for healthcare
consumers using mobile applications. Stud Health Technol Inform. 183, 227-31. PubMed
91. McCartney M: How do we know whether medical apps work? BMJ 2013, 346:f1811
(correction in BMJ 2013, 346:f1974).
92. Visvanathan A, Hamilton A, Brady RR. 2012. Smartphone apps in microbiology--is
better regulation required? Clin Microbiol Infect. 18(7), E218-20. PubMed
http://dx.doi.org/10.1111/j.1469-0691.2012.03892.x
93. SkinVision - https://skinvision.com/
94. Weaver C. Apps Aim to Detect Skin Cancer (The Wall Street Journal (WSJ), 16 Jan
2013) -
http://online.wsj.com/article/SB10001424127887323783704578245973988828066.html
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
22 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
95. Robson Y, Blackford S, Roberts D. 2012. Caution in melanoma risk analysis with
smartphone application technology. Br J Dermatol. 167(3), 703-04.
doi:http://dx.doi.org/10.1111/j.1365-2133.2012.11046.x. PubMed
96. Huckvale K, Car M, Morrison C, Car J. 2012. Apps for asthma self-management: a
systematic assessment of content and tools. BMC Med. 10, 144.
doi:http://dx.doi.org/10.1186/1741-7015-10-144. PubMed
97. McKinstry B: Currently available smartphone apps for asthma have worrying
deficiencies. Evid Based Med 2013, Feb (Published Online First: 16 Feb 2013).
doi:10.1136/eb-2012-101194.
98. O'Neill S, Brady RR. 2013. Clinical involvement and transparency in medical apps; not
all apps are equal. Colorectal Dis. 15(1), 122. doi:http://dx.doi.org/10.1111/codi.12086.
PubMed
99. US Food and Drug Administration. (FDA): Mobile Medical Applications (Last Updated:
22 Dec 2011) -
http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/ConnectedHealth/
MobileMedicalApplications/default.htm
100. US Food and Drug Administration. (FDA): Mobile Medical Applications: Draft
Guidance for Industry and Food and Drug Administration Staff. Silver Spring, MD: US
Department of Health & Human Services; 2011 (July). Available at
http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/Guidanc
eDocuments/UCM263366.pdf
101. Cain M. 2012. One company’s experience: blazing the trail with the first FDA-approved
medical imaging app. Biomed Instrum Technol. 46(Suppl), 87-90. PubMed
http://dx.doi.org/10.2345/0899-8205-46.s2.87
102. US Food and Drug Administration. (FDA): FDA clears first diagnostic radiology
application for mobile devices--Provides wireless access to medical images for iPhone,
iPad users (4 Feb 2011) -
http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/2011/ucm242295.htm
103. FDA approves Welch Allyn foray into iPhone imaging--iExaminer iPhone PanOptic
Ophthalmoscope (24 Jan 2013) - http://www.medgadget.com/2013/01/iexaminer-iphone-
adapter-for-welch-allyns-panoptic-ophthalmoscope-cleared-in-u-s.html
104. Strickland E. The FDA Takes On Mobile Health Apps--Regulation is coming soon to
health apps. Here’s what to expect. IEEE Spectrum 2012, Sep 12: online -
http://spectrum.ieee.org/biomedical/devices/the-fda-takes-on-mobile-health-apps
105. ONCOassist - http://oncoassist.com/
106. European Commission. Enterprise and Industry: CE marking -
http://ec.europa.eu/enterprise/policies/single-market-goods/cemarking/index_en.htm
107. Irish app certified by CE as a medical device (6 Mar 2013) -
http://www.siliconrepublic.com/innovation/item/31753-medtech
108. Medical Device Directive. (93/42/EEC) - EUR-Lex - http://eur-
lex.europa.eu/LexUriServ/LexUriServ.do?uri=CONSLEG:1993L0042:20071011:en:PD
F
109. When is an app classed as a medical device? (Mar 2013) -
http://www.geneticdigital.co.uk/2013/03/when-should-an-app-be-classed-as-a-device/
110. Medicines and Healthcare Products Regulatory Agency (MHRA). Medical Device
Technology Forum on the use of software as a medical device (12 May 2010) -
http://www.mhra.gov.uk/Howweregulate/NewTechnologiesForums/DevicesNewTechnol
ogyForum/Forums/CON084987
111. NHS Health Apps Library - http://apps.nhs.uk/
112. iBreastCheck (NHS page) - http://apps.nhs.uk/app/ibreastcheck/
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
23 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
113. NHS Health Choices app - http://apps.nhs.uk/app/health-choices/
114. Read C: Efficiency. The NHS app store. Health Serv J 2012, 122(6332): Suppl 2-5.
115. HANDI (The Healthcare App Network for Development and Innovation). Proposed NHS
App Store (18 Oct 2012) - http://handihealth.org/proposed-nhs-app-store/
116. Horowitz BT. Happtique Publishes Final Standards for Mobile Health App Certification
(4 Mar 2013) - http://www.eweek.com/mobile/happtique-publishes-final-standards-for-
mobile-health-app-certification/
117. Happtique: Happtique Health App Certification Standards (Final, 27 Feb 2013) -
http://www.happtique.com/app-certification/ - Direct URL to PDF document (October
2013): http://www.happtique.com/docs/HACP_Certification_Standards.pdf
118. Kamel Boulos MN. On quality benchmarking of online medical/health-related
information resources (March 2004) - http://healthcybermap.org/MNKB_Quality.PDF
119. Wac K. 2012. Smartphone as a personal, pervasive health informatics services platform:
literature review. Yearb Med Inform. 7(1), 83-93. PubMed
120. Payne KF, Tahim A, Goodson AM, Delaney M, Fan K. 2012. A review of current
clinical photography guidelines in relation to smartphone publishing of medical images.
J Vis Commun Med. 35(4), 188-92. doi:10.3109/17453054.2012.747174. PubMed
121. Caprani N, O’Connor NE, Gurrin C. Chapter 5: Touch Screens for the Older User. In
Assistive Technologies. Edited by Auat Cheein FA. Intech; 2012:95-118 (ISBN 978-953-
51-0348-6). Available at http://www.intechopen.com/books/assistive-
technologies/touch-screens-for-the-older-user
122. Guillaume L, Nadine V. Influence of age and interaction complexity on touch screen. In
Proceedings of 2010 12th IEEE International Conference on e-Health Networking
Applications and Services (Healthcom): 1-3 July 2010; Lyon, France. 2010:246-253.
doi:10.1109/HEALTH.2010.5556560.
123. Phiriyapokanon T. Is a big button interface enough for elderly users?: Towards user
interface guidelines for elderly users. Lambert Academic Publishing; 2011 (ISBN-13:
978-3845421667).
124. Al-Razgan MS, Al-Khalifa HS, Al-Shahrani MD, AlAjmi HH. Touch-Based Mobile
Phone Interface Guidelines and Design Recommendations for Elderly People: A Survey
of the Literature. In Neural Information Processing (19th International Conference,
ICONIP 2012: 12-15 November 2012; Doha, Qatar), Proceedings, Part IV - Lecture
Notes in Computer Science. Springer; 2012, 7666:568-574. doi:10.1007/978-3-642-
34478-7_69.
125. Mertens A, Hurtmanns J, Wacharamanotham C, Kronenbürger M, Borchers J, et al.
2012. Swabbing: touchscreen-based input technique for people with hand tremor. Work.
41(Suppl 1), 2405-11. doi:10.3233/WOR-2012-0474-2405. PubMed
126. Zhou J, Patrick Rau P-L, Salvendy G. 2012. Use and Design of Handheld Computers for
Older Adults: A Review and Appraisal. Int J Hum Comput Interact. 28(12), 799-826.
doi:http://dx.doi.org/10.1080/10447318.2012.668129.
127. Boulos MN. 2005. British Internet-derived Patient Information on Diabetes Mellitus: Is it
Readable? Diabetes Technol Ther. 7(3), 528-35.
doi:http://dx.doi.org/10.1089/dia.2005.7.528. PubMed
128. Kamel Boulos MN. Social media and mobile health (Chapter 13, under Part B: Taking
action to create and strengthen health literacy-friendly settings). In: Ilona Kickbusch,
Jürgen Pelikan, Franklin Apfel and Agis Tsouros (Editors). THE SOLID FACTS - Health
Literacy. Copenhagen, Denmark: World Health Organization Regional Office for
Europe, 2013, pp. 63-67. Available at
http://www.euro.who.int/__data/assets/pdf_file/0008/190655/e96854.pdf - URL of
unabridged chapter version: http://ow.ly/kpn43
Mobile medical and health apps: state of the art, concerns, regulatory control and certification
24 Online Journal of Public Health Informatics * ISSN 1947-2579 * http://ojphi.org * 5(3):e229, 2014
OJPHI
129. Almost half of Europeans possess limited health literacy (Maastricht University - press
release, 22 Nov 2011) -
http://www.maastrichtuniversity.nl/web/Main/Sitewide/PressRelease/AlmostHalfOfEuro
peansPossessLimitedHealthLiteracy.htm
130. Continua Health Alliance - http://www.continuaalliance.org/
131. Frohner M, Urbauer P, Forjan M, Pohn B, Gerbovics F, et al. 2012. Development of an
Android App in Compliance with the Continua Health Alliance Design Guidelines for
Medical Device Connectivity in mHealth. Biomed Tech (Berl). 57(Suppl 1.).
doi:10.1515/bmt-2012-4203. PubMed
132. Microsoft HealthVault. (Windows 8/RT) - http://apps.microsoft.com/windows/en-
US/app/healthvault/728f1c88-7e2f-4b40-95c1-74fc09983689
133. Van Hoof B. Connecting Devices to HealthVault (June 2009) -
http://download.microsoft.com/download/0/4/9/0498cecf-d0b1-4a72-b9b7-
17eb7d7ada98/Van_Hoof_Connecting_Devices.pdf
134. Happtique - http://www.happtique.com/
135. Happtique: Health App Certification Program Standards & Performance Requirements -
A Side-By-Side Comparison Of Published Draft And Final Standards/Performance
Requirements (27 Feb 2013) - http://www.happtique.com/wp-
content/uploads/Happtique-Side-by-side.pdf
136. Intertek - http://www.intertek.com/
137. Association of American Medical Colleges (AAMC) - https://www.aamc.org/
138. CGFNS International - http://cgfns.org/
139. HONcode certificate & seal -
http://www.hon.ch/HONcode/Patients/sample_certificate.html
140. Hogan NM, Kerin MJ. 2012. Smart phone apps: smart patients, steer clear. Patient Educ
Couns. 89(2), 360-61. PubMed http://dx.doi.org/10.1016/j.pec.2012.07.016
141. Boulos MN, Roudsari AV, Gordon C, Muir Gray JA. 2001. The use of quality
benchmarking in assessing web resources for the dermatology virtual branch library of
the National electronic Library for Health (NeLH). J Med Internet Res. 3(1), e5.
doi:http://dx.doi.org/10.2196/jmir.3.1.e5. PubMed
142. PatientView. European Directory of Health Apps 2012-2013 - A review by patient
groups and empowered consumers. London, UK: PatientView; 2012. Available at
http://www.patient-
view.com/uploads/6/5/7/9/6579846/pv_appdirectory_final_web_300812.pdf
143. Google. + posts by MN Kamel Boulos on smart watches -
https://plus.google.com/u/0/100165894171992699139/posts/ir1iRBjbm4t
144. Google. + posts by MN Kamel Boulos on Google Glass -
https://plus.google.com/u/0/100165894171992699139/posts/7Y1G13Dphrg
145. Vallurupalli S, Paydak H, Agarwal SK, Agrawal M, Assad-Kottner C. 2013. Wearable
technology to improve education and patient outcomes in a cardiology fellowship
program - a feasibility study. Health Technol. 3(4), 267-70.
doi:http://dx.doi.org/10.1007/s12553-013-0065-4.
Recommended