MSc Biomedical Sciences Track: Major Science in Society

User Experiences with YAPILI: A telemedicine pilot for sub-Saharan Africa

by

Sonia Jain Student ID: 2605292 Date: 30th June, 2017

VU supervisor: On-site supervisor: Tomris Cesuroglu Guus ten Asbroek (RN/PhD) (MD/PhD) Sherzel Smith (MSc) Amsterdam Health and Technology Institute (AHTI), Amsterdam

PAGE 1

ACKNOWLEDGEMENT

I would like to take this opportunity to thank my University supervisor and examiner Tomris Cesurogulu (MD/PhD) for her continued support and guidance throughout this internship. I would also like to express my humble gratitude to my on-site supervisors Guus ten Asbroek (RN/PhD) and Sherzel Smith (MSc) for giving me this opportunity to intern with YAPILI and for their constant support, guidance and motivation to complete this research. Both of them consistently allowed this report to be my own, but steered me in the right direction whenever they thought I needed it. This research could not have been completed without them.

I would like to thank the interviewees for their time and valuable feedback. I would also like to thank the entire YAPILI team for all their inputs during the internship. Last but not the least, I would like to express my profound gratitude to my mother and brother for providing me with unfailing support and continuous encouragement throughout my years of study and through the process of researching and writing this report. This accomplishment would not have been possible without them.

PAGE 2

Table of Contents Summary/Abstract .................................................................................................................... 4

List of abbreviations .................................................................................................................. 5

Chapter 1. Introduction .............................................................................................................6

Chapter 2. Contextual background ..........................................................................................9

2.1 Sub-Saharan Africa ...........................................................................................................9

2.2 Origin and definition of telemedicine ............................................................................9

2.3 Telemedicine, ehealth, telehealth, telecare and mhealth ............................................ 10

2.4 Types of telemedicine applications ............................................................................... 11

2.5 Advantages of telemedicine ........................................................................................... 11

2.6 Expansion of telehealth/telemedicine initiatives in SSA .............................................. 11

2.7 YAPILI as a telemedicine platform ................................................................................ 12

2.8 Why evaluate user experiences? .................................................................................... 13

Chapter 3. Theoretical background ........................................................................................ 14

3.1 Theories for evaluating health care ............................................................................... 14

3.2 Theories for evaluating information systems ............................................................... 14

3.3 Theories for information system in healthcare fields ................................................... 15

3.4 Conceptual Framework ................................................................................................. 16

Chapter 4. Methodology ......................................................................................................... 18

4.1 Research design .............................................................................................................. 18

4.2 Study population ........................................................................................................... 18

4.3 Data collection ............................................................................................................... 18

4.4 Data-analysis ................................................................................................................. 19

4.5 Research quality ............................................................................................................ 20

Chapter 5. Results .................................................................................................................... 21

5.1 System, information and service quality ....................................................................... 22

5.2 Use .................................................................................................................................. 26

5.3 User Satisfaction ............................................................................................................ 27

5.4 Quality of care, Access and Productivity ...................................................................... 28

5.5 Prioritisation exercise ................................................................................................... 32

PAGE 3

Chapter 6. Discussion ............................................................................................................. 34

6.1 Difference in doctor and patient needs ........................................................................ 34

6.2 Findings addressed by the framework.......................................................................... 35

6.3 Findings not addressed by the framework ................................................................... 36

6.4 Strengths and limitations of the study ......................................................................... 37

6.5 Recommendations for YAPILI ...................................................................................... 38

6.6 Future research ............................................................................................................. 38

Chapter 7. Conclusion............................................................................................................. 39

References ...............................................................................................................................40

Appendix A – Interview guide ................................................................................................ 45

Appendix B – Coding Sheet ....................................................................................................48

Appendix C – Doctors’ prioritisation exercise ...................................................................... 49

Appendix D – Patients’ prioritisation exercise ...................................................................... 50

PAGE 4

Summary/Abstract Sub-Saharan African (SSA) region holds around 25% of the total global disease burden, yet it has only 3% of the global health workforce. Due to shortage of health professionals, poor healthcare infrastructure, time and cost involved in accessing healthcare clinics, it makes it difficult or prevent people from seeking medical help. Despite having one of the least developed infrastructures, SSA has seen a rapid growth in the adoption of digital technology due to increase in mobile and internet penetration rates. Integration of mobile technology with health strategies can provide innovative ways of improving healthcare access and provision, referred to as mobile health (mHealth).

One such example of an mHealth solution is YAPILI, which focuses on telemedicine and is piloting in six English speaking countries in SSA. Among other reasons, acceptance of technology by end-users and context-specific design of the project based on the needs of the population are important factors for the success of mHealth initiatives. Hence, the aim of this research was to evaluate experiences of users i.e. doctors and patients with YAPILI’s telemedicine platform, in order to improve it.

The main research question that this study addressed was ‘What are the experiences of the users i.e. doctors and patients using YAPILI’s telemedicine platform?’ This was done by conducting semi-structured interviews with doctors and patients, followed by a prioritisation exercise with them. The user interface was perceived to be user-friendly by both doctors and patients. The response time was perceived to be bad by the patients. Furthermore, the doctors had specific technical needs for improving the platform, whereas, the patients’ needs were spread over a broader range. Certain contextual factors like language, culture and internet connectivity can prevent users from accessing YAPILI.

This study investigated the user experiences of two most important stakeholders in telemedicine - the doctors and patients with YAPILI, which other telemedicine platforms in the pipeline can learn from. An interesting highlight of the study was to bring to light the differences in needs of doctors and patients with respect to telemedicine platforms. This research also stresses the importance of contextual factors in successful adoption and usage of a telemedicine platform in cross-border setting like YAPILI.

PAGE 5

List of abbreviations

BE: Benefits & Evaluation Framework

CIS: Clinical Information System

HIS: Health Information System

ICT: Information and Communication Technology

IS: Information System

SSA: sub-Saharan Africa

TAM: Technology Acceptance Model

UTAUT: Unified Theory of Acceptance and Use of Technology

PAGE 6

Chapter 1. Introduction The high prevalence of communicable diseases and also the rise in chronic diseases in sub-Saharan Africa (SSA) is a cause for concern. The region holds around 25% of the total global disease burden, yet it has only 3% of the global health workforce (Figure 1) (Moeti, 2015). To put this in a ratio, 2 doctors serve 10,000 people (~vs 35 to 10,000 in Europe) (World Bank, 2011). Compared to the world average of 30 hospital beds per 10,000 people, Africa has only 17 per 10,000 (Tracking Africa's Progress in Figures, 2014). Also, access to patient medical records is difficult due to poor health record management. Apart from poor healthcare infrastructure, patients have to travel long distances in order to access healthcare facilities (Tey & Lai, 2013). Furthermore, many households are compelled to borrow money or pay for health care by selling their assets (Crul, 2014). Thus, due to shortage of health professionals, poor healthcare infrastructure, time and cost involved in accessing healthcare clinics, it makes it difficult or prevent people from seeking medical help.

(World Health Organisation, 2006)

Figure 1: The vertical axis shows the percentage of burden of disease, the horizontal axis shows the percentage of health workers, and the size of the dots represents total health expenditure

Despite having one of the least developed infrastructures, SSA has seen a rapid growth in the adoption of digital technology. The growth rate of mobile market in this region is one of the highest worldwide. Subsequently, growth in mobile-cellular penetration rates and the boom in 3G connections are high as compared to the developed nations and other developing nations (Crul, 2014). In 2015, the mobile penetration rate was 41% and expected to grow to 49% by 2020. Thus, half of the population would be subscribed to mobile services (GSMA Intelligence, 2015). In the recent years, access to internet has also been increasing in the SSA region. In 2015, 22% of the people in SSA had access to internet (World Bank, 2015).

PAGE 7

Integration of mobile technology with health strategies can provide innovative ways of improving healthcare access and provision. Such integrations have been referred to as mobile health (mHealth). Mobile and internet technology has emerged as a service delivery platform among local and global tech entrepreneurs and innovators. In SSA, one of the aims of mobile technologies is to deliver basic health services, remote diagnosis and monitoring and prevention by engaging the system, health professionals and the patients (GSMA Intelligence, 2016). In recent years, various mHealth pilot projects have been tested across different countries of the SSA region. The projects were employed for different purposes like patient follow-up and medication adherence, health staff training and evaluation, drug supply chain and stock management, disease surveillance and intervention monitoring, data collection/transfer and reporting, health education and awareness. Majority of the projects were found to have positive outcomes, for example, improved data collection and reporting, improved compliance of health workers to treatment guidelines, increased patient uptake of disease testing and improved adherence to treatment by patients (Aranda-Jan, Mohutsiwa-Dibe & Loukanova, 2014). This shows that there is scope and demand for mHealth projects in the region. One such example of an mHealth solution is YAPILI, an online peer-to-peer platform that facilitates remote consultations, linking people in SSA countries understaffed with healthcare professionals to licensed physicians both locally and internationally. This type of initiative focuses on telemedicine, which is the delivery of healthcare services from a distance. The main aim of telemedicine is to provide healthcare to patients irrespective of geographical barriers. The target market for the pilot project are six English speaking sub-Saharan African countries namely, Botswana, Kenya, Nigeria, Rwanda, Tanzania and Zambia (Figure 2).

Figure 2: Map of Africa showing the six countries that YAPILI is based in

PAGE 8

The success or failure of mHealth projects depend on various factors such as project design, proper management, integration into the local healthcare system, partnerships with government and financial support (Aranda-Jan, Mohutsiwa-Dibe & Loukanova, 2014). In addition, familiarity of use of mobile phones and acceptance of technology by its end users are perceived to be one of the main reasons for the success of mHealth initiatives. For example, in Botswana and Uganda, similar technologies have had acceptance and the usage outcomes of the projects were seen positively overall (Azfar et al., 2011; Siedner, Haberer, Bwana, Ware & Bangsberg, 2012). Furthermore, mHealth projects have an increased potential risk of failure when they are not tailor-made or adapted to the specific needs of the population or context (Aranda-Jan et al., 2014; Haberer, Kiwanuka, Nansera, Wilson, Bangsberg, 2010). Thus, in order to understand the acceptance, needs and expectations of the end-users with a telemedicine/mHealth platform and to keep them engaged with the initiative, it is important to understand the user experiences. The aim of this study is to help improve YAPILI’s telemedicine platform by evaluating experiences of doctors and patients with YAPILI. This would not only be helpful in considerations for possible scale-up of YAPILI to better serve the target users, but also to gauge the acceptance of telemedicine by these users and to provide input on future telemedicine platforms. In this report, the contextual background, theoretical and conceptual framework, the methodology and the results will be described, followed by a discussion of the results, conclusion and the recommendations.

PAGE 9

Chapter 2. Contextual background In this chapter, the context and concepts of the study are described. The context includes the setting of the study in African countries and YAPILI as a telemedicine platform. The concept, history and advantages of telemedicine are described in detail. Furthermore, the concept of user experience has also been described.

2.1 SUB-SAHARAN AFRICA

SSA faces the challenge of managing a double burden of disease. Along with the continuously increasing rates of infectious diseases such as tuberculosis, HIV/AIDS and other infections, non-communicable diseases like cardiovascular diseases and diabetes are also increasing at a rapid rate in the region (de-Graft et al., 2010). As mentioned earlier, most of the areas in SSA lack infrastructure and healthcare professionals necessary to offer minimal level of healthcare. Thus, widening the gap between the disease burden and the available healthcare resources in SSA region. Telemedicine is a technological solution for providing health care services that breaks down geographic and information barriers and can improve the effectiveness and accessibility of the healthcare system (Farag, 2015). The concept of telemedicine is explained below.

2.2 ORIGIN AND DEFINITION OF TELEMEDICINE

In 1969, Bird and his colleagues first used the term telemedicine and defined it as the delivery of medical care without the usual patient-physician interaction (Bashur, Reardon & Shannon, 2000). In other words, telemedicine is the delivery of healthcare services at a distance. Telemedicine is known to exist since long before the term was coined. In 1906, a Dutch physician, Einthoven, used the prefix ‘tele’ in a medical setting. He called the telephonic transmission of images of electrocardiograph as ‘telecardiogram’. Furthermore, terms like telegnosis, telefluroscopy and telediagnosis have been used during the period from 1950 to late 1960s (Van Dyk, 2013). There is a relation between the evolution of technology and evolution of telemedicine. Bashur et al., (2000) state that “telemedicine is a product of the information age…” and they distinguish three technological eras:

• Telecommunications era - this era spanned from the 1970s to early 1980s. It relied on broadcasting and television technologies and telemedicine services were not combined with any other clinical data.

• Digital era - this era (late 1980s to 1990s) was defined by the use of telecommunications and computer processing together with transmission of large amount of data on limited bandwidths.

• Internet era - growth of internet facilitates a global communication setting with technology becoming affordable and available to a large number of the population. Growing internet speed has the potential to provide new opportunities in telemedicine (Van Dyk, 2013).

PAGE 10

2.3 TELEMEDICINE, EHEALTH, TELEHEALTH, TELECARE AND MHEALTH

There are various terms that are used interchangeably for telemedicine, they are defined below (figure 3).

(Van Dyk, 2013)

Figure 3: Visual representation of how eHealth, telehealth, telemedicine, mHealth and telecare are related Telehealth The term telehealth was coined by Bennet et al in 1978, in order to extend the scope of telemedicine by including a broader set of activities like patient and provider education. Thus, telemedicine is a subset of telehealth and unlike telemedicine, which focusses on curative aspect, telehealth focuses on preventive, promotive and curative aspects. Telecare Telecare involves the continuous remote monitoring of emergencies and changes in lifestyle over time as a preventive measure. Thus, it is under the scope of telehealth, but not telemedicine. eHealth Telehealth and eHealth are very often used interchangeably. The primary difference between the two terms is that ehealth is not limited to providing healthcare at a distance, as opposed to telehealth. mHealth mHealth is a relatively new term and refers to eHealth application that deliver healthcare services via mobile technology. mHealth is considered to be a subcategory of eHealth, telehealth and telemedicine (Van Dyk, 2013).

PAGE 11

2.4 TYPES OF TELEMEDICINE APPLICATIONS

There are two types of telemedicine services based on the timing of the information shared (Craig & Paterson, 2005). Asynchronous, or store and forward telemedicine is the exchange between two or more individuals of pre-recorded data at different times. An example is, an e-mail description of a medical case sent by a patient or referring physician to an expert, who later returns with an advice regarding the diagnosis and management. On the other hand, synchronous, or real time telemedicine involves simultaneous presence of individuals, in order to exchange information immediately, like in the case of videoconferencing (Rao & Lombardi, 2009). YAPILI is an asynchronous telemedicine platform. For both, synchronous and asynchronous telemedicine, the information can be transmitted in various ways, such as short messaging service (SMS), audio, video or images (Ho et al., 2010).

2.5 ADVANTAGES OF TELEMEDICINE

Telemedicine has been shown to improve the quality and accessibility of health services by allowing healthcare providers at a distance, both locally and globally to diagnose, treat and provide follow-up care to patients, especially in the low-income countries devoid of healthcare resources. This would make it easier for patients to seek treatment earlier and perhaps adhere better to their treatment. Telemedicine has been used when the healthcare professional has limited access to expert help and can offer remote physician access to areas otherwise devoid of specialists. The number of referrals to off-site facilities and patient transfers has been shown to decrease by telemedicine. Remote diagnosis and care provided by telemedicine in less-economically developed nations benefits the patients and the healthcare system by decreasing the distance travelled to obtain specialist care, expenses, time and stress. Furthermore, telemedicine offers opportunities for training and professional development of healthcare workers in remote areas. (Ho et al., 2010).

2.6 EXPANSION OF TELEHEALTH/TELEMEDICINE INITIATIVES IN SSA

There have been a growing number of telehealth/telemedicine initiatives in SSA region. There have been a growing number of telehealth/telemedicine initiatives in SSA region. Various telemedicine projects have been established in partnership with countries outside the African continent. The Pan African e-network project is thus far the biggest project for telemedicine and distance education in Africa, in partnership with India (Pan-African e-Network, 2017). In 2001, the RAFT network was established by the University hospitals and University of Geneva in Mali. It has been expanded into 12 other French-speaking countries of Africa, 5 English-speaking countries and some Portuguese-speaking African countries since it has been established. The RAFT network is involved in interactive courses for health care professionals, video conferences, teleconsultations and knowledge base development by collaboration (Bediang et al., 2014). Another telemedicine project called Africa Teledermatology Project is a collaborative dermatology initiative between Africa, USA and Austria for tropical skin conditions (Weinberg, Kaddu, Gabler & Kovarik, 2009). Furthermore, more localised telecardiology, telepathology, teleobstetrics, teleophthalmology and telepyschiatry initiatives have been initiated in various African countries (Mars, 2013).

PAGE 12

2.7 YAPILI AS A TELEMEDICINE PLATFORM

YAPILI is a peer-to-peer telemedicine platform, connecting people in SSA with doctors both locally and internationally. Before the launch of the pilot, potential users of the platform were asked to pre-signup on the website. These users were reached via personal contacts of team members of YAPILI, via social media and by promotion through local organisations. The pilot was launched in April 2017. After the launch, pre-signed up users were invited by email to register on the platform. This led to signing up of 6 doctors and 6 patients on the platform in the first instance. These doctors were the ones that had been reached through social media and the patients were the ones reached through personal contacts. Within the YAPILI platform, a doctor is referred to as “health adviser” and the individual seeking care is referred to as the “advisee”. The process that the advisee and the adviser goes through from registration as a new user to closing of a case is depicted by the user flows (Figure 4 A and B). In general, both the advisee and the adviser create an account and register on yapili.com by agreeing to the terms and conditions. The account is activated upon being verified by the user. After activation of the account, the user is asked to fill in his/her details that are incorporated onto the user’s dashboard. The advisee is prompted to choose an adviser from a choice of three and the respective adviser is notified of it. When needed, the advisee can start a new case, message the adviser the symptoms and wait for the adviser to reply to the message. The adviser needs to respond within 24 hours of receiving the message from his or her advisee. Thus, facilitating a two way communication between the advisee and the adviser. When the advisee and adviser are satisfied with advice, the adviser can close the case. Here on, to avoid confusion, adviser will be replaced by doctor and advisee by patient. However, these are not patients suffering from any particular disease, but individuals seeking health advice on issues that do not need immediate medical attention. A) Advisee (Patient) Flow B) Adviser (Doctor) Flow

Figure 4: The user flow on the left (A) represents the patient flow and the one on the right (B) represents the doctor flow on YAPILI

PAGE 13

2.8 WHY EVALUATE USER EXPERIENCES?

Different factors could influence the success of a telemedicine initiative like the duration of the treatment could affect patient adherence or motivation, users’ illiteracy, high workload for health professionals, clinical training and experience of the health professionals. Ease-of-use and familiarity of the users with the platform are also important for successful implementation of a telemedicine initiative. User concerns with privacy and confidentiality of data are crucial to build trust in the platform (Aranda-Jan et al.,2014). The international standard on ergonomics of human system interaction, defines user experience as “a person’s perceptions and responses that result from the use or anticipated use of a product, system or service”. User experience comes from the user’s interaction with the technical system (Norman & Draper, 1986; Norman, 1999). In order to improve and serve the users better, it is important to evaluate the user experiences of YAPILI’s telemedicine platform.

PAGE 14

Chapter 3. Theoretical background Frameworks for evaluation of telemedicine have been inspired by frameworks for evaluation of the healthcare system and the information systems. This could seem logical as telehealth initiatives (including telemedicine and mHealth) incorporate information and communication technology (ICT) in the provision of healthcare services. In this chapter, theories and frameworks for the evaluation of healthcare, information systems and information systems in healthcare are described. Furthermore, the conceptual model used for evaluating user experiences of YAPILI is described and operationalised in the context of this study

3.1 THEORIES FOR EVALUATING HEALTH CARE

A foundational model for assessing Quality of Care, the Donabedian’s model has been popularly used for various aspects of health care. The overall dependent variable in this model is quality of care. It consists of three sub-categories of variables - structure, process and outcome. Structure includes the input resources and their characteristics that are used to provide healthcare and the way they are organised. The structure includes the availability, accessibility and quality of resources. Good structure of a system improves the chances of providing better health care. Process of care is the actual set of activities that take place between the physicians and the patients. This process has a direct influence on the health of the patients. Outcome is the difference or change in patient’s current and future health status that depends on the preceding healthcare (Donabedian, 1968).

3.2 THEORIES FOR EVALUATING INFORMATION SYSTEMS

Within the context of information systems (IS), the framework of DeLone and McLean identified six variables to evaluate IS (Figure 5). Three variables are about quality in terms of the information, system and service in question. Information quality consists of completeness, ease of understanding and relevance metrics. System quality is measured by adaptability, availability and response time. Service quality includes responsiveness and assurance. The quality of information, system and service could affect the system use, intention to use by the users and the user satisfaction. The system usage, intention to use and user satisfaction in turn, can have an impact on the individual and organisational level that are together viewed as net benefits (DeLone & McLean, 2003).

(DeLone & McLean, 2003)

Figure 5: DeLone & McLean’s Dimensions of Information System Success

PAGE 15

Another widely used theory that is used to evaluate information systems is the Technology Acceptance Model (TAM) (Davis, 1989). TAM uses a user evaluation approach to study the acceptance of a technology or a service. It lacks clinical and organisational acknowledgement and is thus criticised for using perceived ease-of-use as a relatively poor indicator for acceptance of the technology (Chang, 2015).

3.3 THEORIES FOR INFORMATION SYSTEM IN HEALTHCARE FIELDS

To understand the complex nature of determining acceptance and use of new ICT by healthcare professionals, the Unified Theory of Acceptance and Use of Technology (UTAUT) was established (Venkatesh, Morris, Davis & Davis, 2003). The model proposes four core constructs that impact a user’s behavioural intention - performance expectancy, effort expectancy, social influence and facilitating conditions. A generic framework for evaluation of healthcare ICT is the clinical, human and organisational, educational, administrative, technical and social (CHEATS) approach (Shaw, 2002). The Infoway Benefits Evaluation (BE) Framework is another evaluation framework for evaluation of ICT for health solutions (Lau, Hagens & Muttitt, 2006). The BE framework is developed by Canada Health Infoway and it focuses on effective implementation of a digital health solution, how it can be improved and the resulting outcomes. It is an approach to understand the progress towards objectives, knowing what works and identifying barriers to an initiative (Canada Health Infoway, 2012). It is based on updated DeLone & McLean’s model of IS success and a review on determinants of success of in-patient clinical information systems (CIS) based on the updated IS success model (Van der Meijden, Tange, Troost & Hasman, 2003). The BE framework consists of three dimensions of quality namely, system, information and service, two dimensions of system usage, use and user satisfaction and three dimensions of net benefits in terms of quality of care, access and productivity (Figure 6). In the context of YAPILI, it would be insightful to look at the user experiences with these dimensions because they cover not only the perceived benefits but also the socio-technical aspects. Thus, enabling to evaluate the user experiences in a very comprehensive way, giving insights into the users’ needs and expectations. Hence, the BE framework will be used to evaluate the experiences of the users i.e. doctors and patients with YAPILI.

(Canada Health Infoway, 2012)

Figure 6: Benefits and Evaluation Framework

PAGE 16

3.4 CONCEPTUAL FRAMEWORK

The Infoway Benefits and Evaluation Framework The Benefits Evaluation Framework has adopted the six dimensions of the updated DeLone and McLean model. This section will explain the BE framework as developed by Infoway and explain how a particular dimension will be used in the context of YAPILI to evaluate user experiences. This was done by assessing relevance and applicability of the dimensions to the current research and context, in conjunction with the supervisors of the study. System Quality The system quality will evaluate the user experiences with the product itself, i.e. YAPILI. The functionality and performance of the system would evaluate user experiences with the functional and technical capability of YAPILI with various features available - the registration process, medical history questionnaire and the chatting service used for consultation. It also looks at the need for additional feature to the platform. The security evaluates the technical capacity of YAPILI to protect the information being stored for further use. Information Quality Information quality would evaluate the user experiences with content and availability of the information on YAPILI’s website, in order to know if the information on the website is clear to the users. Service Quality The service quality would evaluate the customer support provided by YAPILI by evaluating the user experiences with the availability of technical support and user training. Use In context of YAPILI, the intention to use is defined by the users who would continue to use the platform and those that would recommend YAPILI to their family and friends. The usage behaviour or actual use and self-reported use will not be evaluated in this study, as the number of users and the current usage of the platform is not long enough. . User Satisfaction The user satisfaction is evaluated by the subjective opinions of users in terms of their perceived expectations and value of YAPILI. It also looks at the competency of the users to use YAPILI and how user friendly and learnable the website is. In order to know whether the needs and expectations of the users are fulfilled, the satisfaction of the users with the ICT platform and the consultation is important to evaluate. It is also crucial to know if the users find the platform easy to use and navigate, as this could influence the system usage. Net Benefits The BE framework defines broad outcomes, as it is used in the context of the Canadian healthcare system. In the context of YAPILI, it is still a small scale pilot project and thus, the net benefits need to be looked in a more specific way with respect to the project. Quality of care is evaluated by appropriateness and effectiveness of using YAPILI for health advice, the doctor-patient relationship and responsiveness during the consultation.

PAGE 17

Perceived safety and health outcomes are difficult to evaluate as this will need actual indicators from the local context and are thus, out of scope of this research. Access is defined by users ability to access health services via YAPILI. Access is evaluated by doctors ability to provide health advice without undue burden, patients ability to get health advice and patients ability to access their own information via YAPILI. The productivity is evaluated by the users efficiency, possible care coordination between the local doctors and YAPILI doctors. The net cost is difficult to assess from this research by only looking at the user experiences. Thus, it is beyond the scope of this study. According to the DeLone and McLean’s model, the quality of information, system and service might influence the system use, intention to use and the satisfaction. The system usage and satisfaction can have an impact on the net benefits. The net benefits, in turn can have an influence on the use and user satisfaction. Based on the aim of the study, the main research question is stated below. The sub-questions are derived from the dimensions of the BE framework.

Main Research question

What are the experiences of the users i.e. doctors and patients using YAPILI’s telemedicine platform? Sub-research questions

1. What are the experiences of users i.e. doctors and patients with the system, information and service quality of YAPILI?

2. How do the doctors and patients perceive the use of YAPILI? 3. What factors influence the satisfaction of doctors and patients on

YAPILI? 4. How do the doctors and patients perceive the quality of care, access

and the productivity upon using YAPILI?

PAGE 18

Chapter 4. Methodology In this section, the methods used in the study to obtain answer to the main research question: “What are the user experiences of the user i.e. doctors and patients using YAPILI’s telemedicine platform?” will be described.

4.1 RESEARCH DESIGN

In this study, a qualitative research design was used. This design helps to obtain deeper and holistic insights into the context of the study, by interacting with individuals, groups, communities and organisations (Gray, 2014). In this context, a deeper understanding of the doctors and patients experiences with YAPILI was obtained by conducting semi-structured interviews via Skype.

4.2 STUDY POPULATION

The research subjects were the users of YAPILI. For the interviews, doctors and patients that signed up on the platform were requested to participate in the research by emailing them. Six doctors and six patients were interviewed. Thus, all users that signed-up on the platform during this research were interviewed. All users had used the platform at least once. Additionally, all patients interviewed had opened a case on YAPILI.

4.3 DATA COLLECTION 4.3.1 Interview design A semi-structured interview approach was chosen for this study. Semi-structured interview has an interview guide with a list of questions and issues based on particular themes, but it allows for probing of views and opinions. Thus, the questions are not restricted to those in the interview guide, giving the researcher flexibility to ask more questions or change the order of the questions, depending on the interview (Gray, 2014).The interviews were based on the dimensions described in the conceptual framework. Every interview lasted for 45-60 minutes. The recorded interviews were transcribed in a day or two after the actual interview and subsequently coded and analysed. After the transcription of every interview, the researcher sent a member check to the interviewee, discussing the main findings of the interview, where he or she had the opportunity to modify, agree or disagree with the summary. The interview guide in Appendix A was used for the interviews. 4.3.2 Piloting the interview questions

The interview questions were piloted with one colleague and one signed up user on YAPILI. Piloting ensures that the designs of the data collection methods are useful in gathering the desired information and whether there are any ambiguities in the questions. It will also allow testing of any shortfalls on how the interview is planned and conducted (Gray,2014). Thereafter, if needed, the interview guide was adjusted accordingly.

PAGE 19

4.4 DATA-ANALYSIS 4.4.1 Interviews

All interviews were transcribed in order to analyse the data (Gray, 2014). The transcription was done by using a transcription software called Express Scribe. The analysis started by reading through the transcripts, in order to familiarise with the data. Subsequently, the data was coded by using Atlas.ti software. Coding began immediately after the first interview was done and transcribed. This was done to increase the reliability of the study (Gray, 2014). The coding process consisted of three steps – open, axial and selective (Gray, 2014). In the first step, open coding was done, in which all the important data from the transcripts was labelled. In the second step, also called axial coding, the data was grouped according to different themes. In the final step, selective coding was done where the grouped data was categorised according to the dimensions of the BE framework. Some of the data could fit into one or more categories and when certain data could not be categorised into one of the categories, additional categories were made. Thus, the coding was done both inductively and deductively. The coding sheet is attached in Appendix B. 4.4.2 Prioritization exercise

After the data analysis was complete, the issues that came up during individual interviews were identified and grouped, to form a list. This list was sent to the doctors (Appendix C) and patients (Appendix D). They were asked to rank these issues from 1 to 10, where 1 being the most important and 10 being the least important. This was done to get an impression of what mattered the most to the doctors and the patients and how YAPILI should prioritise the improvement of the platform. After obtaining the ranking results from doctors and patients, each rank was given a weight, starting in the reverse order. So, rank 1 was given a weight of 10, rank 2 was given a weight of 9 and so on. For both doctors and patients, weighted average for each feature or issue was calculated separately using the formula below:

Weighted Average = 𝑥𝑥1𝑤𝑤1+𝑥𝑥2𝑤𝑤2+𝑥𝑥3𝑤𝑤3….𝑥𝑥10𝑤𝑤10𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑇𝑇𝑜𝑜 𝑛𝑛𝑛𝑛𝑟𝑟𝑟𝑟𝑇𝑇𝑛𝑛𝑟𝑟𝑛𝑛𝑛𝑛𝑇𝑇𝑟𝑟

Where,

x = response count for each rank

w = weight given to the rank

Total number of respondents = 6 doctors and 5 patients

The issue/feature with the largest weighted average was the most preferred choice of the respondents. For the prioritization exercise, responses were received from all the 6 doctors, whereas, only 5 patients responded to this exercise.

PAGE 20

4.5 RESEARCH QUALITY 4.5.1 Reliability and bias

Reliability of a research method implies the extent to which same results would be obtained when a particular method is used by two different researchers on two different occasions (Gray, 2014). In this study, reliability was ensured by designing the interview guide, as the main themes of the questions remained same across the interviews. Also, since the interviews were conducted by one researcher, the bias across interviews can be considered low. Furthermore, the member check was sent to the interviewees to ensure that the interpretation of data by the researcher is in line with what the interviewee implied. 4.5.2 Validity

Internal validity is the extent to which the data is influenced by the values, beliefs and thoughts of the researcher (Gray, 2014). To ensure internal validity of this study, the coding scheme for the interviews was discussed with a team of researchers to come to a consensus. 4.5.3 Ethical considerations

The doctors and patients interviewees were asked for verbal consent to record the interviews. They were informed about the research objectives, the utility of the data obtained and the anonymity and confidentiality of the data. Also, after the interview, member-check was sent to the doctors and patients. The interviewees had an option to leave the research at any stage of the study.

PAGE 21

Chapter 5. Results In this section, the results of this study will be presented. The presentation of the results will be done in accordance with the conceptual framework and the sub-questions. First, the experiences of the doctors with YAPILI are presented followed by the experiences of the patients for each dimension. The respondents’ answers were analysed based on the dimensions of the BE framework. The results of this research are divided according to the four sub-questions listed below:

1. What are the experiences of the users i.e. doctors and patients with the system, information and service quality of YAPILI?

2. How do the doctors and patients perceive the use of YAPILI? 3. What factors influence the satisfaction of doctors and patients on YAPILI? 4. How do the doctors and patients perceive the quality of care, access and the

productivity upon using YAPILI?

Participant characteristics Twelve YAPILI users were interviewed in this study. Six of them were doctors and six were patients. The doctors were from Botswana, Nigeria, Netherlands and Tanzania, aged from 26-49 years, with different specialties (Table 1). Five out of the six doctors were between 26 to 35, except one who was 49 years old. There was 1 female doctor and 5 male doctors. The doctors accessed YAPILI from smartphone, tablet, laptop and desktop. The patients were from Kenya, Rwanda, Tanzania and Zambia, aged from 26-50 years (Table 2). Four of them were employed, three in private sector and one in public sector and among the remaining two, one was a start-up founder and another was a student. There were 2 female and 4 male patients. The patients accessed YAPILI from smartphone, laptop and desktop. Table 1: Characteristics of doctors interviewed . Respondent Gender Country Specialisation

D1 M Nigeria General medicine

D2 F Netherlands International health & tropical medicine

D3 M Tanzania Oncologist

D4 M Nigeria General practice

D5 M Botswana Emergency medicine

D6 M Nigeria Family medicine

PAGE 22

Table 2: Characteristics of patients interviewed Respondent Gender Country Profession

P1 M Kenya Employed in private sector

P2 F Kenya Employed in private sector

P3 F Kenya Employed in private sector

P4 M Zambia Public sector employee

P5 M Rwanda Student

P6 M Tanzania Start-up founder Every sub-question listed before will be presented in a new section. For each sub-question, first the doctors experiences will be presented followed by the patients experiences. To make the distinction clear, quotes from doctors will be highlighted in pink and quotes from the patients will be highlighted in blue.

5.1 SYSTEM, INFORMATION AND SERVICE QUALITY In this section, the results of the first sub-question, ‘What are the experiences of the users i.e. doctors and patients with the system, information and service quality of YAPILI?’ will be described. 5.1.1 DOCTORS EXPERIENCES WITH SYSTEM, INFORMATION AND SERVICE QUALITY

5.1.1 (a) System Quality The system quality is analysed by experiences with different type of features on YAPILI and their functionality - creating an account to sign-up, the chat service on the platform for consulting, patient’s medical history and the security with the platform. It also looks at the need for additional feature to the platform. Technically, making an account and registering on YAPILI was mostly hassle-free for the doctors, except minor issues for example, a particular speciality was missing where the adviser was asked to fill in his or her qualifications or speciality. “It was a very smooth ride, no hurdles at all, I didn’t have any glitches.” (D1, male, Nigeria) As of now, the consultation via YAPILI is facilitated by text messages or by sending images and documents. The interviews demonstrated that 3 out of 6 doctors that have experienced YAPILI liked the consulting features to be as they are now. If it were to be a synchronous live chat, call or video, the doctors expressed concern regarding their privacy, inability to get back immediately to the patient due to busy working schedule or

PAGE 23

not having the appropriate advice at hand. Contrary to these findings, the other half of the doctors thought that the current features would suffice for now, but eventually there should be a live chat, call or video for consulting the patient. “I am more conscious while talking on a video on the internet. Who is going to see it? Is it sufficiently safe and is it just one person watching on the other side? I would be more worried about my privacy if I am talking to patients.” (D2, female, Netherlands) “Definitely text is one level; voice would be a step higher; video would be another step higher. Text and pictures would probably suffice for a start.” (D4, male, Nigeria) On YAPILI, the medical history of a particular patient is filled in by the patient and his or her chosen doctor is able to view the history when consulting. However, apart from just being able to view the medical history, it should be modifiable from both doctor’s and patient’s ends. All doctors on the platform felt comfortable sharing their information and qualifications on YAPILI, showing that they trusted the security on the platform. “I think it will be very useful if I can add things in the background or history. So for instance, if I ask a patient if he or she is HIV positive and they haven’t declared that themselves in the history, I can also modify.” (D5, male, Botswana) “Oh yes, of course, those are necessary information when it comes to my profession, so you need to ask.” (D1, male, Nigeria) 5.1.1 (b) Information Quality The information quality is analysed by the content and the availability of the information on the platform. The information provided in the terms and conditions was mostly clear and found to be reasonable by the doctors. However, from the interviews it became evident that the response time within which the doctors should reply to the patients was not very clear. Only one doctor knew the response time. Two doctors were completely unaware of the response time, whereas three didn’t talk about it. The legal framework for the doctors is also mentioned in the terms and conditions. There were two doctors who talked about it. One doctor acknowledged the fact that YAPILI makes clear what are the liabilities of the doctors. The second doctor had forgotten about it, so he asked if he could have a copy of it for his referral. Thus, showing that the doctors were concerned about the regulatory framework. “There is a telemedicine site I work for and there is this timeframe within which we have to attend to each patient’s questions. I didn’t get to see that on the (YAPILI) website…” (D1, male, Nigeria)

PAGE 24

5.1.1 (c) Service Quality The service quality is analysed by the customer support that YAPILI provides. One of the doctors mentioned that he was constantly updated on the progress of YAPILI by the User Care team. On the other hand, in line with the unclarity, insufficiency or being unaware of the information provided by YAPILI, one of the doctors mentioned that he would like to get more updates from YAPILI from time to time. The same adviser also felt the need that YAPILI should organise some sort of training session or seminar where all doctors could come together and learn more about YAPILI. This view was echoed by another doctor who thought it was a good idea to conduct a meeting for the doctors where they could share their experiences regarding various consultations. However, four doctors argued that there wasn’t any need for a meeting or training, as it was quite easy to self-learn how to use the platform. “Yes some more information and correspondence indeed. There should be an informal training or a formal integration course for most of the people on the platform, especially for people on this side of Africa, just to be on the same page.” (D6, male, Nigeria) 5.1.2 PATIENTS EXPERIENCES WITH SYSTEM, INFORMATION AND SERVICE QUALITY 5.1.2 (a) System quality The system quality is analysed by experiences with different type of features on YAPILI and their functionality - creating an account to sign-up, the chat service on the platform for consulting, patient’s medical history and the security provided by the platform. It also looks at the need for additional feature to the platform. In this paragraph, patients experiences with signing up, followed by their experiences with the chat service will be described. Technically, making an account and registering on YAPILI was mostly hassle-free for the patients, except minor issues for example, error messages displayed in coding language instead of a normal notification. For the chatting service that YAPILI currently has, 3 out of 6 patients mentioned that it should go beyond texting and there should be a possibility of live chat or video chat while talking to the doctor, which would make them feel more comfortable to talk. On the other hand, the three other patients were happy with the texting feature. “I would suggest that at some point there could be a video where you can arrange for a video chat with the doctor. It would be very good. You will be able to say it as it is because sometimes when you write you may forget something. When you are seeing the person it is easier to communicate and to let out everything. Also, you kind of become more familiar with your doctor and then you become more free and share much more details.” (P4, male, Zambia) The medical history questionnaire was found to be similar to the form that one fills in usual clinics to give information about one’s medical background by 3 patients that had experienced YAPILI. However, one of the patients argued that someone who signs up on the platform might not be comfortable to fill in such details immediately for security reasons. Another patient thought it could be difficult for someone to understand the medical terminologies and might not be easy to fill it in when someone is doing it from their phone.

PAGE 25

“I was very fine with it. It is a usual questionnaire that you get when you go to a health practitioner to seek medical advice. Those are some of the things they ask, so it was straightforward.” (P4, male, Zambia) 5.1.2 (b) Information quality The information quality is analysed by the availability and the content of the information on the platform. From the interviews it became evident that certain information on the website was clear and certain wasn’t. The patients liked the fact that the platform states vividly that it is meant for seeking health advice and not medical treatment. When the patients sign-up on YAPILI, they get to choose one doctor from a choice of three and all their questions are addressed to that particular doctor. However, this wasn’t clear to the patients, as they thought they could change the doctor each time. Although stated on the website, the response time from the doctor wasn’t clear to one of the patients. Another patient thought that YAPILI was still in the testing phase and wasn’t live with the pilot project. “The terms and conditions were clear enough. What you need to make more clear is the turnaround time for responses by the doctor, which is my only concern so far. If we can add that your doctor would reply to you in this period of time, so someone can see from the very beginning.” (P3, female, Kenya) 5.1.2 (c) Service quality The service quality is analysed by the customer support that YAPILI provides. Although, the patients did have some communication with the User Care team but only one patient talked about it in the interview. He liked the fact that YAPILI has a User Care team, but he also mentioned that there should be more communication with the users. “...there is need for more interactions. I see that you have the user care and monitoring team that is a good thing to have for the platform, which for me I see as a continuous interaction with users…” (P1, male, Kenya) So far, the doctors and patients experiences with the system, information and service quality have been presented. Now their perception of use of YAPILI will be presented.

PAGE 26

5.2 USE In this section, results of the second sub-question, ‘How do doctors and patients perceive the use of YAPILI?’ will be described. 5.2.1 DOCTORS PERCEPTION OF USE The actual use could not be analysed in this research due to an ongoing pilot and not much usage at the stage this research was done. Thus, the use of YAPILI is analysed by the intention to use it in the future. All doctors that have experienced YAPILI saw themselves continuing to use YAPILI. They also saw themselves referring YAPILI to their colleagues. Although not directly related to the use of YAPILI by the doctors, local doctors based in Africa also expressed concern of the patients not being able to use the platform because of internet issues or simply no awareness of the usefulness of such platforms. The use of YAPILI by the doctors can be affected by different contextual factors such as unstable internet connectivity, ability to use YAPILI, experiences of the doctors with other telemedicine platforms, which could set the bar high for their experience with YAPILI. 5.2.2 PATIENTS PERCEPTION OF USE All patients saw themselves continuing to use the platform. However, one of them mentioned if she was not in the pilot, she wouldn’t be motivated enough to continue using the platform, as it was not interactive. The patients also saw themselves referring YAPILI to their friends and family. Although there is an intention to use YAPILI, it can be influenced by not enough smartphone usage in African countries, poor internet connectivity, cultural factors, no fluency in English and competition with other telemedicine platforms.

PAGE 27

5.3 USER SATISFACTION In this section, the results of the third sub-question, ‘What factors influence the satisfaction of doctors and patients on YAPILI?’ will be described. 5.3.1 FACTORS THAT INFLUENCE DOCTORS’ SATISFACTION The user satisfaction is analysed by the perceived expectations, values, competency and ease of use of YAPILI. The main motivations for doctors to use YAPILI were:

1. Broadening of patient base 2. Ability to help patients remotely 3. Able to see citizens being connected to doctors worldwide 4. Rewarding to talk to patients and help them solve their issues 5. A source of passive income. 6. A way of learning and expanding their knowledge base about different diseases or

problems that they might be exposed to through the consultations and interactions with different patients.

As for the competency, experience of the doctors with other other telemedicine platforms can also have an influence on their satisfaction with YAPILI. The doctors unanimously agreed that the website is user-friendly and has a simple interface, making it easy to use. “I thought it was a really nice initiative. I have worked in three African countries ‐ Zambia, Malawi and Tanzania. I noticed a lot of people do have access to mobile phones and even internet but are very far from medical practitioners. YAPILI is a great initiative to combine the fact that people have more mobiles nowadays but are still coping with the fact that there is not enough medical care in the country they are in.” (D2, female, Netherlands) 5.3.2 FACTORS THAT INFLUENCE PATIENTS’ SATISFACTION The main motivations for the patients to use YAPILI were:

1. To get timely advice on minor health issues and chronic diseases, instead of visiting local clinics/hospitals because of their busy work schedule, long queuing hours, finances involved in travelling and consultation.

2. Ability to talk to the same doctor, unlike the local situation in African countries where the patient is seen by different doctors every time.

3. Avail medical advice from doctors in different parts of the world due to a general belief that foreign doctors are better than local doctors. This is a cultural factor. The conceptual framework doesn’t address cultural factors.

4. Other telemedicine platforms in their countries were not received well due to bad response time and pay per consultation module, whereas, YAPILI can be used free of charge during the pilot and they hope the response time is good.

As for the competency and ease of use, the patients agreed that the platform is straightforward requiring minimum knowledge and skills to use it.

PAGE 28

“I like the idea of having a personal doctor. The local situation one day you speak to female doctor and then the next day you move to another hospital or probably another doctor in the same hospital. It is tedious.” (P3, female, Kenya) Thus far the doctors’ and patients’ experiences with system, information service quality, perception of use and factors that influence their satisfaction have been presented. Now, their experiences with the quality of care, access and productivity will be presented.

5.4 QUALITY OF CARE, ACCESS AND PRODUCTIVITY In this section, the results of the fourth sub-question, ‘How do the doctors and patients perceive the quality of care, access and the productivity upon using YAPILI?’ will be described. Since the platform has not been used long enough, these dimensions will be looked through actual and perceived benefits of YAPILI. 5.4.1 DOCTORS PERCEPTION OF QUALITY OF CARE, ACCESS AND PRODUCTIVITY 5.4.1 (a) Quality of care The quality of care is analysed by the doctor-patient relationship, appropriate and effectiveness of using YAPILI for health advice and the responsiveness of the doctor. Consulting patients through YAPILI was seen as an asset to give one-on-one attention to patients, but keeping in mind that YAPILI is only an additional asset and not a complete replacement for physical consultation. Some barriers that could affect the quality of medical advice given by the doctors via YAPILI were mentioned: not being able to see the patient, not able to examine the patient physically and differences in medical terminologies across countries, language and cultural differences. The response time from the doctor exceeded 24 hours due to three reasons - unclear response time, no appropriate notification to the doctor about the case opened by his/her patient and a doctor being on leave when the patient opened the case. “In some cases it is really good. You can get some more personal attention, but it is true if you have someone locally who you know very well, it can feel safer when you talk personally” (D2, female, Netherlands) 5.4.1 (b) Access Access is analysed by the ability of the doctors to provide health advice through YAPILI without undue burden. Firstly, YAPILI was seen as a convenient way to extend health advice to the population that does not have the best outreach to healthcare services and for extending help on certain diseases that are considered a taboo in African countries. Secondly, the medical history provided by the platform was seen as an asset because African countries are still facing issues with patient records. Thus, allowing the doctors easy access to the patient information, in order to provide timely help. “YAPILI increases the accessibility to render services to the general population. We still have a problem of lack of medical expertise, not enough doctors and we don’t have the facilities. The

PAGE 29

country has some remote areas, so sometimes people fail to travel because of lack of cash or deposit. If YAPILI is working, it is making able to access the services of the doctor. YAPILI is here to connect people” (D3, male, Tanzania) Thirdly, majority of the doctors mentioned that it was easy to integrate YAPILI with their daily work schedule, except one doctor, who felt it was difficult for him to find time to respond to his advisees on the platform because of his extremely busy schedule. “For someone like me who works at the hospital full time, this is quite convenient to use. In the evening I check if someone has contacted me so it is something that you can do next to your own job. That is good.” (D2, Female, Netherlands) 5.4.1 (c) Productivity Productivity is analysed by the efficiency and care coordination that YAPILI can facilitate. YAPILI was seen as a means of narrowing the diagnosis and informing patients about possible alternative treatments. Referral to a local clinic upon narrowing of diagnosis was seen as an efficient way of consulting, as the doctor in the clinic would have a pre-diagnosis from the doctor on YAPILI. Consulting via YAPILI was also seen as a way of prioritising the cases that need immediate medical attention and encouraging the patients to visit hospitals so they can get treatment in time. “It (YAPILI) could maybe make more efficient use of the time of the doctor by making sure that in-person (local doctor in the country) sees the right things. It is hard but I think it will be more efficient in the end….It could be sort of a triage.” (D5, male, Botswana) However, In order to facilitate the above mentioned, firstly, the doctors felt the need to gain more information from their patients when they stated their problem. Secondly, one doctor mentioned the need for information about general functioning of the healthcare system of the country that the patient is in. 5.4.2 PATIENTS PERCEPTION OF QUALITY OF CARE, ACCESS AND PRODUCTIVITY 5.4.2 (a) Quality of care The quality of care is analysed by the doctor-patient relationship, appropriate and effectiveness of using YAPILI for health advice and the responsiveness of the doctor. Majority of the patients felt that being consulted via YAPILI or virtually will not have a major influence on the doctor-patient relationship for different reasons: one, they would eventually get used to it, provided the advice is satisfactory, two, YAPILI would be used for minor health issues, so seeing or not seeing the doctor physically won’t make a big difference. If something is unclear it can be clarified by simply typing another message to the doctor. For appropriate and effectiveness of using YAPILI for health advice, 2 patients chose a local doctor over an international one because they felt that the local doctor could understand their case better. Contrary to that, 2 patients said that they chose an

PAGE 30

international doctor because probably those doctors would be able to give them more attention than a local doctor who is so busy due the patient burden and to be able to get different advice than the ones that were used to getting. Interestingly, 2 patients wanted the leverage to switch doctors depending on the problem they have. The patients were unhappy with the response time. One of the patients who had received a reply from the doctor was not satisfied with the response because in the doctor’s reply he asked for more clarification of her problem and thus, it led to more delay in getting the advice. “Yeah it was 48 hours and in his response he has some enquires that he had as well, so I responded to his enquiries. I haven’t checked today but yesterday there wasn’t any response, so that was actually more than 48 hours for a response.” (P3, female, Kenya) 5.4.2 (b) Access Access is analysed by patients ability to get health advice through YAPILI. The patients perceived YAPILI to be far more accessible than actually going to a local doctor in their country, due to long queuing hours, money spent on the consultation, not receiving enough attention from the doctor, being able to access their own medical records and getting advice from doctors in a different country or context via YAPILI. “The concept is very good. It would be cheaper and faster, it would be time saving. It will expose you to experiences of other doctors that are far away and probably provide a better solution to the problems you have so because of that it is a good thing.” (P4, M, Zambia) Although YAPILI was seen to be more accessible in general, there were certain barriers that came during the interview that would prevent the users from accessing YAPILI. Two patients, one in Kenya and another in Zambia were sceptical about the internet connectivity and smartphone usage, especially in the rural parts of the countries. Thus, poor or no internet connectivity would be a major barrier to access YAPILI. One of the patients mentioned that language can be a barrier to accessing YAPILI, if someone doesn’t know English, but on the other hand another patient argued that the platform needs minimum English language skills to use it. Thus, depending on the context, internet connection, linguistic or cultural barriers could prevent patients from accessing YAPILI. Such contextual factors are not addressed by the BE framework. “Not all Kenyans have that android phone that they can login from just data bundles. There are villages where there is not good network and the data bundle will not work. Such people might have some challenge. Most people in Nairobi wouldn’t have any problem because they have good network.” (P2, female, Kenya)

PAGE 31

5.4.2 (c) Productivity Productivity is analysed by the efficiency and care coordination that YAPILI can facilitate. The patients found it efficient to use YAPILI to get an assurance or confirmation from doctors, for instance getting an opinion on a particular medicine, a minor health issue that one normally ignores or keeps procrastinating. This could be efficient because instead of procrastinating and having a major issue, the patients can go earlier to a doctor if needed. When it comes to the care coordination, the medical history questionnaire was mentioned to be an asset by one of the patients, as it could be used for follow up visits with doctors. “Sometimes you just pass by to the chemist. It could be that you get the wrong medication because you prescribed to yourself. So if you contact a doctor, he or she can say do this or do that. You don’t want to go and line up in a clinic for two hours just to be given painkillers.” (P2, female, Kenya)

PAGE 32

5.5 PRIORITISATION EXERCISE In this section, the results of the prioritization exercise will be presented. 5.5.1 DOCTORS PRIORITISATION After obtaining the ranking from the doctors, the weighted average was calculated for each issue or feature. Subsequently, the one with the highest weighted average was found to be the most preferred issue/feature, according to the doctors. The following table summarises the issues/features in the order of most preferred to the least preferred based on the weighted average. Table 3: Doctors’ prioritisation along with weighted average

Issue/feature Weighted average

Adding a feature in which patients are asked predefined health-related questions in the chat box, in order to receive more and better quality of information from the patient from the beginning of the case

7.33

Being able to edit medical history of the patient from the doctor's account 6.16

Provision of a small summary of healthcare systems of the country within which the patient is based

6.16

Content of notification messages from YAPILI need to be improved, for instance the notification when a patient opens a new case

6.00

Arrange a training seminar/session for all YAPILI doctors 5.66

A new feature to set the maximum limit to the number of patients you can consult via YAPILI (in order to not receive more patients than you can manage)

5.16

Addition of a voice chat function 5.16

Build in a function to say "I am temporarily unavailable", to let your patients know if you are away

5.00

Provision of an Android application for YAPILI 5.00

Addition of a video chat function 4.33 The feature ‘Adding a feature in which patients are asked predefined health-related questions in the chat box, in order to receive more and better quality of information from the patient from the beginning of the case’ was found to be the most preferred feature among doctors with 7.33 as the weighted average. During the interviews, doctors did mention that they had to ask quite more questions to get the immediate background of the problem that the patient has described, and thus, it seems crucial for the doctors to get complete case history of the patient.

PAGE 33

5.5.2 PATIENTS PRIORITISATION After obtaining the ranking from the doctors, the weighted average was calculated for each issue or feature. Subsequently, the one with the highest weighted average was found to be the most preferred issue/feature, according to the patients. The following table summarises the issues/features in the order of most preferred to the least preferred based on the weighted average. Table 4: Patients’ prioritisation along with weighted average

Issue/feature Weighted average

Response time should be shorter 9.80

Provide an Android application for YAPILI for phone use 7.80

Option to choose a different doctor depending on the problem 7.20

Receive notification via email when the doctor has replied to your message 6.80

Website should have some ready health-related content that you can look up on the website

5.20

Addition of a video chat function 4.40

Addition of a voice chat function 3.60

There should be a doctor available to you from the country that you live in 3.60

Website should be more interactive, for instance seeing pop-ups to notify a disease outbreak in a particular country

3.60

Medical history questionnaire should be shorter 3.00

The issue ‘Response time should be shorter’ was found to be the most preferred feature among patients with 9.8 as the weighted average. During the interviews, it became very evident that the patients were not happy with the response time from their doctors, as in all cases it had exceeded 24 hours. Interestingly, the second most feature of provision of an android app for YAPILI and the fourth preferred feature of receiving notification when a doctor has replied have been deployed by YAPILI during the course of this research.

PAGE 34

Chapter 6. Discussion This was a qualitative study to evaluate the experiences of users i.e. doctors and patients with YAPILI’s telemedicine platform. It was done by conducting semi-structured interviews. A brief summary of the main findings are summarised below:

Doctors experiences

a. The user interface was easy to use. However, technical improvements in the features were needed.

b. Certain information on the website was unclear, for instance, the response time. c. There were mixed responses about the privacy and security issues with voice or

video chat. d. YAPILI was seen as a medium to extend patient outreach, although certain barriers

could affect the effectiveness of the medical advice or the doctor-patient relationship, for example, not being able to see the patient physically, cultural / linguistic differences between the doctor and the patient. The doctors found it easy to integrate YAPILI with their everyday work.

e. Doctors felt the need to obtain more information from the patients in the first instance when they state their case and wanted an option to edit patient’s medical history.

Patients experiences

a. The interface was user-friendly with a good layout of the website. b. The response time was perceived to be bad. c. Linguistic differences and poor internet connectivity were seen as some barriers

to access YAPILI. d. The patients trusted the platform and didn’t seem to have security issues. e. YAPILI was seen as a convenient way to obtain health advice on minor health

issues or on chronic diseases. In the following sub-sections, the findings of this study will be presented in the light of any similar previous research or theories and its transferability to other contexts. Furthermore, the strengths and limitations of the study, recommendations to YAPILI and future research will be discussed.

6.1 DIFFERENCE IN DOCTOR AND PATIENT NEEDS During a telemedicine encounter, the doctors and patients are the main participants and they can have different perspectives of the system as they use it. Interestingly, as seen from the prioritization exercise, the doctors suggested very specific needs and improvements to the system as opposed to the patients, who had needs over a broader range. Also, the needs of the doctors were relatively more technical as compared to the needs of the patients, which were more related to getting appropriate advice. This could be explained by the findings of another study that found that patients prefer obtaining health advice via a means that solves the challenges of distance, whereas, doctors want

PAGE 35

to provide reliable service through distance (LeRouge, Hevner, Collins, Garfield & Law, 2004). Another study using the Technology Acceptance Model (TAM) found that the perceived usefulness of the system was found to have a significant influence on doctors’ intention to use the technology (Hu, Chau, Sheng & Tam, 1999). This could explain the technical needs of the doctors, treating technologies as tools that are acceptable only when their usefulness has been proven to incorporate in their practices.

6.2 FINDINGS ADDRESSED BY THE FRAMEWORK In this section, findings that fit the Benefits & Evaluation Framework will be discussed in light of other similar research. The main points discussed below are: security with the platform, doctor-patient relationship, perceived ease of use and perceived usefulness and experience with other telemedicine platforms. 6.2.1 Security provided by YAPILI It seemed that all the patients trusted the platform. They did not seem to be very worried about the security concerns with YAPILI, except one patient who mentioned that although he was comfortable to fill in the medical history questionnaire, somebody could be worried about it. In a study by LeRouge and colleagues in 2004 they also showed that the patients didn’t bring up security issues and they speculated that could be due the lack of knowledge or understanding of the potential privacy issues with such technologies. However, in the case of YAPILI, this doesn’t seem to be true. It can be speculated that the patients didn’t have any security concerns because they had signed-up on the platform through contacts of YAPILI team and thus they trusted the platform. 6.2.2 Quality of care and access In literature, telemedicine is seen as a convenient way for patients who would otherwise be obliged to travel far distances for their appointments (Miller, 2010). This was also identified as an advantage of using YAPILI by both the doctors and patients. The doctors also thought that patients could be encouraged to talk to their doctors about stigmatised issues via YAPILI. This is in line with literature that saw telemedicine as means to overcome inhibition of discussing sensitive issues like psychiatric problems or sexually transmitted diseases (Miller, 2010). However, despite the advantages, lack of “laying on the hands” could compromise the doctor’s’ ability to make the correct diagnosis via telemedicine (Hiratsuka et al., 2013). This was a challenge that was also mentioned by doctors on YAPILI. 6.2.3 Perceived usefulness and perceived ease of use The famous Technology Acceptance Model (TAM) explains user acceptance of technology. According to the model, two factors are particularly important - Perceived Ease of Use and Perceived Usefulness (Davis, 1989). The user interface on YAPILI was perceived to be easy to use by both doctors and patients. Also, when we look at the user satisfaction, particularly the motivation factors mentioned by the doctors and patients, it can be seen that they perceive YAPILI to be useful and valuable. DeLone & McLean’s IS model and the BE framework derived from it also state that the system quality, which looks at ease of use and user satisfaction, which looks at the usefulness/value of a system can both influence the use of a system (DeLone & McLean, 2003). Thus, in the case of YAPILI,

PAGE 36

the perceived ease of use and perceived usefulness can also influence the use of YAPILI by the users. 6.2.4 Competency - Experience, skills and knowledge A Nigerian doctor (D6, male, Nigerian, 49 years) initiated a conversation on receiving training to use YAPILI efficiently, showing his enthusiasm and belief in telemedical practices. An important thing to note here is the age of the doctor. A study in South Africa by Cilliers & Flowerday in 2011 found that healthcare workers older than 40 years were less likely to have received any formal computer training as compared to those younger than 40 years of age. All other doctors on YAPILI were below 35 years of age, and thus, receiving a training was not a top priority for them. One of the doctors on YAPILI (D1, male, Nigerian) had experience with other telemedicine platforms as well. This can be an important factor when it comes to the user experience, as it leads to constant comparison of different telemedicine platforms. This can be seen in both positive and negative ways. Positively, YAPILI can learn and improve from the doctors experiences with other telemedicine platforms, but on the other hand, it could make the doctors disinterested in YAPILI if the other platforms outperform it.

6.3 FINDINGS NOT ADDRESSED BY THE FRAMEWORK In this section, findings that do not fit the Benefits & Evaluation Framework will be discussed in light of other similar research. 6.3.1 Contextual factors The BE framework was developed by Canadian Infoway, a non-profit Canadian organisation for implementation of digital health solutions in Canada. Thus, it was developed for a homogenous context. It is a very comprehensive evaluation framework that looks at socio-technical aspects of a digital solution. However, this framework does not consider the application of a digital health solution in a cross-cultural settings like YAPILI. Thus, other contextual factors also become important: 6.3.1 (a) Language and culture Telemedicine is a socio-technical system, involving human and technological aspects across borders. In the past, simple factors like language and timezone barriers have been reasons for failure of telemedicine initiatives (Afarikumah & Kwankam, 2013). Thus, due to cross-border interactions among doctors and patients via YAPILI, it is crucial to consider the inter-cultural and multi-national differences, as the users can have different values, expectations and needs. Majority of the patients on YAPILI expressed frustration with the response time from the doctor and thus, felt that YAPILI is not keeping up with its promise and isn’t keeping up to their expectations. According to the researcher, this could be related to a cultural factor of not being valued as a patient. Studies in literature found a common theme among how SSA patients’ positively perceive the healthcare they received, i.e. being valued as a patient - feeling of being well-treated and that the physicians took a real interest in addressing their issues (Farag, 2015). This feeling of a patient not being valued by the doctor on YAPILI can have further implications when it comes to the use and user

PAGE 37

satisfaction with YAPILI, as according to the BE framework, the net benefits (quality of care in this case) can affect both those dimensions. 6.3.1 (b) Legal/regulatory issues The Benefits & Evaluation framework overlooks the legal and regulatory issues for a digital health solution like telemedicine. However, in a cross-border setting legal issues are very crucial, for example, which country’s laws apply in case of a mistake? (Ho et al., 2010) Concern with such legal issues was also brought up by doctors during the interviews. 6.3.1 (c) Internet connectivity Despite the high internet penetration rate in SSA region it faces internet technology issues due to low bandwidth, high demand-supply gap and inequality in distribution favouring the urban areas over the rural ones (Mbarika & Tan, 2007). The issue of unequal distribution was also brought up during the interviews. Thus, improper ICT infrastructure can have a major influence on the utility and experiences of the users in SSA. 6.3.1 (d) Doctor’s approach to privacy The approach to privacy among doctors could be related to the context of the country. In this study, a doctor from the Netherlands seemed to be concerned about the privacy when it comes to call or video chat, as opposed to a doctor from Nigeria. The sample size in this research was too small to conclude significant differences among doctors from different countries. However, it should be noted for future research and application. 6.3.2 Other findings not addressed by Benefits & Evaluation Framework Contrary to findings in literature about resistance of doctors to telemedicine (Poon et al., 2004), the doctors on YAPILI seemed quite enthusiastic about telemedicine. This could be because the doctors signed up voluntarily on YAPILI, as opposed to a clinic or hospital adopting telemedicine practices, where they would be, for example, forced to respond to patients online.

6.4 STRENGTHS AND LIMITATIONS OF THE STUDY Strengths of the study This research evaluated the user experiences of the two most important stakeholders of telemedicine i.e. doctors and patients. The study looked at the user experiences of the doctors and patients with YAPILI from a socio-technical perspective. Thus, evaluating the user experiences in a very comprehensive way. This will be beneficial for YAPILI during potential scale-up in the near future. Since the sample size was small and not representative, the findings cannot be generalised, but they can be transferred to another context, where a new telemedicine platforms in SSA region can learn from this research. Limitations of the study While conducting this research, all patients (6) and doctors (6) on the platform were interviewed. The doctors and patients had used the platform at least once. This could be a limitation, due to inexperience of the users with the platform. In this study, the low number of users and their inexperience with YAPILI could mean that there could be more or different issues/experiences that could have emerged with a bigger and more experienced sample. This could also imply that perhaps data saturation wasn’t reached.

PAGE 38

Thus, as the users get more experienced with YAPILI, a similar study should be conducted. Although all interviews were conducted in English, it was difficult for the researcher to understand the pronunciation of certain words or phrases of the doctors and patients. This was because all the patients and majority of the doctors were based in African countries and they had different accents of English than the researcher. This was however compensated by summarising and if necessary, clarifying things during the interviews and in the member-checks.

6.5 RECOMMENDATIONS FOR YAPILI Taking into account the results, this study has come up with the following recommendations for YAPILI:

1. The response time should be mentioned clearly to both the doctors and the patients. Also, the current window of responding, which is 24 hours should be made much shorter.

2. In order get the immediate background of a case from the patient in the first instance, pre-defined questions should be added to the text box where the patient type his/her symptoms to the doctor. Some examples of these pre-defined questions can be:

a) When did the complaint start? b) How bad is it? c) What did you already try?

3 Make some ready content available on the website for both the doctors and patients, for example, for the doctors, a short summary of the local healthcare system that the patient is based in and for the patients some health-related information that they can look up on the website.

6.6 FUTURE RESEARCH For future research YAPILI should keep these in mind:

1. Since the sample was small and the users were inexperienced with YAPILI in this study, further qualitative research should to be done with a bigger and a more experienced sample in the near future.

2. In order to be able to generalise the findings, a quantitative study should be done at a later stage when the user base is much bigger.

3. Although not very evident in this study because of low usage of the platform, users from different countries and cultures can have different needs and expectations. Thus, further research could be done on the cultural differences among users, in order to better sustain the platform.

PAGE 39

Chapter 7. Conclusion This study investigated the user experiences of two most important stakeholders in telemedicine - the doctors and patients with YAPILI, which other telemedicine platforms in the pipeline can learn from. An interesting highlight of the study was to bring to light the differences in needs of doctors and patients with respect to telemedicine platforms. The study showed that the doctors had more specific technical needs, as opposed to more broader needs of the patients. It also provides some recommendations to YAPILI, in order to improve the telemedicine platforms, as per the needs of the users. These include, reduction of response time, addition of pre-defined questions, in order to obtain as much information as possible from the patients in the first instance and to have some ready health-related content on the website. This research also stresses the importance of contextual factors in successful adoption and usage of a telemedicine platform in cross-border setting. In addition to the developmental factors, for example, the internet connectivity in a country, the users from different countries can have differing needs and expectations. Thus, it is important for YAPILI to investigate that in the near future, in order to sustain the platform.

PAGE 40

References Afarikumah, E., & Kwankam, S. Y. (2013). Deploying actor-network theory to analyse telemedicine implementation in Ghana. Science, 1(2), 77-84.

Aranda-Jan, C., Mohutsiwa-Dibe, N., & Loukanova, S. (2014). Systematic review on what works, what does not work and why of implementation of mobile health (mHealth) projects in Africa. BMC Public Health, 14(1). doi:10.1186/1471-2458-14-188

Azfar RS, Weinberg JL, Cavric G, Lee-Keltner IA, Bilker WB, Gelfand JM, Kovarik CL. HIV-positive patients in Botswana state that mobile teledermatology is an acceptable method for receiving dermatology care. J Telemed Telecare. 2011;17(6):338–340. doi: 10.1258/jtt.2011.110115. Bashshur, R., Reardon, T. and Shannon, G. (2000). Telemedicine: a new health care delivery system. Annual Review of Public Health, vol. 21, no. 1, pp. 613637. Bediang, G., Perrin, C., Ruiz de Castañeda, R., Kamga, Y., Sawadogo, A., Bagayoko, C. O., & Geissbuhler, A. (2014). The RAFT Telemedicine Network: Lessons Learnt and Perspectives from a Decade of Educational and Clinical Services in Low- and Middle-Incomes Countries. Frontiers in Public Health, 2, 180. http://doi.org/10.3389/fpubh.2014.00180 Canada Health Infoway. (2012). Canada Health Infoway Benefits Evaluation Indicators Technical Report. Retrieved from https://www.infoway-inforoute.ca/en/component/edocman/resources/reports/450-benefits-evaluation-indicators-technical-report-version-2-0 Cilliers, L., & Flowrday, S. V. (2013). Will computer literacy affect telemedicine acceptance among health care workers. Chang H. Evaluation Framework for Telemedicine Using the Logical Framework Approach and a Fishbone Diagram. Healthcare Informatics Research. 2015;21(4):230-238. doi:10.4258/hir.2015.21.4.230. Craig J, Patterson v. Introduction to the practice of telemedicine. Journal of Telemedicine and Telecare, 2005, 11(1):3–9.

PAGE 41

Crul, S. (2014). The mHealth opportunity in Sub-Sahara Africa The path towards practical application (1st ed.). Retrieved from https://www2.deloitte.com/content/dam/Deloitte/nl/Documents/technology-media-telecommunications/deloitte-nl-mhealth.pdf Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13(3):319–340. de-Graft Aikins, Ama et al. “Tackling Africa’s Chronic Disease Burden: From the Local to the Global.” Globalization and Health 6 (2010): 5. PMC. Web. 29 June 2017. Delone WH, McLean ER. The DeLone and McLean model of information systems success: a ten-year update. J Manag Inf Syst. 2003;19(4):9–30. Donabedian A. The evaluation of medical care programs. Bull N Y Acad Med. 1968;44(2):117–124. Farag, J., (2015). Cross Cultural Telemedicine: Sub-Saharan Africa. Intersect 9 (1) Gray, D. (2014). Doing research in the real world (1st ed.). Los Angeles, Calif.: Sage. GSMA Intelligence,. (2015). The Mobile Economy Sub-Saharan Africa 2015 (1st ed.). Retrieved from http://www.gsma.com/mobileeconomy/archive/GSMA_ME_SubSaharanAfrica_2015.pdf GSMA Intelligence,. (2016). The Mobile Economy Sub-Saharan Africa 2016 (1st ed.). Retrieved from https://www.gsmaintelligence.com/research/?file=3bc21ea879a5b217b64d62fa24c55bdf&download Haberer JE, Kiwanuka J, Nansera D, Wilson IB, Bangsberg DR. Challenges in using mobile phones for collection of antiretroviral therapy adherence data in a resource-limited setting. AIDS Behav. 2010;14(6):1294–1301. Hiratsuka, V., Delafield, R., Starks, H., Ambrose, A. J., & Mau, M. M. (2013). Patient and provider perspectives on using telemedicine for chronic disease management among Native Hawaiian and Alaska Native people. International Journal of Circumpolar Health, 72, 10.3402/ijch.v72i0.21401. http://doi.org/10.3402/ijch.v72i0.21401

PAGE 42

Ho, K., Cordeiro, J., Hoggan, B., Lauscher, H., Grajales, F., Oliveira, L., & Polonijo, A. (2010). Telemedicine: opportunities and developments in Member States: report on the second global survey on eHealth 2009. (1st ed.). World Health Organisation. Retrieved from http://www.who.int/goe/publications/goe_telemedicine_2010.pdf Hu, P. J., Chau, P. Y., Sheng, O. R. L., & Tam, K. Y. (1999). Examining the technology acceptance model using physician acceptance of telemedicine technology. Journal of management information systems, 16(2), 91-112.

LeRouse, C., Hevner, A., Collins, R., Garfield, M., & Law, D. (2004, January). Telemedicine encounter quality: comparing patient and provider perspectives of a socio-technical system. In System Sciences, 2004. Proceedings of the 37th Annual Hawaii International Conference on (pp. 10-pp). IEEE.

Mars, M. (2013). Telemedicine and advances in urban and rural healthcare delivery in Africa. Progress in cardiovascular diseases, 56(3), 326-335.

Mbarika, V., & Tan, J. TELEMEDICINE TRANSFER IN SUB-SAHARAN AFRICA: INVESTIGATING INFRASTRUCTURE AND CULTURE.

Miller, E. (2010). Telemedicine and the Provider-Patient Relationship: What We Know So Far. Retrieved from http://nuffieldbioethics.org/wp-content/uploads/Miller-E-2010-Evidence-review-Telemedicine-and-the-Provider-Patient-Relationship-what-we-know-so-far.pdf

Moeti, M. (2015). Toward Universal Health Coverage in Africa - WHO | Regional Office for Africa. Afro.who.int. Retrieved 14 February 2017, from http://www.afro.who.int/en/rdo/articles/4823-toward-universal-health-coverage-in-africa.html

Norman, D. A. (1999). The Invisible Computer. MIT Press

Norman, D. A. & Draper, S. (1986). User Centered System Design: New Perspectives on Human-Computer Interaction. Hillsdale: Lawrence Erlbaum.

PAGE 43

Pan-African e-Network. (2017). Panafricanenetwork.com. Retrieved 27 June 2017, from http://www.panafricanenetwork.com/

Poon, E., Blumenthal, D., Jaggi, T., Honour, M., Bates, D. and Kaushal, R. (2004). Overcoming barriers to adopting and implementing computerized physician order entry systems in US hospitals. Health Affairs, 23(4), 184–190.

Rao B, Lombardi A II. Telemedicine: current status in developed and developing countries. Journal of Drugs in Dermatology, 2009, 8(4):371–375. Shaw NT. 'CHEATS': a generic information communication technology (ICT) evaluation framework. Comput Biol Med. 2002;32(3):209–220. Siedner MJ, Haberer JE, Bwana MB, Ware NC, Bangsberg DR. High acceptability for cell phone text messages to improve communication of laboratory results with HIV-infected patients in rural Uganda: a cross-sectional survey study. BMC Med Inform Decis Mak. 2012;12:56. doi: 10.1186/1472-6947-12-56. Tracking Africa's Progress in Figures. (2014) (1st ed., p. 11). Retrieved from https://www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/Tracking_Africa%E2%80%99s_Progress_in_Figures.pdf Van Der Meijden, M. J., Tange, H. J., Troost, J., & Hasman, A. (2003). Determinants of success of inpatient clinical information systems: a literature review. Journal of the American Medical Informatics Association, 10(3), 235-243. Van Dyk, L. (2013). The development of a telemedicine service maturity model (Doctoral dissertation, Stellenbosch: Stellenbosch University). Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Q. 2003;27(3):425–478. Weinberg, J., Kaddu, S., Gabler, G., & Kovarik, C. (2009). The African Teledermatology Project: Providing access to dermatologic care and education in sub-Saharan Africa . The Pan African Medical Journal, 3, 16.

PAGE 44

World Bank (2011). Sub-Saharan Africa Data. Worldbank.org. Retrieved 22 February 2017, from http://data.worldbank.org/region/sub-saharan-africa

World Health Organisation. (2006). Working together for health.. Retrieved from http://www.who.int/whr/2006/whr06_en.pdf

PAGE 45

Appendix A – Interview guide Introduction I am Sonia, a Master’s student at the University of Amsterdam. I am currently working as a Monitoring and Evaluation intern for YAPILI’s pilot project. YAPILI connects you with a licensed doctor to get health advice. As a YAPILI beta tester, you have tested the platform. I will briefly explain the whole process of what exactly happens on YAPILI. First, you register as either an advisee, a person who wants to obtain health advice or an adviser, a person who provides the health advice. In order to register, you are asked to fill in a questionnaire with various personal details. Once the account is activated, you can choose one adviser from a choice of three, who would be your physician on YAPILI. When the advisee wants to obtain some health advice, he/she can send a message to his/her adviser. The adviser will be notified of the incoming message, to which he/she can reply with the health advice. When the advisee and adviser are happy with the health advice, the adviser can then close that particular case of consulting. The YAPILI team will like to know experiences of users/doctors, like you with YAPILI, for example what you like and dislike about the platform. I would be asking you some questions about your experiences from the time you signed up on YAPILI. Your experiences will be valuable to YAPILI as they will help us improve the platform and make the platform as per your needs. Please feel free to talk about your experiences, either positive or negative. Everything you speak will be kept private and confidential. The interview will take around 45 minutes. I would like to record the interview with your permission. If you have any questions right now or at any point during the interview, please feel free to ask me. Interview questions 1. What were the reasons for you to sign up on YAPILI?

2. From what device do you normally use YAPILI? How did it go? - in terms of being accessible from your device, information provided, finding what you were looking for, or knowing what to do next, technical support if needed 3. When you were on the YAPILI website, registering as a new user, what was it that you liked? 4. When you were on the YAPILI website, registering as a new user, what was it that you didn’t like? -ask about the adviser and advisee questionnaire - was the length of the questionnaire acceptable, would you like to change something about it? 5. Did you feel comfortable sharing the information asked in the questionnaire?

PAGE 46

6. Will you prefer to choose an international or a local adviser and why? 7. At this moment, the only way that you can talk to your doctor/patient is via text messages and by attaching some pictures or documents. Is there something that you would like to change about the way you communicate with your doctor in the future? 8. Do you think talking to the doctor/patient via YAPILI could have an influence on the doctor-patient relationship? -response time, local or international doctor/patient, any cultural differences, level of comfort for talking with the doctor/patient. 9. While using YAPILI, was there any moment or any situation that made you feel unsafe?- any security or privacy issues, confidence in the system 10. How was the work load for you to handle? Too many, too less, just right number of patients (doctor specific question). 11. Would you need training to use such a platform? (doctor specific question) 12. In your perspective, what are the advantages of using YAPILI? 13. Would you continue to use YAPILI in the future? What would your reasons be to do so and would you recommend it to a family member or a friend? 14. What was your overall experience with YAPILI? Positive or negative things about platform.

End of interview guide

PAGE 47

N.B. The following table indicates which question from the interview guide corresponds to which dimension of the framework. This table isn’t part of the interview guide, but shown here for the reader’s clarification.

Dimension of BE

framework Interview question

System quality 3, 4, 5, 7, 9

Information quality 2

Service quality 2, 11

Use 13

User satisfaction 1, 3, 4, 11, 12, 14

Quality of care 1, 6, 7, 8, 12

Access 1,12

Productivity 1,12

PAGE 48

Appendix B – Coding Sheet

PAGE 49

Appendix C – Doctors’ prioritisation exercise

A few weeks ago, I contacted you to discuss your experience, or anticipated experience, with YAPILI. On behalf of the YAPILI team, I would like to thank you for your valuable feedback and inputs. Based on your inputs as well as those of other doctors on YAPILI, certain suggestions of additional issues/features have been put forth for improving the platform. I have listed, in short, the main suggested issues/features below. In order to better help us improve the platform so as to best serve you, we would greatly appreciate if you ranked these issues/features in order of importance from 1 to 10, where 1 is the most important and 10 is the least important issue/feature in your perspective. Kindly send in your responses before Wednesday, 14th June. Thank you once again. Please fill in the ranking after every statement.

• Being able to edit medical history of the patient from the doctor's account • Adding a feature in which patients are asked predefined health-related questions in

the chat box, in order to receive more and better quality of information from the patient from the beginning of the case

• Addition of a voice chat function • Addition of a video chat function • Arrange a training seminar/session for all YAPILI doctors • Provision of a small summary of healthcare systems of the country within which the

patient is based • Content of notification messages from YAPILI need to be improved, for instance the

notification when a patient opens a new case • A new feature to set the maximum limit to the number of patients you can consult via

YAPILI (in order to not receive more patients than you can manage) • Build in a function to say "I am temporarily unavailable", to let your patients know if you

are away • Provision of an Android application for YAPILI

PAGE 50

Appendix D – Patients’ prioritisation exercise

A few weeks ago, I contacted you to talk about your experience, or the experience you expect to have, with YAPILI. On behalf of the YAPILI team, I would like to thank you for your helpful feedback and inputs. Based on your feedback as well as those of other users on YAPILI, several suggestions for additional issues/features have been put forth for improving the platform. I have listed the main suggested issues/features below. In order to better help us improve the platform so as to best serve you, we would greatly appreciate if you ranked these issues/features in order of importance from 1 to 10, where 1 is the most important and 10 is the least important issue/feature in your perspective. Kindly send in your responses before 17th June. Thank you once again. Please fill in the ranking after every statement.

• Addition of a voice chat function • Addition of a video chat function • Response time should be shorter • Medical history questionnaire should be shorter • Option to choose a different doctor depending on the problem • Website should have some ready health-related content that you can look up on

the website • Website should be more interactive, for instance seeing pop-ups to notify a disease

outbreak in a particular country • There should be a doctor available to you from the country that you live in • Receive notification via email when the doctor has replied to your message • Provide an Android application for YAPILI for phone use