TEEM’15

PHD PROGRAMME KNOWLEDGE OF SOCIETY

PORTO 7-9 OCTOBER 2015

SYNOPSIS OF A DISCUSSION SESSION ON DEFINING A NEW QUALITY PROTOCOL FOR MEDICAL APPS

LAURA BRIZ-PONCEJUAN ANTONIO JUANES-MÉNDEZ

FRANCISCO JOSÉ GARCÍA-PEÑALVO

Technological Ecosystems for Enhancing Multiculturality TEEM’15

Photo retrieved from http://teemconference.edu

TEEM’15

INTRODUCTIONMOBILE LINES IN THE WORLD1

USE OF MOBILE DEVICES IN EDUCATION2

NUMBER OF MOBILE APPS IN MARKETPLACES4

+6.800 MILL

+67% STUDENT

S

+ 1 MILL

QUALITY PROTOCOL FOR EVALUATING MEDICAL APPS

USE OF MOBILE DEVICES BY PHISICIANS360% SMP

44% TABLETS

1International Telecommunications Union. 2013. ICT Facts Figures 2013. Retrieved from: http://www.itu.int2Educause Center for Applied Research [ECAR] 2012. ECAR Study of Undergraduate Students and Information Technology. Louisville, CO:3Reston, V. (2012). New Study on Physician Online Behaviors. Comscore Inc. Recuperado a partir de https://www.comscore.com/ 4Number of mobile apps available in Apple Store y Google Play (2013) Statista. Retrieved from: http://www.statista.com

TEEM’15

PREPARATIONINTRODUCTION

INFORMED CONSENT

DOCUMENTATION DISTRIBUTION

FILLING CONTEXT QUESTIONNAIRE

TASK LISTSAPP EVALUATION BY USERS

DATA ANALYSIS AND VALORATION

PROCESS

TEEM’15

WOMEN80% STUDENT

S50%

PHYSICIANS50%

PARTICIPANTS

MEN20%

SMP50%

SMP&TABLET

40%

18-2550%

46-5540%

+5510%

NONE10%

TEEM’15

APP: BRAIN SYSTEM 3D

This application offered a 3D simulation for medicine

Professionals of Medicine or students can select any morphological structure of the human brain and will be able to rotate it with different angles

TEEM’15

Descripción

Nivel 3

Nivel 2

Nivel 1

Nivel 0ReliabilityAssess the information clarity of the external data (description, target of the app, developer, responsible of the app, etc)

NavigationRelated with the use of mobile app (nº clics, screen, suitable adjusted, navigation map, relevant information easy to find…)

70%-80%Security/Privacy

Privacy Policy (explanation of how the data are treated, provide information of the store of it, if applies, etc)

ContentSources of the content, keep updated information, expers involved in the development, useful content

Design/UIAssess the user interface (buttons, images, pictures, help information, simplicity , easy to memorize, fast to download, visual content, etc) contenido visual.)

CER

TIFICA

CIÓ

N

Nota1: Previous research came out with the selection of the dimension for this QoP2.

PROTOCOL

TEEM’15

DOCUMENTATION

TASK LISTS

EVALUATION TEST

• 22 TASKS

• PASS/NO PASS

• TASKS GROUPED ACCORDING TO DIMENSIONES

• 45 ITEMS

• LIKERT SCALE

• GROUPED ACCORDING TO THE DIMENSIONS

TEEM’15

TASK LISTS92% Stud vs 85% Profess performed tasks OK

9% Stud vs 15% Profess performed tasks NOK

FREQUENCY FISHER FREQUENCY FISHER

Task %OK %NOK Task %OK %NOK

T1 80% 20% 0,44 T12 90% 10% 1

T2 50% 50% 0,206 T13 90% 10% 1

T3 40% 60% 0,048 T14 80% 20% 0,444

T4 90% 10% 1 T15 100%

0% cte

T5 100%

0% cte T16 100%

0% cte

T6 100%

0% cte T17 100%

0% cte

T7 100%

0% cte T18 100%

0% cte

T8 90% 10% 1 T19 100%

0% cte

T9 100%

0% cte T20 100%

0% cte

T10 100%

0% cte T21 90% 10% 1

T11 100%

0% cte T22 100%

0% cte

RELATIONSHIP TASKS

Significance difference on Task 3 (Fix the image) between the result and the profile of participants

RESULTS

Nota: non-parametric Chi-square method applied

TEEM’15

020406080

100 SCORE DIMENSION

Score students Score professionals

SCO

RE

SCORE DIMENSION

SECURITY

RELIABILIT

Y

NAVIGATIO

N USER

INTERFACE

CONTENT

• Relationship between the score of dimensions and the participants’ profile, except reliability dimension

• No relationship between score and the type of mobile devices

RESULTS

PROFILE VS SCORE DIMENSION  T-student

(normal)

Reliability 2,262 0,054Navigatio

n3,391 0,009

  U Mann Whitney

 

User Interf

0 0,009

Security 1,5 0,019

Content 2,5 0,036

MOBILE DEVICE VS SCORE DIMENSION  ANOVA(Normal)

Reliability

0,375 0,7000

Navigation

0,234 0,798

  Kruskal-Wallis  User

Interf1,458 0,482

Security 1,551 0,460

Content 1,955 0,376

• User Interface and Navigation dimension were the ones best scored.

• Students gave

higher scores to all dimension. RELATIONSHIP SCORE DIMENSION

TEEM’15

RESULTSTOTAL SCORE

• Relationship between the total score and the participants’profiles

• No relationship between the total score and the type of mobile device owned by participants

Box Plot Total Score

Median score for students is higher than for professionals

  Profile Mobile Device  T (T-Student

Method) F (ANOVA

Method)

Total Score 4,66 0,002 0,715 0,522

RELATIONSHIP TOTAL SCORE

TEEM’15

VALORATION

FINAL EVALUATION 50% NOT ENOUGH TO HAVE THE CERTIFICATION LEVEL

62 % Studen

ts

TOTAL VALORATION

APPS

CRITERIA• % Participants

that gave score ≥ 4

• Needed Val App> 70%

39 % Profession

als

TEEM’15

PHOTO GALLERY

TEEM’15

CONCLUSIONSeem that the profile of the participants

influences on the score of the apps .Seem that the ownership of mobile

devices does not influence on the final score of the apps

The number of samples is small. Future research must be performed to increase the number of samples

This study is the first step to make a final valoration of the apps.

THANK YOUTEEM’ 15