UAMS Way Finding App
Updated 10/11/2014
UALRUAMS Way Finding App Using the Methodology of HTI
Mario D. Wallace, Arlena Johnson, and Lisa Smith 10/11/2014
ContentsProject: UAMS Way Finding App4Purpose of
Agreement:4Project Timeline and Assignment4Method of
Communication5Need Finding-Team Way Finding6Questions and
Answers61. What is the performance gap/need you have identified?62.
How did you determine this need? (Observation methods, interview,
etc.)?63. In what environment or context does this need occur? Are
there any factors about this environment that would affect
potential solutions?64. Who is the audience affected by this need
and by any potential solution? Are there any factors about this
audience that would affect potential solutions?75. How does this
solution meet your already-defined needs?76. Describe
implementation of your solution. If its software, what device would
it run on and how would it be used?77. If its hardware, how will it
be designed and used?78. How specifically will the audience
interact with your solution?8Usability Testing: UAMS Way finding
App9Observation Form9Analyzing of Data: Observation
Statistics10UAMS Way finding Questionnaire11Analyzing of
Questionnaire Data: Emergent Themes12Proposal of Evaluation
Plan13Statement of Purpose13Method13Analysis14Instruments14Timeline
of User Experience14Usability Test Study15Methods Used15Analyzing
Methods15Analyzing of Data: Observation Statistics16Analyzing of
Questionnaire Data: Pattern Seeking17Results of Usability Test
Study18Future Developments20Reference21
Project: UAMS Way Finding AppPurpose of Agreement: This contract
describes the responsibilities of UAMS Way Finding team and spells
out roles in a timeline approach. This contract is agreed upon by
the parties named in this document: Mario Wallace, Lauren Smith,
and Arlena Johnson. This contract is consented by Dr. Mong.
Project Timeline and Assignment Project NameProject
MilestoneAssigned toMilestone Description
Need Finding
9/30/2014Revise the Need Finding charter Arlena JohnsonReview
the existing Need Finding charter and make revisions
10/5/2014Proof readMario Wallace and Lauren Smith Make edits to
the documents based on Arlena's suggestions
10/6/2014Submit updated to Need Finding assignment Mario Wallace
Submit to group
Project NameProject MilestoneAssigned to Milestone
Description
Prototype development
10/6-10/8/2014Review storyboard PPT and make suggestions Team
Review the existing design and make comments using evidence-based
research
10/8/-10/11/2014Collect data Mario WallaceResearch the internet
for similar prototypes and determine best fit, interface, and
aesthetics
10/11/2014Revise prototype Mario WallaceMake revisions as
needed
10/12/2014Submit updated prototype Lauren Smith Submit
assignment to professor
Project NameProject MilestoneAssigned toMilestone
Description
Create evaluation plan
10/15/-10/20/2014Determine method for testing productIndividual
assignmentDetermine evaluation strategy
10/20-11/3/2014Create instrument Individual assignmentDevelop
testing tool
11/3/-11/11/2014Time user experienceIndividual assignment
Explain user testing
11/12/2014Submit final report Individual assignmentSubmit final
draft
Project NameProject MilestoneAssigned toMilestone
Description
User testing
11/13/-11/17/2014Synthesis evaluation plan Team (each team
member vote on portions from each member's evaluation plan)
Synthesis material for draft
11/18/-11/24/2014First draft of evaluation plan Arlena
JohnsonPiece together portion of individual evaluation with the
highest vote and consolidate into a draft
11/21/2014Edit draft of evaluation plan Lauren Smith Edit the
draft
11/23/2014Team discussion of draft Team (Discussion board)
Approving draft
11/24/-12/1/2014Implement our plan Team (test 2 people per team
member) Perform usability testing with at least 5 people
12/1/-12/5/2014Analyze the data Mario WallaceSummary data from
testing
12/8/2014Submit Usability testing assignment Mario WallaceSubmit
assignment to drop box
Method of CommunicationThe parties of this contract agree with
the method of communication listed below: First method of
communication for immediate needs is email correspondence via group
email in Team Way Finding page (we need to check daily); the second
method of communication is telephone usage. The communication
method for discussion and revisions of documents is via the team's
discussion board. If the discussion consist of an image, HTML, or
PDF attach to discussion post. Name EmailCell phone
Mario [email protected]
Arlena [email protected]
Lauren Smith [email protected]
Need Finding-Team Way FindingQuestions and Answers1. What is the
performance gap/need you have identified? An indoor GPS way finding
app is needed at University of Arkansas for Medical Science (UAMS).
Patients, students, family members, students, vendors, and
employees at UAMS lose their way daily around the 5 million square
foot campus. Given that we do not have an effective way finding
system, inpatient and outpatient clinics are losing millions of
dollars annually because of missed appointments by patients,
students are not reporting to class in a timely manner, vendors get
frustrated and leave supplies in unauthorized locations, and
employee are late for work and meetings. Also, based on evidence
provided by the clinic managers, patients are taking their health
care needs to other facilities because of way finding concerns. 2.
How did you determine this need? (Observation methods, interview,
etc.)? The 2012 Leadership Institute at UAMS conducted a
Patient-family-centered care study using a qualitative analysis of
the current state of the institution. Patient, family members, and
employees participated in the study and the outcome of the study
indicates that an Indoor GPS way finding app was needed. Students
at UAMS were not included in the scope of the Patient-Family
Centered Care assessment, but for this project, we will survey them
to get their feedback to assist in determining if the Indoor GPS
Way Finding app is applicable at UAMS. Daily data from patients,
family members, and employees who visit the information desk
seeking directions. Daily observations of wandering patients,
family members, vendors, employees. Data gathered from medical
charts and calendar appointments in the inpatient and outpatient
clinics regarding missed appointments. 3. In what environment or
context does this need occur? Are there any factors about this
environment that would affect potential solutions? This need occurs
at UAMS, which is a health care, academic, research, and employment
environment. As patients, family members, students, vendors, and
employees enter the campus, they get turned around based on the
poor signage and parking locations. When they enter the buildings,
many appear disoriented or lost. The funding of this venture
including the cost of development and advertisement, which could
affect the potential solution. The download of the app will be
offered for free, but there will be an associated cost for
development, advertisement, and marketing with the release and
upkeep of the app. 4. Who is the audience affected by this need and
by any potential solution? Are there any factors about this
audience that would affect potential solutions? Patients: patients
miss appointments because they are not familiar with the layout of
the institution; they become confused with the current signage of
the way finding system. Older patients and or visitors (digital
immigrants) who do not use Smartphones and other technologies will
affect potential solutions. Family Members: family members become
frustrated with the current signage way finding system. Students:
students that come to study at the facility are affected by this
need. Students who are not familiar with Little Rock or UAMS, would
need to know how to find their way around and also be informed
about the app and how to help the other audience members mentioned.
On campus and off campus Employees: on campus employees are not
familiar with all of the locations of the institution; off campus
employees Vendors: vendors do not know delivery sites; thus, not
knowing creates frustration. Special needs audience: members and
individuals that are hearing and vision impaired need to be
considered. 5. How does this solution meet your already-defined
needs? This solution will meet already defined needs by aiding the
audience that enters the campus, with a way to stay on track. The
app will use the current layout of the hospital to aid the audience
through audio and visual routing. The design will aid the special
needs audience by making sure that the app can work with the
accessibility options provided on smartphones and other
technologies. For example: Apples accessibility options allows for
bold text, a hearing aid mode that allows the phone to sync to the
hearing aid to improve audio quality. It also allows text size to
be increase for those with vision problems. Text to speech is
another added feature to aid in accessibility among many
others.
6. Describe implementation of your solution. If its software,
what device would it run on and how would it be used? The app will
be available on all platforms for Smartphone and smart device
users. 7. If its hardware, how will it be designed and used? It is
actually a software that will be designed by the IT department at
UAMS. I also have the name of the company that developed the indoor
app for Miami Children's Hospital. 8. How specifically will the
audience interact with your solution? The audience will interact
with it via any mobile device. The Indoor GPS way finding app
differs from any other app because it currently does not exist here
in Arkansas hospitals. It will be very beneficial to end a UAMS
campus-wide problem with missed patient appointments, last student
arrivals, missed employee meetings, and frustrated vendors. The
UAMS way finding app is used for indoor navigation purposes and
will not be available off campus. The main campus consist of more
than 20 buildings and this makes navigation throughout the campus
difficult. The app will mainly operate via Wi-Fi, campus wide. A
cell connection will be required only if the Wi-Fi connection is
unavailable at the time of access.
Usability Testing: UAMS Way finding AppObservation
FormParticipant Name: _____________________________ Date:
__________________________________TaskDescriptionTimeoutCriterionErrorsAssistsTimeCompleted
AOpening the Way finding app 0:00:10
Comment:
BSigning In the Way finding app0:01:50
Comment:
CSelecting navigation paths0:00:30
Comment:
DSelection locations0:00:25
Comment:
ESelecting from the Directory0:00:50
Comment:
FNavigating the path0:01:50
Comment:
Analyzing of Data: Observation Statistics ParticipantsAssisted
Total Task Completion Rate %Unassisted Total Task Completion Rare
%Total Task Time (h:mm:ss)ErrorsAssists
1
2
3
4
5
6
Timeout Criterion
Mean
Standard Dev
Min
Max
UAMS Way finding Questionnaire Participant Name:
__________________________________ Date:
____________________________
Strongly DisagreeAverageScoreStrongly Agree
1. The app had a clear, clean, uncluttered interface
design12345
Comment:
2. The app perform the way you expected 12345
Comment:
3. There wasn't anything confusing about the app 12345
Comment:
4. The steps in the app's sign in were minimum and reasonable
12345
Comment:
5. The app's navigation selection was clear and easy to
operate12345
Comment:
6. The app has a easy to use directory 12345
Comment:
7. The app's color scheme is suitable and not overwhelming
12345
Comment:
8. The app's icon is recognizable as a Way finding app 12345
Comment:
9. The app is very effective for its purpose12345
Comment:
10. The app provides all of the features for
functioning12345
Comment:
Analyzing of Questionnaire Data: Emergent
ThemesParticipantsStrongly Disagree Strongly AgreeTotal Score
Emergent Theme: ScoreEmergent Theme: Comments
1. Participant A
2. Participant B
3. Participant C
4. Participant D
5. Participant E
6. Participant F
Proposal of Evaluation PlanStatement of PurposeThe University of
Arkansas for Medical Science (UAMS) Way finding App is one of the
first indoor GPS systems for a healthcare facility in the state of
Arkansas. When visiting UAMS' main campus, Patient, family members
(visitors), students, and employees will be able to use the app to
navigate throughout the 5 million square feet facility to reach
their destinations.
In the recent past, a survey was administered to patients,
family members, students, and employees to assess the current
signage way finding system and the results follow:
Patients: patient miss appointments because they are not
familiar with the layout of the institution; they become confused
with the current signage was finding system. Family Members: family
members become frustrated with the current signage way finding
system. Students: students that come to study at the facility are
affected by this need. On campus and off campus Employees: on
campus employees are not familiar with all of the locations of the
institution; off campus employees are not familiar with even less
location than their counterparts
The results of the survey was quite alarming and prompted an
investigation into a more viable and effective way finding system,
which lead to the development of the way finding app. The way
finding app will be accessible via any mobile devises: Smartphone,
tablet, etc.MethodThe stratified random sample of 6 (n=6)
participants will be used in this usability study. The group will
include a mixture of patients, family members, employees and
students. There will be two methodologies used in the usability
study: empirical research (Observation) and non-empirical research
(Questionnaire). The observation study will be conducted with one
participant and the observer per setting. While the participants
attempt to perform the function of the way finding app in the
presence of the observer, the observer will annotate performance
and assist when called upon. According to empirical research seeks
to "Develop or explore models that describe or predict behavior"
(Mackenzie).
The questionnaire will be conducted minutes after the
observation study by the observer. The observer will administer the
questionnaire, and the participant to independently complete it.
The observer will assist with reading and/or clarifying questions
if necessary. According to non-empirical research seeks to
understand the thought, feeling, attitude, emotion, reflection,
sentiment, opinion, mood, outlook, manner, approach, strategy, of
the participant in the study" (Mackenzie). Analysis The data from
the usability study will be analyzed using both quantitative and
qualitative methods. The quantitative, descriptive method for the
study will analyze the mean, standard deviation, maximum and
minimum observation scores; the qualitative method for the study
will use the Emergent themes approach. The data from the
questionnaire will be analyzed for the purpose of identifying
emergent themes. Instruments The way finding usability study will
include two instruments for gathering data. The observation form
was developed as an instrument that the observer will use to assess
the performances the participants to determine time of completions,
averages, standard deviations and etc per participant and as a
group. The form consist of sections "A" through "F" and includes
the measurable factors time, errors, assists, and comment themes;
the questionnaire form includes 10 questions, a comment section per
question, and a score per question. The questionnaire was developed
as a tool for gathering qualitative data that specifically solicits
emergent themes. Timeline of User Experience There will be a time
factor per activity. The factor is contingent per performance task.
See attached observation form.
Usability Test StudyMethods UsedQualitative
approach(Observation): The observation study was conducted with one
participant and one observer per setting. While the participant
attempted to perform the function of the way finding app in the
presence of the observer, the observer annotated performance and
assisted when called upon. All participants were observed at
various locations (environments) and times throughout the day, so
that they can felt comfortable during testing.Quantitative
approach(questionnaire): The questionnaire consisted of 10
questions using a5-point Likert scale.The participant had three
options for completing the questionnaire: hard copy, mobile device,
or email. The hard copy questionnaire was administered by the
observerminutes after the observation study and collected by the
observer for analysis. The mobile device questionnaire was
administered minutes after the observation or within 24 hours of
the testing. The observer did not have to administer the
questionnaire and the participant submitted it via mobile device.
The email questionnaire can be administered within 24 hours after
the testing. The observer does not have to administer the
questionnaire and the participants can email it to the group member
who conducted the observation.
Once the participant completed the observation and
questionnaire, the testing for the participant ended.Analyzing
Methods Quantitative approach 1: the data from the observation was
analyzed using an observation form that includes the following
measures 1) errors, 2) assists, 3) time of completion, and 4)
observer comments.The group used percentages to rate errors,
assists, and time of completion for each participant and for the
group. The time per participant was compared to the completion time
of an expert way finding user.
Quantitative approach 2: The questionnaire will be analyzed
using a 5-point Likert scale. The group analyzed themes discovered
from comments and scoring.Identify at least 5 people who will
volunteer for product testing (you don't need to share specific
names ... daughter, co-worker, etc.) Each member of the way finding
group identified two participants that fit the description of our
targeted population, and although we administered the observation
study at various times, all of the information was collected and
analyzed for reporting. Each participant was known by a member of
the group and each participant understood that the evaluation was a
study for a graduate level course.
Analyzing of Data: Observation Statistics ParticipantsAssisted
Total Task Completion Rate %Unassisted Total Task Completion Rare
%Total Task Time (h:mm:ss)ErrorsAssists
1. Participant A0%100%0:1:1200
2. Participant B66%33%0:2:6614
3. Participant C50%50%0:4:0103
4. Participant D0100%0:2:3600
5. Participant E50%50%0:3:0203
6. Participant F0100%0:1:5800
Assisted/Unassisted Group Total Task Completion Rare %50%50%
TimeoutCriterion0:5:58
Grouped Total0:14:751 (2%)10 (27%)
Mean2.4
Standard Dev1.0
Min0:1:12
Max0:4:01
Analyzing of Questionnaire Data: Pattern
SeekingParticipantsStrongly Disagree Strongly AgreeTotal Score
Pattern Seeking: ScorePattern Seeking: Comment
1. Participant A(36/10) 3.6363Three was the most prevalent
selection
2. Participant B(30/10) 3Average(30/10) 3Average303Three was the
most prevalent selection
3. Participant C(46/10) 4.6465Five was the most prevalent
selection
4. Participant D(48/10) 4.8485Five was the most prevalent
selection
5. Participant E(44/10) 4.4445Five was the most prevalent
selection
6. Participant F (42/10) 4.2425Five was the most prevalent
selection
TotalMeanStandard DevMinMaxStrongly Agree
246416.78304883%
Results of Usability Test Study To analyze the data from our
usability test study, we examined observational data as well as
data from a questionnaire with a Likert scale from 1 through 5. The
observational study used a timeout criterion, the average time per
step as determined by the apps designers to complete the
observation test, as a gauge to measure the participants
performance and the following categories below: 1. Assisted total
task completion rate2. Unassisted total task completion rate3.
Total task time 4. Errors5. Assists
The usability test study included 6 participants of which three
were women. All of the participants scored less than the timeout
score 0:5:58, which was a good indicator that the app provides both
utility and usability. The total grouped time was 0:14:75, the
minimum time was 0:1:12, and the maximum time was 0:4:01. The mean
completion time was 2.4 minutes, and the standard deviation
completion time was 1.0 minutes. In fact, the minimum time for
completion is less than 4 standard deviations way from the timeout
time criterion score. There were only 10 (27%) total assists, which
50% of the group needed assistance; only one (2%) error was
observed throughout the test. The purpose of the 10 question
questionnaire was to gauge each participant overall score and to
seek patterns in the response to data. Using a 1 to 5 point Likert
scale, the participants had to rate each question and comments on
the rating. The categories for the questionnaire follow: 1.
Strongly disagree2. Strongly agree3. Total score4. Pattern seeking
score5. Pattern seeking comments The total score was 246 in which
the minimum score of 6 participants was 30 and the maximum score
was 48. The mean was 41 and the standard deviation was 6.78. The
pattern seeking scores were quite interesting. Eighty-three (83%)
of the participants strongly agreed with the utility and usability
of the app. Two (2) of 6 (33%) participants gave an average 3
rating to the questions and the other participants 4 (66%) gave an
average 5 rating. Fifty percent of the participants gave the
following four categories a 66% score. 1. The app had clear, clean
and uncluttered interface (66%)2. The steps of the apps sign in
were minimum and reasonable (66%)3. The apps color scheme is
suitable and not overwhelming (66%)4. The app is very effective for
its purpose (66%)5. The app provides all of the features for its
functions (66%)
The scores in the other categories were 30% and less, and the
questions are not as impactful as the other categories were
insignificant Regarding comments, only 50% of the participants
chose to comment, and their comments discussed the effectiveness of
the apps utility and usability. Hence, the usability test study
provides evidence that the apps usability and utility are
effective. The feedback given provided information that leads to
the conclusion that more testing is needed. The results showed that
the application is successfully but just how much of an impact will
it make.
Future DevelopmentsThe future developments concerning should the
application should include but are not limited to: A larger sample
size of participants is needed. The sample size should be random
and include those of ages from young adult to senior citizens. This
way, more data will reveal just how useful this application is to
individuals of various backgrounds. The level of technology
experience. Based on the feedback that was received, some of the
users are not very comfortable with technology. Especially
something that is new to them. Participants should be encouraged to
provide more feedback, in order to get a better measurement of how
well the application did or did not perform for them. This will
also provide more insight into how the participant really feels
about the application. For example, is it something that they will
continue to use in the future? Do they feel that it will be
beneficial? Why and how? Future developments should also include
additional functions for the way finding app. The app should
include functions for ordering items from the gift shop and the
eateries at UAMS. Patients and visitors should have the convenience
of ordering room service for food from the 6 eateries wanted items
from the gift shop. This application is beneficial to the lives and
careers of all parties involved. Starting with the student that is
attending classes at UAMS for the first time and on to the friends
and relatives of a loved one. This way finding application will
change things for the better but only after more testing to ensure
proper and effective distribution.
ReferenceMackenzie, I. (2002, February 15). Empirical Research
Methods in Human-Computer Interaction. Retrieved November 2, 2014,
from http://www.yorku.ca/mack/CourseNotes.html
3
