TryMyUI Usability Report

8/20/2019 TryMyUI Usability Report

1/24

TrymyuiUX wars case competition

& sta travel

student universe


2/24

Table ofContents

Executive summary 1Purpose of The Study/ Introduction 2Methods 3Student Universe Results 5STA Travel Method Results 8

Findings & Recommendations 11Site Comparison 13Appendix 14


3/24

Prior to the start of the usability test, facilitators provided participants with instructionsfrom a predetermined script to ensure a uniform experience for all. A total of eight par-ticipants were tested and each was administered apre- and post-test questionnaire. TryMyUI Usability Testing Services was used to recordscreen captures of each user’s clicks and audio of each user’s verbal feedback duringtesting. Usable testing data was obtained from all eight participants.

The results of the usability test are as follows:

Executive

Summary

1

Both StudentUniverse.com and STATravel.com are student-centered travel bookingwebsites. Both websites provide flight, hotel and tour booking services for studentswho are planning for individual or group travel. Each website boasts cheap rates,discounts and deals for their target users. Both websites also allow users to findflights, hotels and tours from prominent navigation at the top of their homepages.

The purpose of this usability study was to compare the online bookingexperiences of StudentUniverse.com and STATravel.com users to determine theoverall effectiveness of each website’s design, the ease-of-use of booking features,

and whether or not the flow or process of information on eachwebsite is efficient. Points of interest included whether or not participants couldsuccessfully book a flight from and to specified airports and within a determinedbudget, successfully book a hotel room within a determinedbudget and distance from the destination airport, locate hotel ratings, and book atour in the destination city. To answer these research questions, ascenario was created to test users’ ability to book a flight, hotel room and tour onboth websites.

The scenario and task list, presented digitally via TryMyUI Usability Testing Ser-vices, asked participants to plan a seven-day trip to Phuket, Thailand, during March18-28, 2016. Following an impression test of each website, participants needed tofind a round trip flight departing from Raleigh-Durham International Airport (RDU)

to Phucket International Airport (HKT) under $1700, find a four or five star hotelwithin 30 miles of the airport for less than $200 a night and book a tour during thetrip to Phuket.

COMPLETION RATE PER TASK Student Universe

Task 1 - 100%Task 2 - 75%

Task 3 - 50% Task 4 - 100%

STA Travel Task 1 - 100% Task 2 - 100% Task 3 - 100% Task 4 - 100%

COMPLETION TIME PER TASK Student Universe

Task 1 - 1:53Task 2 - 1:49

Task 3 - 0:41Task 4 - 1:52

STA Travel Task 1 - 1:50 Task 2 - 0:75 Task 3 - 2:24 Task 4 - 5:34

STUDENT UNIVERSE ERRORSMultiple participants had ahard time finding a tour when

they searched by the full name of the destinationusing both the city and country.

STA TRAVEL ERRORS Two out of four of the par-ticipants did not select the “student” option whenbooking a flight.All participants showed frustration and/or confu-sion while trying booking a hotel room after book-ing a flight. Three out of four of the participantsverbalized confusion while booking a tour at somepoint during the task.

TASK LEVEL SATISFACTION Student Universe Usability average - 68.75% Letter grade of a “C” STA Travel Usability average - 64.83% Letter grade of a “C”

RECOMMMENDATIONS Our research concludes that Student Uni-verse should implement consistent design acrosspages, organize listed information in chronologicalor alphabetical order and employ visual elements

to establish a stronger information hierarchy. DorSTA Travel our research suggests strengtheningthe flow of information from page to page.


4/24

Purpose

Upon launch, StudentUniverse.com and STATravel.com allow users to findflights and hotels from a prominent search bar at the top of the page. Inaddition, STATravel.com allows users to search for tours in the same searchbar, while StudentUniverse.com does not. For flights, users are prompted for

from and to locations, dates, whether or not the dates are flexible, and thenumber of persons on the trip. For hotels, users are prompted for thedestination, check-in and check-out dates, and the number of personsneeding a room. Furthermore, STATravel.com asks for the number of roomsneeded. Both websites offer additional information, such as details aboutdiscounts, deals, travel insurance and other travel services.

The features chosen for evaluation included booking options for flights, hotelsand tours on each website. The target users for these websites includestudents looking to travel the world at affordable, reduced rates. While userscan book domestic travel, the websites offer the most options for studentstraveling abroad.

The goal of this usability study was to compare the online bookingexperiences of StudentUniverse.com and STATravel.com users to determinethe overall effectiveness of each website’s design, the ease-of-use ofbooking features, and whether or not the flow or process of information oneach website is efficient. Participants completed a pre-test questionnaire,then began the test by completing a visual impression test of the websites togain a sense of their understanding of what the websites are about and whatthey offer. During the test, participants interacted with booking componentsfor flights, hotels and tours, which included entering destination details, datesand other factors. These booking components are the core functions of bothwebsites, therefore, their functionality, organization and effectiveness are critical to each website’s ability to attain and retain users. After the test,participants completed a post-test questionnaire and rated their experiences

with each task.

2

Both StudentUniverse.com and STATravel.com are student-centeredtravel booking websites. Both websites provide flight, hotel and tour bookingservices for students who are planning for individual or group travel. Eachwebsite boasts cheap rates, discounts and deals for their target users. Bothwebsites also allow users to create accounts, however, this is not necessary toutilize search features of the website.


5/24

METHODSPARTICIPANTS. Test participants included eight users, all age 22, whohad no prior experience using StudentUniverse.com or STATravel.com.

Participants were selected via in-person requests. The entire groupconsisted of undergraduate and graduate college students, of which sixwere female and two were male. Under no segmentation criteria, fourparticipants were selected to test StudentUniverse.com and theremaining four were selected to test STATravel.com.

Two participants had prior experience using StudentUniverse.com and/or STATravel.com. Six participants had previously used online book-ing/travel websites to plan travel, and all but one had traveled abroad1-3 times in one year. All participants planned their travel online, halfvia third party websites and half via airline website. Individual flightsbooked in the past year by the eight participants varied from 0-6. Allparticipants were interested in traveling abroad, and three had plans tostudy abroad.

According to Alexa.com, the average users of both StudentUniverse.com and STATravel.com are overwhelmingly college-educated femalesin the United States who access the website mostly at work, but also athome or school. Considering these demographics and the target userdemographics outlined by each website, the participants of this usability test are among the intended audience for both StudentUniverse.com and STATravel.com. users ability to complete eachbooking task.

3

TASKS. Participants were introduced to TryMyUI Usability TestingServices, which was the online tool used to conduct the usability tests.Within the system, participants were presented with a scenario andfive tasks, which included an impression test and reading the “frame ofmind” or scenario out loud. The scenario was designed to test anindividual user’s complete booking experience, from travel to activitiesin the destination. The following content was provided to each p

articipant via TryMyUI:

Scenario

You are going on a spring break trip to Phuket, Thailand to explore thearea on your own. Plan a seven day trip between March 18th-28th toThailand. Leaving from the Raleigh-Durham Airport (RDU) to PhucketInternational Airport (HKT).Tasks 1. Perform a short impression test. 2. Read the frame of mind scenario out loud. 3. Find a round trip flight from Raleigh Durham International

airport to Phuket, Thailand under $1700

4. Find a 4 or 5 star hotel within 30 miles of the airport for less than $200 a night 5. Book a tour during your trip to Phuket, Thailand

The scenario and tasks were chosen to test the overall effectiveness ofeach website’s design and the ease-of use of booking features.Furthermore, the tasks were chosen to test whether the flow or processof information on each website is efficient. It was assumed that thevariance in the organization of each website, specifically an abundanceof information versus a simple structure, would greatly impact eachusers ability to complete each booking task.


6/24

4

FACILITIES. Participants were tested on Mac desktop and laptopcomputers in the Interactive Media Masters Program editing bays onthe second floor of Elon University’s Powell building. Testing facilitatorsremained in the room for the pre-test questionnaire, and observedtesting and the post-test questionnaire through a glass wall fromoutside of the room.Test Administrator Tools

The identities of test participants will remain confidential. In thereport, participants are referred to by number rather than by name.The TryMyUI tool used the computer’s webcam and microphone torecord screen captures of each user’s mouse movements and clicks andaudio of each user’s verbal feedback during testing. The system alsoincluded post-task questions, which asked participants if they wereable to complete the task and how difficult they thought each taskwas. Pre- and post-test questionnaires were presented to participantsin printed, physical form.

The pre-test questionnaire included 10 open- and closed-endedquestions that centered on each participant’s prior experience with

online travel booking websites and their interest in traveling abroadand/or plans to travel abroad. Demographic questions regarding eachparticipants age and sex also were asked. The post-test questionnaireincluded 13 questions about each participant’s experience booking onthe website in which they tested. In addition to users rating theiroverall experience using the website and an open-ended question aboutwhat they felt were the most glaring difficulties or problems duringtheir test, the remaining questions were presented using Likert scales,from 1-5. Please refer to the Appendices for the complete post-task,

PROCEDURES. Facilitators greeted participants and introducedthemselves as participants arrived to the second floor of the Powell

building. Participants were then taken to editing bays for privacy andto minimize distractions while testing. Next, facilitators used apredetermined script to relay the nature and purpose of the testing andto give instructions to each participant. Participants were then asked toread and sign consent forms, followed by the completion of the pre-testquestionnaire.

Participants were encouraged to think out loud while completing tasks.Mouse movements and clicks and audio of each participant were recorded via TryMyUI throughout testing. Facilitators observedparticipants from outside of the editing bays, in case assistance wasneeded for any technical difficulties during testing. Upon completion ofall tasks and post-test questions within the TryMyUI system,facilitators administered printed post-test questionnaires to eachparticipant, leaving and observing from outside of the room. Finally,participants were thanked for their cooperation and dismissed withoutcompensation.


7/24

5

STUDENTUNIVERSE

RESULTSTASK COMPLETION RATE A total of four college-aged students tested thestudentuniversel.com website. All participants were able to complete task 1,75% of the participants were able to complete task 2 and 50% were able tocomplete task 3 during the usability test. Each participant made verbal orwritten notes about what they felt was unclear, confusing and/or frustratingduring a particular part of their testing experience.

TASK TIME AVERAGE On average, it took 1 minute and 53 seconds for allfour participants to complete task 1 “perform a short impression test”. Fortask 2 “read the frame of mind scenario out loud,” it took an average of 1minute and 49 seconds for all participants to complete the task .When itcame to task 3 “find a round trip flight from Raleigh Durham Internationalairport to Phuket, Thailand under $1700” it took an average of 41 seconds

for all participants to complete this task. For task 4 “find a 4 or 5 star hotelwithin 30 miles of the airport for less than $200 a night” it took 1 minutesand 52 seconds. Lastly, for task 5 “book a tour during your trip to Phuket,Thailand” it took an average of 5 minutes and 35 seconds for allparticipants to complete. Based off of the average it can be seen that mostparticipants struggled more with task 5 the most during the usability test.


8/24

SATISFACTION RESULTS. Overall, studentuniverse.com recived ausability average of 68.75% with a letter grade of a “C”. Like STATravel, for every test question each participants was able to ratetheir overall experience ranging from 1(indicating the task wasvery difficult to complete) to 7(indicating the task was very easy tocomplete). The below chart indicates the numeric grade that eachparticipant gave the corresponding task based off of the above

feedback scale. Aside from participants rating each tasknumerically, we also asked each participant to provide verbal orwritten feedback after each task or at the point of confusion.

PARTICIPANT COMMENTSParticipant 1 :

• “The worst thing about my experience were the pop-up windows. Ihate that.”

• “I would improve the color scheme/textual hierarchy of information.”• “I enjoyed the amount of photos on the landing page, offered some

solid brand identity.”“Overall,I feel like overall design could be improved. Certain buttons/elements could be augmented so they’re easier to find.”

Participant 2:

• “The worst thing about my experience was not being familiaritywith the website.”

• “I would not improve anything, I felt the website was fairly easy tonavigate.”

• “I really enjoyed that the website was really visual, not just astandard website with a ton of external ads.”

• “Overall, a really cool website.”

Participant 3:

• “I hated how there was no alphabetical order. That was terrible. Fora first time user I began to see that this site was organized but afterseeing the lack of organization with the “country list” or “conti-nent list” it just upset me.”

• “The appearance of the lists that are clearly not in alphabeticalorder.”

• “I like how professional it was! The feature that stood out to methe most was definitely how I was allowed to compare flights withleading competitors in flight booking.”

• “I think this site is great aside from the alphabetical order. Is thissite real? If so I will be doing my travel booking on here. I just wantto make sure that this is not just third party booking like “cheap-o-air” and I’m not able to actually be seated on a plane without myseat being secure.”

6


9/24

Participant 4:

• “Finding the hotel was hard because it didn’t connect to thefinding the flight process, like when I am checking out buyingmy flight ticket I would of wanted to see hotel listing near theairport below my ticket purchase.”

• “I didn’t like how it connected me to an entirely new site when Ilooked for a tour, I felt like I had to learn an entirely new layoutand site and that is confusing.”

• “The simplistic (for a travel site) layout and clear, obviousbuttons (except the feedback button is tiny idk why).”

• “I would advise the owner of the site to: put as much energy asyou did making the flight ticket finding and purchasing aspect,into the tour and hotel areas of the site. Those two aspects ofthe site should be of equal importance as the flight purchaseaspect but it seems like when this site was created those werethought of as added perks and not created as thoroughly as theflight ticket part.”

ERROR RATE.Participant 1

Error:During task 2 the participant accidentally pressed a “popup” that generated during the test directing the participant to athird-party site

Participant 2 Error: During task 4 the participant felt the need to complete aGoogle search to find hotel rating because she failed to see this op-tion on the actual site.

Error: During task 5 the participant had a hard time finding a tourwhen they searched by the full name of the destination using boththe city and country.

Participant 3 Error: During task 5 the Participant had a hard time finding a tourwhen they searched by the full name of the destination using boththe city and country.

7

Error: During task 5 the participant had a hard time looking throughthe “country” and “region” due to the countries not being alphabetized.

Participant 4:

Error: During task 4 the participant had a hard time booking a hoteldue to being unfamiliar with the site. Had a few errors due to lack oflearning curve and previously using other booking site.

Common Errors:Error: Multiple participants had a hard time finding a tour when theysearched by the full name of the destination using both the city andcountry.


10/24

STA TRAVELRESULTS

TASK COMPLETION RATE. A total of four college-aged students testedthe statravel.com website. All participants were able to complete eachtest question presented during the usability test. Although each task

was completed by all of the participants, every participant made eitherverbal or written notes about what they felt was unclear, confusingand/or frustrating during a particular part of their testing experience.

TASK TIME AVERAGE: On average, it took 1 minute and 50 secondsfor all four participants to complete task 1 “perform a short impressiontest”. For task 2 “read the frame of mind scenario out loud,” it took anaverage of 0.75 seconds for all participants to complete the task. Whenit came to task 3 “find a round trip flight from Raleigh DurhamInternational airport to Phuket, Thailand under $1700” it took anaverage of 2 minutes and 24 seconds for all participants to completethis task. For task 4 “find a 4 or 5 star hotel within 30 miles of theairport for less than $200 a night” it took 5 minutes and 34 seconds.Lastly, for task 5 “book a tour during your trip to Phuket, Thailand”it took an average of 2 minutes and 21 seconds for all participants tocomplete. Based off of the average it can be seen that mostparticipants struggled more with task 4 and task 5 the most during theusability test. Coincidentally, these task generated the most commentsfrom participants when it came to providing feedback.

8

SATISFACTION RESULTS: Overall, statravel.com recived a usabilityaverage of 64.38% with a letter grade of a “C”. For every test questioneach participants was able to rate their overall experience ranging from1(indicating the task was very difficult to complete) to 7(indicating thetask was very easy to complete). The below chart indicates the numericgrade that each participant gave the corresponding task based off ofthe above feedback scale.Aside from participants rating each task

numerically, we also asked each participant to provide verbal or writtenfeedback after each task or at the point of confusion. With this, manyparticipants commented on the a wide variety of things from the speedof the website and usability functions such as booking a hotel. Coinci-dentally, the task that received the highest rating received little to nonegative feedback. Whereas the task with the lowest numbers were notonly consistent, but the comments were consistent as well.

PARTICIPANT COMMENTSParticipant 1• “The load time while booking a flight and a hotel was horrible”• “The drop down section for the hotels was a bit overwhelming”• “The worst thing about my experience was booking the tour. It was

very challenging. I didn’t know how to specify specifically where I

was going in Thailand. The page was also very different from theflight and hotel pages.”

• “The website could be better marketed towards students. I didn’treally get that at first glance.”


11/24

• “I liked the simplicity of it. The site didn’t have a lot of extra elementsin the way.”

• “The load time was horrible. It took too long. I would definitely tryanother site before this one.”

Participant 2:

• There were a lot of things going on while I was using the site”• “I found it hard to filter through the hotel section of the site”• “It was hard to filter through when trying to find a hotel. You had to

click in a certain area in order to receive the correct search.”• “There was a lot of things going on. It was a little distracting looking at

everything and then reminding myself of the task at hand.”• “I liked how the website had multiple tabs at the navigation. It was

super easy to select what you were looking for. However, the searchbar needs some work because what I was looking for did not show upbut then when I clicked on a different part of the website it did showup.”

• “I think the website would be more efficient if the search bar wasupdated to find all the cities though I know that is a lot to ask forsometimes.”

• “I really liked how the ONE website allowed me to search for THREEdifferent things that relate to traveling abroad where usually I feel likeI would have to book my flight through one site, hotels through

• another, tours through another, etc.”

Participant 3:• “The worst thing about my experience was probably navigating from

the flight page to the hotel page. I was thinking that it was going tobe a combined experience, so I just assumed there was going to be a“book a hotel” link right from the flight landing page.”

• “Making it clear what other selections I had made in the past couldhave been improved. I don’t believe there was a “cart” or anything thatI could refer back to.”

• “It was well-designed given the amount of information on thepages. I liked how based on what you clicked (find hotel, flight, etc.)the “booking” section was right in the middle of the page. Travelsites are tough just because of the amount of info.”

• “The website was solid and welcoming, good first impression whichis super important.”

Participant 4:

• “Navigating from flight to hotel was hard to do.”• “I didn’t like how once I had my flight selected I had to go all the

way back to the home screen to search for hotels and tours aroundthere.”

• “It would be awesome if after selecting the flight different optionsfor hotels and things to book to do in the area would then show up.”

• “I realize that this system is likely having to reach out to otherservers and things to pull information relating to flight and hotelprices but it still seems like it takes too long. I, as someone in thisprogram who understands these things was fine waiting for it butother, less patient people, might not be.”

Repeating Comments • “Hard time differentiating between “Adult” or “Student” option when

selecting a flight.• “long waiting time throughout the website”• “There are items on the site i.e. finding a location that I feel can be

simplified”

9


12/24

ERROR RATE:Participant 1:

Error: During task 3, the participant was asked to find a 4 or 5hotel within 30 miles of the airport. The participant clicked onthe Hotel/Hostel link in the navigation page which led her to thehotel booking page. While on the page the participant was asked“Where would you like to go”

Error: During task 4 the participant selected “adult” when findinga hotel but chose “student” when booking a hotel.

Participant 2 Error: During task 4 the participant entered the hotel abbreviationrather than typing her desired city when booking a hotel.

Error: During task 5 the participant had a hard time finding a tourbecause she was unable to locate an area on the page where shecould select and/or enter her desired city.

Participant 3: Error : During task 3 the participant had a hard time decipheringbetween being a “student”, “under 26” or being an “adult”. Due tothis the participant select two different options “student” and “un-der 26” which caused price confusion when booking a flight. Sincethe participant chose two different options the website displayedprices for two people.

Participant 4:Error: During task 4 the participant had a hard time figuring outhow to book a hotel room. The participant found his own errorduring this process which was: He selected his flight and expectedto see a link where he would be able to find a hotel, but that was

not the case.

10

Error: During task 5 the participant looked at the Google mapto see his proximity to the airport, but realized while on Goo-gle map that the information he wanted to find was not ac-cessible on the map. He ended up guessing his proximity tomove forward with the task.

Common Errors:Two out of four of the participants did not select the “student”option when booking a flight.All participants showed frustration and/or confusion whiletrying booking a hotel room after booking a flight.Three out of four of the participants verbalized confusionwhile booking a tour at some point during the task.


13/24

STUDENT UNIVERSE SITE Student Universe’s current site is seen asprofessional, fun, and geared toward the student traveler, but our study showsthat the lack of credibility, flow of information, and information hierarchy

loses users through the purchasing process.

Results from 75% of users stated that credibility was not established initially.This can be reversed by eliminating the new window for price comparisonwith sites like “Priceline” and making it a new page within the current site.Users also noted that the comparison sites tended to load faster than thehost site, Student Universe, diminishing credibility there as well.

The navigation design of the interface guides users through various parts ofthe site with a specific order of action. Users expressedfrustration navigating from the flight page to both the hotel and tourspages. They felt that they should be walked through from step to step.

FINDINGS &

RECOMMENDATIONS

11

In this case, booking a hotel should be clearly highlighted as the next thingto do after selecting a flight and the option of a tour should follow. Whensearching for a location users often got misdirected when inputting both

the city and country. The search database didn’t recognize both, but thatwas not communicated to the users. Some users unknowingly spelled thelocation wrong and couldn’t find results. These minor problems can be fixedby using an auto-fill search bar that guesses the input of the user to avoidspelling errors and eliminate confusion.

On the tours page some users noticed an interface change while others didnot. The changed interface suggests a completely new site forcing users totake time and figure out if this site is credible and how to use the site tocontinue their task. Once on this new site users cannot navigate back tothe newer site design and must use a new browser to get to other parts ofthe site. To eliminate this, the tours page can be designed just as the otherparts of the site and coded to work in tandem with the other pages.Another point of frustration was with the order of various lists found on thetour page. Users commented that the country and dates lists found on theleft navigation bar was not in any apparent alphabetical or chronologicalorder and seemed unorganized and unprofessional.

By streamlining the design of this page and arranging information in aspecific order, Student Universe can gain credibility and users can easilyuse the site as intended


14/24

STA TRAVEL SITE STA Travel’s current site is seen as casual, clear,and welcoming, but our study shows that the primary issue with thesite was the flow of information throughout the purchasing process.Many of these issues were due to inconsistencies with design andunclear guidelines. One major issue was on the flight page whenusers were asked to select if they were an adult, student or under 26.Many users noticed that this question was not all-inclusive and chosemore than one option, thus creating an error on the system later that

attempted to book for more than one person. A common suggestionwas an inclusion of a “cart” function that saved the flight informationmaking it possible for users to move from the flight page to the hotelpage without losing their flight information. Minor recommendationsfrom users were to see an exact number of days once the travel timewas chosen on the calendar. Another point of frustration was pop-upsexperienced by one user, slow loading times that can easily be adjustedthrough web development.

12


15/24

Both STA Travel and Student Universe are both seen as fun andwelcoming sites geared towards the student traveler. While StudentUniverse’s site struggles to establish credibility with some issues ininformation hierarchy, both sites can improve the flow of information fromone page to the next. Interlacing design interfaces and concepts from pageto page helps the user smoothly through the purchasing process withoutlosing focus and getting frustrated. Both sites can achieve this seamless

flow by linking to the next “step” or page after each task is completed. Forexample, after one selects a flight there should be a link to book a hotel inclose proximity to the flight confirmation button. Overall, both sites have astrong foundation for a successful student centered booking site thatimpressed and encouraged the users to use the site in the future.

Points of

Comparison

13


16/24

Test Script

Hi, . My name is , and I'm going to be walking you through this session.

You probably already know, but let me explain why we've asked you to come here today: We're

testing a web site to see what it's like for actual people to use it.

I want to make it clear right away that we're testing the site, not you. You can't do anything

wrong here. In fact, this is probably the one place today where you don't have to worry about

making mistakes.

We want to know exactly what you think, so please don't worry that you're going to hurt our

feelings. We want to improve it, so we need to know honestly what you think.

If you have questions, just ask. I may not be able to answer them right away, since we're

interested in how people do when they don't have someone sitting next to them, but I will try to

answer any questions you still have when we're done.

With your permission, we're going to videotape the computer screen. The video will be used

only to help us figure out how to improve the site, and it won't be seen by anyone except the

people working on the project.

Throughout the test we will be administering questions to better gauge your experience. We will

start with pre-test questions to determine your background and your prior experience with the

topic. After each task is completed there will be 2-3 questions within the test. Finally, following

the completion of the test, there will be post test questions assessing your overall experience.


17/24

INFORMED CONSENT FORM

Elon University

Title of Project: COM560A Group Research Projects for Class

Principal Investigator: Qian Xu

Assistant Professor

School of Communications, Elon University

2850 Campus Box

Office: (336) 278-6454|E-Mail: [email protected]

Co-Investigator: Jordynn McKnight

Nicole McEwan

Erin Turner

Tarah Holland

1. Purpose of the Study: The purpose of this study is to gather information about Elon students’ and

Elon community members’ uses and evaluation on a number of various media interfaces in their

daily life. The study has been designed as part of a graduate course project in interactive media

strategies (COM560A) and will therefore reflect a number of different research questions being

examined by multiple subgroups of students. This project is designed to give practical experience

to students in usability test for communication media. This project will look into the uses and

effects of communication media on users.

2. Procedures to be followed: You will be asked to first interact with some media and then answer a

f i i d b h d h fill i i

mailto:[email protected]:[email protected]:[email protected]:[email protected]


18/24

7. Right to Ask Questions: You can ask questions about this research. Contact Qian Xu at

336-278-6454 with questions. You can also call this number if you have concerns about thisresearch, or if you feel that you have encountered any problem in the study.

8. Compensation: There is no compensation involved in this study.

9. Voluntary Participation: You do not have to participate in this research. You can end your

participation at any time by telling the person in charge. You do not have to answer any

questions you do not want to answer. Refusal to take part in or withdrawing from this study will

involve no penalty or loss of benefits you would receive otherwise.

10. Video/Audio: We will videotape the computer screen and record audio while you are performing

the usability testing. The video and audio will be used only to help determine how to improve the

site. Only the person in charge and the student researchers will know your identity. All data

related to this study will only be accessible to the principal investigator and the student

investigators for this study and will be kept in locked closets and/or secured computers of the

investigator and student investigator desks, in Elon university offices.

You must be 18 years of age or older to consent to participate in this research study. If you consent to

participate in this research study and to the terms above, please sign your name and indicate the date

below.

You will be given a copy of this consent form to keep for your records.

Participant Signature Date


19/24

Scenario & Tasks

Scenario

You are going on a spring break trip to Phuket, Thailand to explore the area on your own.

Plan a seven day trip between March 18th-28th to Thailand. Leaving from the Raleigh-Durham

Airport (RDU) to Phucket International Airport (HKT).

Tasks

2. Read the frame of mind scenario out loud.

3. Find a round trip flight from Raleigh Durham International airport to Phuket, Thailand

4. Find a 4 or 5 star hotel within 30 miles of the airport for less than $200 a night

5. Book a tour during your trip to Phuket, Thailand


20/24

Pre Test Questions

Background info

1. Do you have prior experience with STA Travel and/ or Student Universe?2. Interest in traveling abroad?3. Have you travelled before using online booking/travel website?4. Do you plan on studying abroad?(Closed-ended)5. How often do you travel abroad in a year?

I don’t travel abroad1-3 times4-6 times

7-10 times11 or more

Questions

1. How do you usually purchase your flights? (Open-ended)2. How many flights have you booked online in the last year?3. How often do you use third party booking sites such as Orbitz, Hotwire,

Priceline, etc.)

4. How often do you use booking sites like (Kayak,!

)?5. How often do you purchase flights directly from the airline’s sites?

Demographic Questions

1. What is your age?2. What is your sex?(Optional)


21/24

All ResultsPre Test Background 1 Background 2 Background 3 Background 4 Background 5 Question 1 Question 2 Question 3 Question 4 Question 5 Age Sex

STA Participant 4 NO YES YES YES 1-3 times Online using USAirways 0 Not often Never Usually, but I do check prices with other site 22 M

STA Participant 3 YES YES YES YES Less than once/year Travel or airline websites 3 Not very often Not very often Most often 22 M

STA Participant 2 NO YES YES YES 1-3 times Jetblue.com 5 Occasionally, 5-10 tim Rarely, 1-3 times Most often; almost every time 22 F

STA Participant 1 NO YES NO NO 1-3 times Online 1 Never Never Every time 22 F

SU Participant 4 NO YES YES NO 1-3 times Online 2 or 3 Every time Never Not often 22 F

SU Participant 3 YES YES YES NO 1-3 times Kayak & Priceline 6 Frequently Frequently Every so often 22 F

SU Participant 2 NO YES NO MAYBE 1-3 times Google search; American Airl 3 Not too often Occasionally Most often; usually 22 F

SU Participant 1 NO YES YES NO 1-3 times Kayak 4 or 5 Always Kayak 25% of the time 22 F

STA TravelPre Test Background 1 Background 2 Background 3 Background 4 Background 5 Question 1 Question 2 Question 3 Question 4 Question 5 Age Sex

Participant 4 NO YES YES YES 1-3 times Online using USAirways 0 Not often Never Usually, but I do check prices with other site 22 M

Participant 3 YES YES YES YES Less than once/year Travel or airline websites 3 Not very often Not very often Most often 22 M

Participant 2 NO YES YES YES 1-3 times Jetblue.com 5 Occasionally, 5-10 tim Rarely, 1-3 times Most often; almost every time 22 F

Participant 1 NO YES NO NO 1-3 times Online 1 Never Never Every time 22 F

22 F

Student UniversePre Test Background 1 Background 2 Background 3 Background 4 Background 5 Question 1 Question 2 Question 3 Question 4 Question 5 Age Sex

Participant 4 NO YES YES NO 1-3 times Online 2 or 3 Every time Never Not often 22 F

Participant 3 YES YES YES NO 1-3 times Kayak & Priceline 6 Frequently Frequently Every so often 22 F

Participant 2 NO YES NO MAYBE 1-3 times Google search; American Airl 3 Not too often Occasionally Most often; usually 22 F

Participant 1 NO YES YES NO 1-3 times Kayak 4 or 5 Always Kayak 25% of the time 22 F


22/24

Post-Test Questions

Rate your overall experience with the STA Travel and Studentuniverse websitea. Very Unpleasantb. Unpleasantc. Neutrald. Pleasante. Very Pleasant

Overall the site was easy to navigate?Disagree 1 2 3 4 5 6 7 Agree

Booking a tour on the site was easy to navigate?

Disagree 1 2 3 4 5 6 7 Agree

Booking a hotel room on the site was easy to navigate?

Disagree 1 2 3 4 5 6 7 Agree

Did you find the needed information needed to complete all of the task? ( Changethe way the question is phrased)

Not at all satisfied Very satisfied

1 2 3 4 5

Rate your experience on the website in regards to finding and booking a hotel

Very difficult Very easy

1 2 3 4 5


23/24

Are you willing to register to be a member before you book the ticket/theproduct?

Not agree Agree

1 2 3 4 5

If you can change anything about your experience on the site what would youchange?

In general what was the most glaring problems you experience?


24/24

AllResults

Post Test Question 1 Question2 Question 3 Question 4 Question 5 Question6 Question 7 Question 8 Question9 Question 10 Question 11 Question 12 Question 13

SU Participant 3 Pleasant 6 7 7 4 4 5 4 5 5 4 I dislike how the lists f or countries/cities wasnt in al[habeticall order Same as a bove

SU Participant 1 - 6 6 5 4 4 4 4 4 4 4 I just was unfamiliar with it. It would get easier with more use! -

SU Participant 2 Pleasant 7 3 6 4 5 3 4 2 3 2 Pop-up windows; Couldnt find a tour for Phucket, Lack of textual hierarchy; some info hard to read; color scheme

SU Participant 4 Neutral 5 2 1 1 2 1 3 5 3 1 I didnt like that Ihad to exit the flight ticket purchasing area to find ahotel -pretty annoying that Ihad to re-input the same destination and dates into asearch bar to find ah

STA Participant 1 - 5 1 6 3 5 1 4 4 4 3

STA Participant 2 Pleasant 5 3 6 4 5 4 4 2 3 4 More filters when searching/booking a tour There was a lot to look at so I would scan and then fogret what I was doing and then backtrack because everything looked so cool

STA Participant 4 Pleasant 6 3 6 4 4 2 4 3 4 1 Speed and offer suggestions not linking the three services together

STA Participant 3 Neutral 5 6 5 2 2 4 5 5 4 2 The flow of the site was choppy Navigation, flow, finding relevant information

STATravel


Participant 3 Neutral 5 6 5 2 2 4 5 5 4 2 The flow of the site was choppy Navigation, flow, finding relevant information

Participant 4 Pleasant 6 3 6 4 4 2 4 3 4 1 Speed and offer suggestions not linking the three services together

Participant 2 Pleasant 5 3 6 4 5 4 4 2 3 4 More filters when searching/booking a tour There was a lot to look at so I would scan and then fogret what I was doing and then backtrack because everything looked so cool

Participant 1 - 5 1 6 3 5 1 4 4 4 3

StudentUniverse


Participant 4 Neutral 5 2 1 1 2 1 3 5 3 1 I didnt like that Ihad to exit the flight ticket purchasing area to find ahotel -pretty annoying that Ihad to re-input the same destination and dates into asearch bar to find ah

Participant 3 Pleasant 6 7 7 4 4 5 4 5 5 4 I dislike how the lists for countries/cities wasnt in al[habeticall order Same as above

Participant 2 Pleasant 7 3 6 4 5 3 4 2 3 2 Pop-up windows; Couldnt find a tour for Phucket, Lack of textual hierarchy; some info hard to read; color scheme

Participant 1 - 6 6 5 4 4 4 4 4 4 4 I just was unfamiliar with it. It would get easier with more use! -

Alsodidnt likehow ittook me toa new sitefor tours,I couldsee howtha twould confuseman y people(it feltlike Iha dclicked on anad by accident).

MArketitotwards studentsifthatis youraudience;improvethetourpage.It didntfunctionwell; allusers toput inmoreinforbeforesearchingsuchas price,starratings,etc.;increasefontsizeonsomefeatures

MArketitotwards studentsifthatis youraudience;improvethetourpage.It didntfunctionwell; allusers toput inmoreinforbeforesearchingsuchas price,starratings,etc.;increasefontsizeonsomefeatures

Alsodidnt likehow ittook me toa new sitefor tours,I couldsee howtha twould confuseman y people(it feltlike Iha dclicked on anad by accident).

Documents

TryMyUI Usability Report