1

IBJ.com

Redesigned and Rediscovered

Usability Evaluation conducted by Erin SheetsDecember 2014

3

Key Recommendations

2

1234

IBJ.com provides invaluable resources to the community of Central Indiana, however their website has several areas that could be improved to increase usability and decrease user frustration.

I performed an expert heuristic evaluation and a user-based evaluation of the IBJ website to determine areas of the site that need improvement.

I performed the heuristic inspection using both Nielsen and MILE+ heuristics as guidelines. The usability test was performed with six different participants to evaluate how casual IBJ users maneuvered through the site to perform various tasks.

Through these two evaluations, I have formulated the following recommendations for the IBJ.com:

• UpdatetheHomepagetoonethatismoreModernandMinimalist

• RestructuretheMenus

• ProvideLandingPagesandDescriptions

• FixtheSearchFeature

The following pages are a detailed account of how this study was performed and the results I found. I also include a detailed plan for following through with the four outlined recommendations.

Update the Homepage to one that is more Modern and Minimalist

Restructure the Menus

Provide Landing Pages and Descriptions

Fix the Search Feature

Key Recommendations Executive Summary

4 5

A Heuristics Evaluation is another inspection method used to identify usability problems with a website. It is a fairly informal method in which a usability expert examines a website a judges it based on standard usability principles calledheuristics.Forexample,accordingtousability“guru”JakobNielsen,asystemorwebsite should:

Visibility of system status Always keep users informed about what is going on, through appropriate feedback within reasonable time.

Match between system and the real world Speaktheuser’slanguage, with words, phrases and concepts familiar to the user, rather than system-oriented terms.

User control and freedom Usersoftenchoosesystemfunctionsbymistakeandwillneedaclearlymarked“emergencyexit”toleavetheunwantedstatewithout having to go through an extended dialogue.

Consistency and standardsUsersshouldnothavetowonderwhetherdifferent words, situations, or actions mean the same thing.

Error prevention Even better than good error messages is a careful design, which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Recognition rather than recallMinimizetheuser’smemoryloadby making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another.

Flexibility and efficiency of use Accelerators—unseen by the noviceuser—mayoftenspeedupthe interaction for theexpertusersuchthat thesystem can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

Aesthetic and minimalist designDialoguesshouldnotcontaininformation that is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

Help users recognize, diagnose, & recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

Help and documentation Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Anysuchinformationshouldbeeasytosearch,focusedontheuser’stask,listconcretesteps to be carried out, and not be too large.

Usability Testing UsabilityTestingelicitsusabilityissuesasobservedin actual user experiences. It aims at maximizing saliency of user experience issues. Not only does this additional test confirm and further deepen problems uncovered during my heuristics inspections, but it also shed light on new issues.

UsersThere are awide-rangeofuserswho frequently visit the IBJwebsite. Forthe scope of this project, I will be focusing only of business owners who utilize the IBJ to advertise and sponsor events in an effort to promote their business to the Indianapolis public.However, for the usability testing, Iwill be testing a random sample of IUPUIstudents and staff.

Heuristics EvaluationHeuristics Evaluation

6 7

12 3

5 4

6

TheCEOofalargefirmdecidestosponsoramajorIBJevent.Sheneedstofindthephone number to call in order to get more information on sponsoring an IBJ event.

Asmallbusinessownerislookingformoreadvertisingoutletsforhisbusiness.Hedecides to submit a request for the IBJ 2015 Multimedia Guide.

Theownerofinsurancecompanyjusthiredtwonewvicepresidents.Shedecidestouse the IBJ to reveal this news.

A retail outlet is experiencing a boon in business. The manager decides to hire three additionalsalesassistants.HewantstoplaceaclassifiedadwiththeIBJ.

Aftersubmittingthead,themanagerhasn’theardbackfromtheIBJ.HedecidestospeakwiththeDirectorofAdvertising.Helooksuphernameandemailaddress.

A store owner hears the IBJ provides a comprehensive list of local resources & information. ShelooksforthelistoflocalAdvertisingAgenciesanddownloadsitasaPDF.

Scenarios Based on the user archetype of a business owner looking to sponsor and advertise through the IBJ, I developed six different scenarios to test.

Rating MethodIusedNielsen’sseverityratingsforusabilityproblems,which take into account factors of frequency, impact, and persistence. The scale is as follows:

0=Idon’tagreethatthisisausabilityproblematall

1 = Cosmetic problem only: need not be fixed unless extra time is available on project

2 = Minor usability problem: fixing this should be given low priority

3 = Major usability problem: important to fix, so should be given high priority

4=Usabilitycatastrophe:imperativetofixthisbeforeproductcanbereleased(inmycase, this rating means that users are unable to complete their tasks)

The following sections describe the results of my inspection and are organized into fmy key categories: navigation, site architecture, design, and content.

Scenarios Rating Method

8 9

Requirements for Improvement

SourceHeuristics

Severity

Example

Problem Description

Homepageisclutteredandrequiressignificantscrollingandclicking.Sure,alothappensinIndianapolis,it’sarelativelylargecity,butuponloadingthehomepageoftheIBJ,it’ssimplyoverwhelming.Scrollingmarquees,currentstocks,advertisements, hourly news updates, blogs, featured images, events,subscriptionoffers,listsandpolls—it’salljusttoomuch!

3: Major usability problem: important to fix, so should be given high priority

Redesigninghomepageisanecessarychange.Notonlywouldthismake visiting the IBJ a more aesthetically pleasing experience, but also it would increase the ease of finding information. There are many different ways to overhaul a website, and for the IBJ, perhaps thinking outside the box or more creatively could be extremely advantageous,especiallyforanindustrythat’sonthedeclineandneeds its digital media to really shine.

One idea, the webpage could resemble an actual newspaper and upon hovering the mouse over the cursor, that part of the paper wouldbeemphasizedwiththeuseofaCSShovereffect:example:http://tympanus.net/Development/HoverEffectIdeas/.Thiswouldallow for progressive disclosure, which helps maintain the focus ofauser’sattentionbyreducingclutter.Another ideawouldbeto utilize parallax scrolling and scroll down through a realistic-looking paper. Here is an example of the power of parallaxscrolling: http://www.agencelatelier.com/.

Aesthetics and Minimalist design

Problem #1Cluttered Homepage

Problem #1Cluttered Homepage

10 11

Requirements for Improvement

SourceHeuristics

Severity

Furthermore, not all of themainmenus (Home,News, Regional,ThisWeek’s Paper, etc.) havelandingpages.Somedoandsomedon’twhichisconfusing.Thisconfusionisthenexacerbatedbythe fact that when there is a landing page, the landing page gives no clue as to what example the page is,andwhatotherpagesfallunderit.Forexample,whentheuserclicks“people”theuseristakentowhatappearstobeaPeoplelandingpage,butitisn’tclear.Perhapsthispageissimplythefirstpageinthemenu,itmaybe“PeopleOnline”thereisnoblurbexplainingwheretheuseris,andnolinkstotaketheusertotheotherpagesthatfallunderthePeoplemenu.

Navigation is challenging due to confusing menu structure. ThemenutitlesarenotobvioustonoviceusersofIBJ.com.Forexample,PeopleandListswillnotmakesensetonewuserswho are unfamiliar with the subsidiary activities conducted by the IBJ. Listing additional trigger words as part of page introductions will help users to understand whether they are in thecorrectplacetofindtheinformationthey’reseeking.

3: Major usability problem: important to fix, so should be given high priority

The menu names should be kept concise; they should not be edited. Instead, adding a short introduction to the menu on the landing page so that the user can better understand what subjects can be found in that menu would be helpful. This will allow users to more easily reach their goal and to feel more confident in doing so.

This week’s paper either does not need to have a long list ofsubmenus, or the page could be broken up so that the user is taken to the section of the newspaper that they are expecting. The additional step of choosing the newspaper twice is inefficient.

Overall,beconsistentwiththesetupofpages.Forexample,allmainpages should have a landing page that also links to all the subpages within that menu.

RecognitionratherthanrecallFlexibilityandeaseofuse

Problem #2Menu Structure Causes Confusion

Menu Confusion Problem #2

Example

Problem Description

Thisscreenshotshowsthedifferentmenus.Whilesome,suchasNewsandEvents,mayseemobvious,not all are. The menus should lead to sub-pages that are obvious to the user, but this is not currently thecase.Forexample,iftheuserwastoclickPeople,theyaretakentoarticlesonPeopleintheNews,butthisfailstoexplainallthefeaturesofthePeoplemenu,whicharenumerous.

Othermenusaddconfusionduetoinconsistencies.Forexample,regardlessofwhichsub-menuunderThisWeek’sPapertheuserschooses,hewillbeledtothissamepage,wherehemustthenchoose the sub-menu again from the list of 8 blue buttons that run horizontally across the page. Not only is the confusing, as the user may believe he is in the Arts & Entertainment section, whenreallyhe’sonthefrontpage,butit’salsoredundant,whyoffertheoptionofsub-menusifnone of them actually lead to that page.

12 13

Requirements for Improvement

SourceHeuristics

Severity 3: Major usability problem: important to fix, so should be given high priority

All landing pages need to be checked for consistency. There should be a standard as to where the user is taken when he clicks the main menu or the first menu option.

Problem #3Landing Pages Confusing

Problem #3Landing Pages

Example

Problem Description

If the user clicks the main menu, he may be taken to a landing page that describes the sub-menus, or he may be taken directly to the first sub-menu. It is unclear when each of these two scenarios will occur and it is fairly disorienting.

In this first example, clicking on the word Classifieds in the main menu can accesses a landing page. However,thislandingpagedoesnotprovide the user with the various subcategories of classifieds, though it appears that way. The first link opens up outlook to email the contactaboutthelandadvertised,thesecondlinkdirectstheusertotheDomontStudioGallery,and the career opportunities link takes the user to kriegdevault.com. It may not be obvious at first, as thereareonlythreeclassifiedads,thatthesearetheClassifieds,notanoptiontopostanad.Forthat,the user must click the small link on the upper right-hand side of the page. Thus, in this example, clicking the menu name takes the user to the first subpage

In this second example, however, there is not the option to click the menu name, subscriptions. The user must choose one of the six options.

In this third example, the menu name, Lists, is hyper-linked, but clicking it takes the user to a page not listed as a sub-menu. It takes the user to a page titled Lists. This inconsistent pattern can cause confusion and make it difficult for the user to create a mental map of the website.

Consistency and standardsError preventionRecognitionratherthanrecall

14 15

Requirements for Improvement

SourceHeuristics

Severity

The search feature does not always function, does not provide feedback, and is not located in a consistent location. Furthermore,whilethesearchbarispresentonmostpages,itisnot on all pages.

3: Major usability problem: important to fix, so should be given high priority

At a minimum, if there are not search results, the user should be toldthatstatuswithwords,“SearchResults:0.”Also, thesearchbar should be located in the same place on every page.

Visibility of system statusHelpusersrecoverfromerrorsConsistencyandStandards

Problem #4Search Feature is Unhelpful

Problem #4Search Feature

Example

Problem Description

ThereisanoptionintheListfeaturetosearchthroughthelists.However,whenIusedthefeaturetosearchforlawyers,notonlywasnothingfound,butthepagewasblank.FromtheURLIcouldtellthatasearchforthekeywordLawyerswasconducted,butthereisnouserfeedback to show that the search happened. It appears that this function is not working at all, asnotermIenteredgeneratedanyfeedback.However,itisdifficulttotellforsureasthereisno feedback as to whether or not the search was completed.

Furthermore,astheimagesbelowshow,theredoesn’tappeartobeaclearreasonwhenthesearchbarispresentandwhenitdisappears.It’ssimplyunclearastowhyitisn’tomnipresent.

Furthermore,as the imagesaboveshow, theredoesn’tappear tobeaclearreasonwhenthesearchbarispresentandwhenitdisappears.It’ssimplyunclearastowhyitisn’tomnipresent.

16 17

UsersSixuserswererecruitedforthisstudy.Typicallyfiveisthe“magicnumber”whenitcomes to usability studies (see below for an explanation). The following information about each participant was gathered from the pre-survey:

Usability TestingWhiletheexpertreviewrevealedseveralusabilityissues within the IBJ site, I also conducted a usability study to uncover additional usability issues from a user perspective. This also allows us to gather first-hand accounts of what a user faces when he is navigating a website.

Usability Test Plan A total of six participants will be chosen to participate in the user study. They will be compensated for their participation and time with a $5 BarnesandNoblesgiftcard.Theuserswillallbetestedseparatelyandwillbegivena set of six tasks to complete. The tasks will be examined to determine three different performance metrics:

1. TaskSuccess(thisiseitheryesorno,whiletherearemultiplepathstoasuccess, the definition of success is clearly defined in the task description)

2. TimeonTask(alltaskswillbetimed)

3. Errors:Errorswillbecounted.Ifafterthreeerrors,theuserisnotmakingprogress towards his goal, the task is considered a failure (see below for a detailed scoring rubric).

Scoring Rubric1 = No problem. The participant successfully completed the task without any difficulty or inefficiency.

2 = Minor problem. The participant successfully completed the task but took aslightdetour.Hemadeoneortwosmallmistakesbutquicklyrecoveredandwas successful.

3 = Major problem. The participant successfully completed the task but had major problems. She struggled and took a major detour in her eventualsuccessful completion of the task.

4=Failure/gaveup.Theparticipantprovided thewronganswerorgaveupbefore completing the task, or the moderator moved on to the next task before successful completion.

Pre-Survey Before beginning the actual study, the participants will complete an online pre-study questionnaire, which can be found here: https://www.surveymonkey.com/s/PFRS98B.

User # 1 2 3 4 5 6Gender Female Female Female Female Female Female

Age 25-34 25-34 18-24 18-24 18-24 18-24

Level of Education Bachelors Masters Masters Bachelors Bachelors Bachelors

Self-Perceived Computer Abilities High Average High High Average Average

Hours of Weekly Internet Use 80+ 20-40 20-40 80+ 40-80 20-40

Likelihood Getting News Online Very likely Likely Likely Likely Very likely Likely

Why 6? According to Jakob Nielsen, when it comes to users in a usability study, there is a diminishingrateofreturnafteraroundfiveorsixusers.Asyougathermoreusers,youaren’tnecessarily going to find more errors or find new mistakes.

WhenaskediftheywerefamiliarwiththeIBJ,foursaidtheywere,but“onlyalittlebitfamiliar”and that they had “visited it, but I’mnot a regular user.”Users said they had the followingimpressions of the website:

• “Thereistoomuchinformationoverload”• “Simplistic,somewhatbland”

UsersUsability Testing

18 19

Usability Test Plan A total of six participants will be chosen to participate in the user study. They will be come users were told each scenario based on a pre-written script. They were timed as they complete each task. The timer is stopped while each task is explained and then stopped again only when the user says he has completed the task.

Aftereachtask,theuserwasaskedtocommentonthattask.Specificallyifthetaskwaseasy,medium,difficult,or“impossible”andwhytheusergivesitthisrating.Ifthe task was not easy, the participant was ask how the site can be changed to make the task easier.

Post-SurveyAfterthetasksareallcompleted,theparticipantwillbeaskedtocomplete a brief post-survey, which can be found here: https://www.surveymonkey.com/s/PBHGLQD.

Usability Testing Results Overall, results from the usability test were quite mixed, and it takes more than just a look at the quantitative data to understand the trends and how to use them to improve IBJ.com. Below is a summary of the comments and observations gathered from the study. Each set of task comments is followedbyan“ImmediateRecommendation,” this isaquickfix tosolveeach problem, however, as I will detail later, the website could be vastly improved with a major restructuring that would make many of these problems irrelevant.

User Participant’s Comments Testing Evaluator’s Comments

1 “Thiswaseasy” No errors made by participant

2 “Thistaskwaseasy,butIhadadifficulttimeseeingthenumber,it’ssosmall.”

UserwentPeople>Events>SponsoranEvent>butthenshescrolledthroughtheentire page before seeing the number at the top of the page

3

“Ifoundthesection,butthephonenumberwasn’tclear,soI’dgiveitamedium.Iwasexpecting the phone number to be larger or agraphic.”

Usertookidenticalstepstouser2

4 “Thatwaseasy.”Didasearchfor“sponsor”thenopenedthepage and read text until she saw the phone number

5 “Medium.Thenumbershouldbebigandinthecontactlisttoobecauseit’sacontact.”

Firstwenttocontactlist,nosuccess.Sighedwithfrustration.Foundeventsponsorshippage,scrolledtobottom,muttering,“itshouldbeonthispage.”Scrolledupandfound it.

6 “Thatwaseasy,it’ssetuplikemostotherwebsites.”

Userwentdirectlytothefooterandclickedeventsponsorship.Shescrolleddownthepage before scrolling back up and finding the phone number.

Scenario #1 TheCEOofalargefirmdecidestosponsoramajorIBJevent.Sheneeds to find the phone number to call in order to get more information on sponsoring an IBJ event.

Recommendation Make the phone number larger, or put it in a colored box so that it is easier to find.

Observation Notes

Scenario #1Usability Test Results

20 21

User Participant’s Comments Testing Evaluator’s Comments

1“Thiswasadifficulttask,ifIwasabletokeep trying, I think I would have eventually foundit.”

UserfirstwenttoIBJExtras,clickedMul-timedia,butthepagewasnotfound.Shesearched the header, unaware of the footer. Usedthesearchbar,unsuccessfully.Afterthree errors, I moved on.

2

“Iwasabletofindthemultimediaguidelink,butthelinkwasdead,soI’djustcallthem.Forthatreason[thelinkwasdead]I’dratethistaskmedium.”

Userfirstsearchedforthemultimediaguide,nosuccess.SoshewenttoIBJExtras>Multimedia,thelinkwasdead.Shethenscrolled through contacts. At three errors, shesaidshe’djustcallsomeonefromthecontact list for help.

3

“Iratethistaskasdifficult,becauseIthoughtIdidtherightthing,butdidn’tgetthepage,Icouldn’tthinkofanotherwaytofindit.”

Userfirstlookedunderclassifieds,thenshesearched for multimedia guide, no results, thensheclickedIBJextras>Multimediaguide.Sheaskedmetorepeatthequestion,she then gave up.

4

“IwouldratethisasmediumbecauseIfoundit,butonlybyaccident.Iwouldn’thave known to scroll all the way to the bot-tom to see advertising, I was just exploring andfoundit.”

Userfirstdidasearchfor“2015Multimediaguide”,thensearched“MultimediaGuide”,scrolledtobottomofthepageandaftersome study, clicked IBJ advertising and was surprised and excited that she had complet-ed the task.

5

“IgotconfusedbetweensubmitadvertisingandIBJAdvertising.Whyaretheyseparate?It made it hard to find. They really should be combined.”

TheuserwenttoSubmitAdvertisingatthebottomofthepage,butcouldn’tfindtheguide, so she went to IBJ Advertising and found it.

6 “Easy.” UserwenttothefooteragainandclickedSubmitPeoplebutton.

Scenario #2 A small business owner is looking for more advertising outlets for his business.HedecidestosubmitarequestfortheIBJ2015MultimediaGuide.

Recommendation RepairtheMultimedialandingpageunderIBJextrassothat the user can find the Multimedia guide in this intuitive spot.

Observation Notes

User Participant’s Comments Testing Evaluator’s Comments

1“Thistaskwasdifficultforme.Ifyouaren’tfamiliarwiththewebsite,youwon’tknowwheretogo.”

No errors made by participant

2 “Thistaskveryeasy.”

UserwentPeople>Events>SponsoranEvent>butthenshescrolledthroughtheentire page before seeing the number at the top of the page

3

“Difficult.Ithinkitisconfusingthatthemenuisatthebottomofthepage.‘SubmitPeople!’Idon’tknowwhatthatevenmeans,there’snocontext!

Usedthefootertofindinfo.,whichmakessense, as it seems to have more information about submitting information to the IBJ, whereas the top menu is more concentrated ongettinginformationfromtheIBJ.Still,user was confused

4

“ThistaskwasdifficultbecauseIwasn’tclearaboutthetask.Ididn’tknowyoucouldsub-mitinformationthatyou’vehiredpeople.Ithought the IBJ was just for finding people tohire.”

Userspentquiteabitoftimejustlookingat the page, then went to the menu at the bottom and clicked submit people.

5“Thiswaseasy,atfirstIclickedawards,butIrealizedthatwasn’tit,sothenItriedsubmitpeople.”

Userfirstwenttothebottomofthepageand clicked nominations in the footer. Then she tried again, this time clicking submit people.

6 “Thiswasmedium.Iwasn’tsureifSubmitPeopleisforhiringorreportinghires.”

Again, this user went directly to the footer menu.Shestudiedtheoptionsthenchose“SubmitPeople.”Shemadenoerrors.

Scenario #3 The owner of insurance company just hired two new vice presidents. ShedecidestousetheIBJtorevealthisnews.

Recommendation CreateaPeoplelandingpagethatdescribeswhattheusercandowithinthePeoplemenu.

Observation Notes

Scenario #3Scenario #2

22 23

User Participant’s Comments Testing Evaluator’s Comments

1 “Thiswaseasy” Usermadenomistakes.Shewentdirectlyto classifieds and selected place and ad.

2 “Easy.” Usermadenomistakes.

3“Thistaskwaseasybecausethewordingwasreallyclear.ItsaidClassifiedsandPlaceanAd.Iknewexactlywhatthatmeant.”

Usercompletedtaskveryquicklywithnomistakes.

4 “Medium.Ittookmeawhiletoconnectthetermclassifiedwithhiring.”

Userfirsttriedpeople,thentriedclassifieds,then submit an ad.

5

“Thiswasn’tdifficult,butitwasn’teasyeither.IthinkitwasbecauseIdidn’tknowwhat classifieds meant. If I saw the main classified page menu and saw the words Ca-reer Opportunities I would have found it.

Userspentalotoftimestudyingthetopmenu, unsure where to click first.

6 “ThiswasdifficultformebecauseIdon’tknowwhat[theword]‘classifieds’meant.”

Userstudiedthewebsiteforawhile,lookedat each menu, chose a couple incorrect menus,thenfinallyfoundthelinktoSub-mit an ad.

Scenario #4 A retail outlet is experiencing a boon in business. The manager decides to hire threeadditionalsalesassistants.HewantstoplaceaclassifiedadwiththeIBJ.

Recommendation None, leave as is..

Observation Notes

User Participant’s Comments Testing Evaluator’s Comments

1

“Thetaskwaseasy,butittookawhilebe-cause they just assumed that everyone uses outlook, they should put an actual email there.”

TheuserwenttoIBJExtras>ContactUs.Shefoundthenameeasilybutgotstuckhaving to set up an outlook account before seeing the email address.

2“Ialreadynewhernamefromtheprevioustask, I saw it, it was easy for me to get her emailaddress.”

UsersusesOutlookforheremail,andwasable to click on the users name and see her email address immediately in a blank email.

3

“Findingthenamewassupereasy,buttheemail was difficult, it should have a pop-up HTMLtagorsomething,becauseIdidn’twant to set up an email account, I just use myGmailandwasn’tabletoseetheemailatall.

Userwasabletofindthename,butdidn’tfindtheemailaddressbecauseshedidn’tfeel comfortable going through the email wizard the link prompted because she doesn’thaveoutlookonhercomputer.

4 “Thatwaseasy.”

Userfirstsearched“Advertisingdirector”there were no results, probably due to mis-spelling“advertising”.TheuserthenwenttoContacts>Salescontacts.Sheclickedthename and it took her to outlook, which she doesn’tuse.Sosherightclickedthenameand chose to open the link in a new tab.

5 “Ididn’tlikehowitopenOutlook,whoevenusesOutlook?Otherwisethetaskwaseasy.” No issues

6 “Thatwasimpossible.”

UserwenttoAdvertising>Contactsandsaw the person she was looking for, but struggled to get her email address in the samewayotherusersstruggled.Shegaveup.

Scenario #5 Aftersubmittingthead,themanagerhasn’theardbackfromtheIBJ.HedecidestospeakwiththeDirectorofAdvertising.Helooksuphernameandemailaddress.

Recommendation Include email addresses in the text as well as embedded in thename,sothatuserswhodon’tutilizeMicrosoftOutlookcaneasilycopyandpastetheemail address into their preferred email provider.

Observation Notes

Scenario #5Scenario #4

24 25

User Participant’s Comments Testing Evaluator’s Comments

1 “Thiswaseasy,themenunamewasself-ex-planatory,soIfounditrightthere.”

Userwentdirectlytolists>individuallists>skipad>download.

2“IwouldratethistaskasmediumbecauseIhad to search through the book to find the list,andthattooksometime.”

The users opened the book of lists instead of individual lists, she then realized that the list she wantedwasonpage112.Shescrolledindividu-ally through each page before deciding to use the menu to go to that page. This caused it to take a long time.

3

“Thistaskwasn’thard,butitcouldhavebeeneasier.Forexample,individuallistsdoesn’tmakealotofsense,thewordingistoo ambiguous. Maybe instead of individual listsitcouldbeIndexofLists.”

Userwasabletocompletethetaskbutstruggledto choose between book of lists and individual lists.

4“IgivethataratingofmediumbecauseIhad a hard time differentiating between the booksoflistsandtheindividuallists.”

Userwenttothefooter>lists>bookoflists.Iprompted her to get the individual list instead ratherthanwatchingherstrugglethewaythatP2had.

5

“IjustneverthoughtlistsiswhereIwouldlook.It’sobviousinhindsight,butjustnotwhereI’doriginallylook.Maybeitshouldbetitled‘Resources’insteadof‘Lists.’‘Lists’isaveryambiguousterm.‘Listofwhat!?”Forsomeone not familiar with the IBJ and what theydo,theywon’tknowtolookforlists.Then I thought it was hard to differentiate between individual lists versus book of lists.

Userscrolledpast“Lists”severaltimes.Shethensearched“accountingfirms”shedidn’tseewhatshe was looking for in the results, so she searched “accountingfirmsIndianapolis.”Stillnothing.Shefinally went back and clicked lists and found the PDF.

6 “Thistaskwasdifficult.Ihadnoideawheretolook.”

Userlookedaround,lookedatthefootermenu,then gave up.

Scenario #6 A store owner hears the IBJ provides a comprehensive list of local resources & information.ShelooksforthelistoflocalAdvertisingAgenciesanddownloadsitasaPDF.

Recommendation Add a landing page that explains what the Lists are, or create an additionalsub-menutitled“AboutLists,”whichwouldredirecttheusertotheListlandingpage.Thiswould help user understand what the lists are and how to look up individual lists versus browsing through the entire book.

Observation Notes

Scenario #6

Quantitative Results I can further understand where users struggled by looking at the time it took to complete each task. As you can see from the chart below (Table3),thereisalotofvariationintheamountoftimeittookdifferentuserstocomplete a task. The tasks that do not appear on the graph are those for which the user gave up and was given the time of 0.

Forexample,onlytwousersoutofsix,or33.3%completedtasknumbertwo.Whatelseis interestingisthatP3,aself-described“highly-skilleduser”wasonlyabletocomplete50%ofthetask,yetofthosetasks,mostofthemshecompletedmuchfasterthan all other users. In response to why she was unable to complete the other three tasks,hercommonanswerwas,“thewordingonthissitejustisn’tclear,itmakesithardtofindthings.”Iwillgomoreintodepthonthesubjectof“wording”laterinthereport.

Most tasks took less than two minutes to complete, however, it is troublesome is the numberoftasksparticipantsfeltunabletocomplete.Severaluserssaidtheywouldsimply call the IBJ to ask their question. They felt this was an obvious solution to the problem, but the purpose of the IBJ website, presumably, is to provide users with information, not leave them hunting for details and forcing them to then call the IBJ.

Table 3: Time on Task

Quantitative Results

26 27

In addition to time on task, I also measured errors for each task based on the scale mentioned earlier. A score of one or two represents either no errors, or very few, minor errors. A score of three represents some very serious errors and detours before completing the task. A score of four indicates that the user was unable to complete thetask.Thechartbelow(Table4)isagraphicalrepresentationofthescoresearnedby each user for each task:

It only takes a glance to see that several tasks were not accomplished. However,as included in the comments table above, there are several reasons for the lack of completion. One major issue with task five is that users were unable to find Lisa Bradley’semailaddress.TheprimarycauseofthisissimplythatmostusersdonotuseMicrosoftOutlookandthelinkissetupwithOutlooktocreateanemailshortcut.Simplykeepingthisshortcutwhilemakingtheemailaddressvisibleonthepageas well would drastically improve the score on this task.

Another cause of incomplete tasks was a lack of basic newspaper knowledge. For example two users were unsure of the term “Classifieds,” which is a basicnewspaper term. Both these users were under the age of twenty-four and perhaps theword“Craigslist”hasreplaced“Classifieds”undertheirterminology.Whilethismisunderstanding occurred twice, I would be very hesitant to recommend that the term be changed, as it is a classic newspaper function

Table 4: Score on Task

Wordingmayhavebeenafactorinlongtasktimesandincompletetasks.Iprovidedbackground information about the task to put the user in the shoes of a businessperson, but it could be possible that this caused further confusion as users tried to take in all this data and occasionally found themselves unsure of the task they were attempting tocomplete!PerhapsifIstatedtasksixas“findindividuallists”insteadofprovidinga story about a crooked accountant the user would have struggled less and found the pagefaster.However,Ididn’twanttogivetheusertoomuchinformation,Iwantedto simulate an actual situation and see how the user would handle it.

Wordingandsitestructurewasamajorissueforallusers.TheIBJhasthechallengeof presenting a plethora of information to a vast audience. Because the IBJ provides information to virtually anyone in central Indiana, no two users are going to be similar.Awidevarietyofaudienceageandeducationlevelisinevitable.Unfortunately,for this study, all participants happened to be female and have at least some college completed. All users rated themselves average to high in terms of computer abilities. Therefore, given this information, one would assume that these basic tasks should be easy for them, but they were not. All users stated that the menu labels would make sense to someone familiar with the IBJ and its offerings, but for those new to IBJ.com, theyfoundmenutitlessuchas“People,”“Lists,”and“A&E”confusing.Fartoomuchtime was spent studying the menu and trying to decipher where these ambiguously namedmenuswouldtakethem.Oftentheusersguessedincorrectly,whichexplainswhy no user was able to complete all tasks error-free.

In my expert review, I uncovered that the menu set up does not provide for ease of navigationthroughthesite(problem11).Whilemyusabilitytestingparticipantsdidnot point out this error specifically, I instead witnessed them struggled at the hands of this problem. Let us clarify this further. Most users eventually realized that there are two menus. There is a header menu at the top of the page and there is also a footer menuatthebottomofthepage.Forthemostpart,thesemenusappeartobedividedinto two categories: the top menu is where users can get information from the IBJ and the footer menu is where users can submit information to the IBJ. But upon further study,itbecomesclearthatthisisnotthecase.Perhapsthen,thetopmenuislinksmost commonly used by users and the bottom is for everything else. But if this is the case,myusersdon’tseemtoapproachthewebsitethisway.Someusersstucksolelyto the top menu (at the expense of not being able to complete tasks) while other users always scrolled down to the bottom menu (again this was at the expense of completing tasks efficiently). A clear distinction between the top and bottom menus appearstobelackingfromtheuser’sperspective.

Score on Task Score on Task

28 29

Furthermore, users experience frustration because some of the menu titles werelinked to a landing page, but others were not linked, as no landing page existed. WhenIletparticipantsexplorefreely,theywerefrustratedandconfusedastowhythere were some landing pages for certain menus, but others seemed not existent and merely took them to the first item in the menu. This inconsistent pattern can cause confusion and make it difficult for the user to create a mental map of the website.

Post-Survey Results After completing the usability testing, eachparticipant was then asked to complete a brief post-survey, which can be found here: https://www.surveymonkey.com/s/PBHGLQD.

The users were each asked to rate whether they agree or disagree with six general questions about the website. They were then given an opportunity to provide comments. The results of the survey are as follows:

Q1: The wording of the text was clear.

Strongly Disagree 0%

Disagree 50%(3)

Neutral 0%

Agree 50%(3)

Strongly Agree 0%

Q2: The content (including text, pictures, audio, and video, etc.)was easy to understand.

Strongly Disagree 0%

Disagree 0%

Neutral 50%(3)

Agree 33.3%(2)

Strongly Agree 16.7%(1)

Q3: I could quickly get to know the structure of the website by skimming the homepage.

Strongly Disagree 0%

Disagree 33.3%(2)

Neutral 0%

Agree 33.3%(2)

Strongly Agree 33.3%(2)

Q4: The website was arranged in a way that make it easy tolocate the information I needed.

Strongly Disagree 16.7%(1)

Disagree 33.3%(2)

Neutral 16.7%(1)

Agree 16.7%(1)

Strongly Agree 0%

Q5: Under each section of the website, the web pages were well organized.

Strongly Disagree 16.7%(1)

Disagree 16.7%(1)

Neutral 16.7%(1)

Agree 33.3%(2)

Strongly Agree 16.7%(1)

Post-Survey Results Post-Survey Results

30 31

Q6: Using the website was effortless.

Strongly Disagree 16.7%(1)

Disagree 33.3%(2)

Neutral 16.7%(1)

Agree 16.7%(1)

Strongly Agree 0%

Q7: Overall, how would you rate the IBJ website?

Very Bad 0%

Bad 16.7%(1)

Somewhat Bad 16.7%(1)

Somewhat Good 16.7%(1)

Good 33.34 (2)

Very Good 16.7%(1)

Q8: Is there anything you would like to change about the IBJ’s website?

It’sjustsoclutteredandnoteasytouseasitshouldbe.

The naming convention was not consistent. It would be great if the site would use titles that were clear and recognizable.

Thewordingofafewofthesections.Forexample,A&Eshouldbeundernewsandit should say Arts and Entertainment. List should be a less ambiguous term, like

resources. The emails of contacts should be underneath their names.

N/A

There was a dead link on the multimedia page.

The menu

Conclusion

User Feedback What’sinterestingabouttheresultsfromthepost-surveyisthatthestudyparticipantsarequitedivided.Inquestion1,“thewordingofthetextwasclear,”halfoftheparticipantsgreed,whiletheotherhalfdisagreed.Theyweredivided. The information gathered from users was full spectrum. As one can see from table5,answersareallovertheplaceandthereisn’tmuchconsistency.

Reviewingquestions3and4isalsointeresting,asthemajorityofparticipantsstatedthat the site structure was easy to get to know simply by skimming the homepage, yet themajorityofparticipantsdisagreeswiththestatement,“thewebsitewasarrangedinawaythatmadeiteasyformetolocatetheinformationIneeded.”Therefore,Icanconclude that the participants believed the site was structured clearly, but that they couldn’teasilylocatetheinformationwithinthatstructure.

Participantsweresplitagainwhenasked if theybelieved thepageswithin thesitewerewellorganized.Perhapstheywouldbelessdividediftheyweregiventimetoexplore the pages with this question in mind.

Importantly,question6revealsthatoutofsixusers,onlyonethat“usingthewebsitewaseffortless.”Themaingoalofthisusabilityevaluationistoimprovethesitesothatif I were to conduct this study again, all users would agree or strongly agree with this statement.

ShortcomingsWhilethisstudydidrevealmanyimportantusabilityissueswiththeIBJ,Ididexperiencesomeshortcomingswiththestudy.Someusersstatedthat the scenarios were difficult to understand. One user, for example explained of task4,“Thiswasn’tdifficult,butitwasn’teasyeither.IthinkitwasbecauseIdidn’tknow what classifieds meant. If I saw the main classified page menu and saw the words CareerOpportunitiesIwouldhavefoundit.”Whilethismayseemfairlyridiculous,itis important to keep in mind that this is a real user who is college educated.

In the future, I will make the scenarios more clear and concise so that users will not be confused or misled by them, and instead allow the user to completely focus on the website in front of them rather than just on understanding the task.

Post-Survey Results Conclusion

32 33

According to Jakob Nielsen, the guru of usability, five is the magic number when it comes to conducting a usability study. I went above that with six users, but perhaps with more variety of participants, I could have gotten more consistent findings.

Recommendations Earlier in the report, I outlined several recommendations both within the expert review as well as within the results of the usabilitytest.However,atahigherlevel,therearesomechangesthatcouldbedonetoimprovethesite.Severalofthesechangesarenotquickfixes,theyaremoredrastic,but I believe they will vastly improve the user experience.

Update the website to make it more modern and minimalist

First impressions are important.Overall, userswere not impressedwith IBJ.com.Redesigning the entire site would not only would make visiting the IBJ a moreaesthetically pleasing experience, but it would increase the ease of finding information as well. There are many different ways to overhaul a website, and for the IBJ, thinking outside theboxcouldbeextremelyadvantageous, especially foran industry that’sfacing a lot of change. As more and more people go digital for all their needs, IBJ needs its digital media to really shine.

One idea: the webpage could resemble an actual newspaper, only each part would be muted, and upon hovering the mouse over the cursor, that part of the paper would be emphasizedwiththeuseofaCSShovereffect,seethisexample:http://tympanus.net/Development/HoverEffectIdeas/. This would allow for progressive disclosure, which helpsmaintainthefocusofauser’sattentionbyreducingclutter.Theuserwouldbeable to see all the sections of the website as though it was newspaper, and hovering over the section would offer a description as to what is contained within that section.

Parallaxscrollingandtheuseofarealistic-lookingpapercouldfurtherenhancethevisualappealofthewebsite.Hereisanexampleofthepowerofparallaxscrolling:http://www.agencelatelier.com/.

Restructure the Menus

It was noted several times that users had a hard time finding information within the site, and I believe much of this is caused by poor menu structure. Creating a clearer distinction between the information to be found in the top menu and the information within the footer menu would provide greater clarity to users.

Provide Landing Pages and Descriptions

Many of themainmenus (Home, News, Regional,ThisWeek’s Paper, etc.) havelanding pages, but others do not. This is confusing, the pages should structured in a way that helps orient the user through every part of the website. Leaving breadcrumbs and using landing pages to clarify what the menu entails and where sub-menu information lies is crucial in preventing users from becoming disoriented.

Fix the Search Feature

One user in my study relied solely on the search feature to navigate through the website. Search features canbe extremelyhelpful, especially forwebsites asdenseastheIBJ.However,Iencounteredseveral issueswiththesearchfeature.Thefirstproblemwasthatthesearchbardidn’thaveaconsistenthomewithinthesite.Movingittothetopofthepage,underthemainmenubarandtotherightoftheQuickLinksrow may be a good option.

There were also many errors associated with the search feature. As mentioned in the expert review, at one time the search feature was not functioning, yet it did not provide feedback that it was not working. The search option also fails to help users findbasicpartsofthewebsite.Forexample,whensearchingforthesitemap,Iusedthe search functioning. The search resulted in 11 results, none of which were the sitemap.

Theadvancedsearchalsodoesn’tappeartobefullyfunctional.Forexample,underNewspaper Edition, the only options are December 13th, 2014, May 4th, 2013,August24th,2009,andJune5th,2009.Theniftheuserchoosestobrowsebytopic,there are literally hundreds of topics. This is not particularly helpful and should be reconsidered.

Overall IBJ offers an important service to the city of Indianapolis, with some changes to its website, the site can be even more helpful to users and better serve the needs of the IBJ as well.

Recommendations Recommendations