CUTGroup 28 Open Data Portal Report by Sonja Marziano Project Coordinator Christopher Gumienny CUTGroup Proctor

Smart Chicago Collaborative

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Introduction For the 28th Civic User Testing Group (CUTGroup) test we tested the redesigned homepage of the City of Chicagorsquos Open Data Portal The Open Data Portal allows users to find resources and various datasets regarding the city of Chicago The City of Chicago Department of Innovation and Technology is working with Socrata to redesign the Open Data Portal focused currently on the homepage to be more user-friendly while representing multiple data and technology initiatives and applications created with open data Target Group

On September 22 2016 we sent an email to 1172 CUTGroup testers who lived in Chicago and notification via text message to our testers who prefer to receive notification via text We wanted to know if they would be available for an in-person test on September 28 When segmenting our testers we were interested in testing with

Testers who use a variety of devices Android iOS and laptops We wanted to have an almost equal representation on all these devices types to better understand the homepagersquos functionality on different devices

Testers who have used the Open Data Portal in the past and those who have never used the Open Data Portal before We wanted to have people compare both data portals while including new first impressions

Testers who have all levels of data experience to see how user-friendly the search functionality is

Test type

In-person test proctored on both mobile and laptop devices We included 6 testers on laptop devices 6 testers on Android mobile devices and 5 testers on iOS mobile devices (total of 17 testers)

Testers were paired with proctors to record their interactions and comments or opinions about the City of Chicago Open Data Portal homepage

Goals of Test The main goal is to understand how testers who have some familiarity with the data portal (even minimum) respond to the changes made to the homepage We want to capture how residents search and what is the best structure for the homepage to make searching easier Lastly we want to see how responsive testers are to the other content that relates to the programs and tech initiatives of the City of Chicago

1

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Responses 17 testers completed the in-person CUTGroup test of the City of Chicagorsquos Open Data Portal homepage at Woodson Regional Library 9525 S Halsted St Chicago IL 60628 on September 28 2016

Background questions

What successes or challenges have you had finding data that you were looking for or using the Open Data Portal Tell us about any of your past experiences 6 out of 17 testers had no previous experience with datasets or using the Open Data Portal 8 testers expressed positive experiences using Chicagorsquos Open Data Portal For example

CommunityResearcher 201601 (11) saidldquoSomewhat successful Used to research a paper a number of years ago [and] was able to find what I needed on the elderly populationrdquo

3 testers shared challenges using Chicagorsquos Open Data Portal

Syliva (3) saidldquoItrsquos not that data is difficult to access but the results are not relevant If Im looking up restaurant data I dont want school data for instance Results are not always relevantrdquo Blessed (6) saidldquoI usually look for crime in the immediate block Theftrobbery person or house [It] wasnrsquot accuraterdquo

We learned from this CUTGroup test that there was some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website Some testers very clearly used the Open Data Portal while a few testers shared experiences finding city services which led us to believe that they were speaking about the official City of Chicago website

What are your expectations of what the City of Chicago Open Data Portal would be useful for Here are some responses from testers about what they expect the City of Chicago Open Data Portal would be useful for (number indicates number of times a response was given as testers sometimes had multiple answers)

6 ndash Search for specific data 3 ndash Analyze data

2

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

3 ndash Find City services 2 ndash Creates transparency 2 ndash Find jobs 1 ndash Find doctors 1 ndash Mapping 1 ndash Develop apps using the data 1 ndash Pay parking tickets 1 ndash Communicate with Aldermen

Gabireads (2) said that the Open Data Portal is useful for ldquoMapping Creating thematic maps Census crime restaurant neighborhood data to make maps and analyze datardquo Batman (14) saidldquo To have datainfo on various municipal systems licenses historical data on various going-onsrdquo M (15) mentionedldquoGreat if everything that is not confidential was available Would enable better social science and community involvement services security etcrdquo

Initial Homepage Review

Review the homepage and tell us about your first impressions While you review try not to click on the next page just yet but feel free to scroll up and down the page When reviewing the redesigned homepage for the first time 13 testers had initial positive impressions 2 had critical impressions and 2 had neutral impressions

Mobile users The majority of mobile testers had a positive reaction to the homepage with 5 testers mentioning that it seemed straight-forward 3 testers had negative responses when viewing the homepage on their mobile device 4 out of 11 testers specifically mentioned the categories whereas only 1 tester mentioned the search bar Other things testers mentioned Divvy (3) ldquoService Trackerrdquo (2) clearreadable font (2) specific datasets (2)

Funky Burrito (10) says ldquoWell laid out straightforward Right amount of pictures versus text content Feels easy to navigate based on criteria of informationrdquo

VP (12) thought that the homepage is ldquoWell organized Preview of what is to come is on the banner I assume that I would be able to swipe but you cannot Also lots of scrolling Visually seems very cramped not alignedrdquo

3

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

GPBlight (4) believe that ldquoFlashing promotions are unnecessary It should focus on the top three items people search for not what they are pushing in relation to the Divvy advertisement

Laptop users 4 out of the 6 testers who viewed the homepage mentioned the categories in their initial review while no one mentioned the search bar Most testers had a positive reaction to the homepage while 1 tester specifically said that there was too much information Other things testers mentioned Divvy (1) ldquoChicago appsrdquo (1) ldquoHow to videordquo (1)

Batman (14) says ldquoLooks clean and modern The old one wasnt easy to navigate and this page has broken up categories to find stuff more easily Info is more visualrdquo TJ (5) thought there is ldquoToo much information and stuff on the homepage lsquoDigital Chicagorsquo section should be biggerrdquo

Lhibbard (7) says ldquoNot exactly what I expected Didnt expect to see a map or information about Divvy Thought it would highlight city services Noticed the Map first Looks more fresh and updatedrdquo

What are three actions that you know you can do just from reviewing this first page Here are responses we have heard from testers about actions they can do just from reviewing the homepage (number indicates number of times a response was given as testers sometimes had multiple answers)

6 ndash Browse data catalog by choosing category 6 ndash TutorialsVideos 4 ndash Learn about community 4 ndash Use Chicago apps 4 ndash Request city services (street sweeping garbage pick-up 311 etc) 2 ndash Find events 2 ndash Navigate crime data 2 ndash Click on menu 2 ndash Type information in search box 2 ndash Transportation 2 ndash Find health and human services 1 ndash Environmental data

4

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

1 ndash Review free maps

Thrones (1) mentionedldquoLook at different events coordinate routes and find out whats going on in your communityrdquo Syliva (3) saidldquoClick on menu can type in info in search box or I can choose categories from belowrdquo Lhibbard (7) saidldquoLook up community health and human services look up information about transportationrdquo

Who do you think this website is designed for Why

14 testers mentioned that this Open Data Portal is designed for the general publicresidents to help find city servicesresources (3) government transparency (1) and access data (1) Testers who said it was for residents said so because it is ldquoeasy to use and not laden with jargonrdquo (Gabireads 2) and ldquoto help people find resources in the City of Chicagordquo (Thrones 1) 3 testers specifically said the Open Data Portal is designed with people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis 1 tester said it was built for business owners while another said it was built for developers

After reviewing the homepage what is the first thing you want to do or click

Clicked on a data catalog category 59 (10) Search to find a specific dataset 18 (3) Clicked on a Digital Chicago topic 6 (1) Clicked on a Chicago App 6 (1) Other Clicked on menu on the upper right 6 (1) Other Clicked on Divvy banner 6 (1)

What did you search Or what was the category you chose Category FOIA (3) Education (3) Events (2) Transportation (1) Health amp Human Services (1) Search terms Food (1) Building safety data (1) Community (1)

Tell us more about the first thing you did and why you did it What were you expecting to happen 5 testers expected to find more general information about what the website does or the type of information listed under a category 4 testers had specific topics to query 1 tester expected city service options

5

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Clicked on a data catalog category

Mobile users (6) FOIA 50 (3) Education 33 (2) Events 17 (1)

Laptop users (6) Human Health Services 32 (2) Transportation 17 (1) Community 17 (1) Education 17 (1) Events 17 (1)

TJ (5) chose the transportation category because he ldquowanted to find CTA maps dealing with train transportationrdquo and it ldquomet the expectationrdquo

ldquoSearch to find a specific datasethelliprdquo

Mobile users (1) ldquoFoodrdquo 100 (1) Syliva (3) searched the term ldquofoodrdquo and saidldquoFood because I enjoy fine dining Was expecting to see a list of restaurantsrdquo

Laptop users (1) ldquoBuilding safety datardquo 100 (1)

ldquoOtherrdquo

Mobile users (4) Clicked on a Chicago App OpenGrid 25 (1) Clicked on a Digital Chicago topic 25 (1) Other Clicked on menu on the upper right 25 (1) Other Clicked on Divvy banner 25 (1)

VP (12) clicked on Divvy banner because the user ldquothought it would take me to a list of subcategories but it didnrsquotrdquo

6

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for a dataset youre interested in learning more about How did you search

Chose a data catalog category 59 (10) Search to find a specific dataset box 35 (6) Used search in navigation 6 (1)

Search term (if used) Animal control (1) CPS (1) Crime statistics for Lake View (1) Government (1) Lead testing schools (1) Nursing (1) Potholes (1)

Category (if used) Buildings (2) Community (1) Events (1) FOIA (1) Historic Preservation (1) Sanitation (1) Service Requests (1) Transportation (1)

Tell us more about your search experience Did you find what you were looking for

8 testers had difficulty finding datasets they wanted or said the ldquolanguagerdquo made it hard to find desired results 2 testers expected city services such as employment or 311 service requests 3 testers were still exploring the site generally trying to learn more about what is listed in certain categories 2 testers mentioned that the data appeared outdated while 2 other testers could not distinguish a rational order to the results

Miko (13) typed in ldquopublic transportation and received ldquoOpen Spacesrdquo results which were not relevant to her She mentioned how this site uses ldquoA certain type of lingo for certain type of peoplehellip not geared towards a regular Jane or Joerdquo

Did the search meet your expectations Out of the 7 testers who searched

Yes 29 (2) No 71 (5)

Out of the 10 testers who chose a category

Yes 30 (3) No 70 (7)

Why or why not

When expectations were not met a lot of the time it was because it took too long or too many steps for the tester to find their specific query Some testers again expected city services or

7

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal

Lhibbard (7) did not find what they were looking for and chose the service requests category and saidldquoI found the news about 311 service requests I thought I might be able to make a personal request directly to the aldermanrdquo Thrones (1) searched ldquonursingrdquo and it did not meet expectations as tester was looking to review human resources or employment information

For those that had expectations that were met testers were pleased to find and that the homepage provided more options and details

TJ (5) found what they were looking for and chose a data category They said they were ldquoLooking for libraries and found locationshoursrdquo Batman (14) searched for ldquopotholesrdquo and found what he expected ldquoCurious that they list recent pothole fixes Wanted to see potholes patched recently and got what tester expectedrdquo

Search for the building permits dataset How did you search Buildings Category gt Building Permits dataset 70 (12)

Search to find a specific datasethellip box 24 (4) Buildings Category gt Building Permits data lens 6 (1)

One tester Miko (13) first searched but the results did not load on their mobile device so the tester then clicked on the ldquoBuildingsrdquo category

Search term (if used) Building permits (4)

Anything else to note about this search experience Most testers agreed this search was simple and straightforward but GPBlight (4) suggested that in the data results there ldquoshould be an option for ward which is not on the maprdquo and Batman (14) thought it was ldquoInteresting that some things go back 2 years others 10 years Might be useful if data can be accessed further backrdquo

Next search for a map of crime data How did you search Public Safety Category gt Crimes map 35 (6) Public Safety Category gt Crimes - 2001 to present dataset 23 (4) Public Safety Category gt Crimes - 2001 to present data lens 18 (3)

8

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Introduction For the 28th Civic User Testing Group (CUTGroup) test we tested the redesigned homepage of the City of Chicagorsquos Open Data Portal The Open Data Portal allows users to find resources and various datasets regarding the city of Chicago The City of Chicago Department of Innovation and Technology is working with Socrata to redesign the Open Data Portal focused currently on the homepage to be more user-friendly while representing multiple data and technology initiatives and applications created with open data Target Group

On September 22 2016 we sent an email to 1172 CUTGroup testers who lived in Chicago and notification via text message to our testers who prefer to receive notification via text We wanted to know if they would be available for an in-person test on September 28 When segmenting our testers we were interested in testing with

Testers who use a variety of devices Android iOS and laptops We wanted to have an almost equal representation on all these devices types to better understand the homepagersquos functionality on different devices

Testers who have used the Open Data Portal in the past and those who have never used the Open Data Portal before We wanted to have people compare both data portals while including new first impressions

Testers who have all levels of data experience to see how user-friendly the search functionality is

Test type

In-person test proctored on both mobile and laptop devices We included 6 testers on laptop devices 6 testers on Android mobile devices and 5 testers on iOS mobile devices (total of 17 testers)

Testers were paired with proctors to record their interactions and comments or opinions about the City of Chicago Open Data Portal homepage

Goals of Test The main goal is to understand how testers who have some familiarity with the data portal (even minimum) respond to the changes made to the homepage We want to capture how residents search and what is the best structure for the homepage to make searching easier Lastly we want to see how responsive testers are to the other content that relates to the programs and tech initiatives of the City of Chicago

1

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Responses 17 testers completed the in-person CUTGroup test of the City of Chicagorsquos Open Data Portal homepage at Woodson Regional Library 9525 S Halsted St Chicago IL 60628 on September 28 2016

Background questions

What successes or challenges have you had finding data that you were looking for or using the Open Data Portal Tell us about any of your past experiences 6 out of 17 testers had no previous experience with datasets or using the Open Data Portal 8 testers expressed positive experiences using Chicagorsquos Open Data Portal For example

CommunityResearcher 201601 (11) saidldquoSomewhat successful Used to research a paper a number of years ago [and] was able to find what I needed on the elderly populationrdquo

3 testers shared challenges using Chicagorsquos Open Data Portal

Syliva (3) saidldquoItrsquos not that data is difficult to access but the results are not relevant If Im looking up restaurant data I dont want school data for instance Results are not always relevantrdquo Blessed (6) saidldquoI usually look for crime in the immediate block Theftrobbery person or house [It] wasnrsquot accuraterdquo

We learned from this CUTGroup test that there was some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website Some testers very clearly used the Open Data Portal while a few testers shared experiences finding city services which led us to believe that they were speaking about the official City of Chicago website

What are your expectations of what the City of Chicago Open Data Portal would be useful for Here are some responses from testers about what they expect the City of Chicago Open Data Portal would be useful for (number indicates number of times a response was given as testers sometimes had multiple answers)

6 ndash Search for specific data 3 ndash Analyze data

2

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

3 ndash Find City services 2 ndash Creates transparency 2 ndash Find jobs 1 ndash Find doctors 1 ndash Mapping 1 ndash Develop apps using the data 1 ndash Pay parking tickets 1 ndash Communicate with Aldermen

Gabireads (2) said that the Open Data Portal is useful for ldquoMapping Creating thematic maps Census crime restaurant neighborhood data to make maps and analyze datardquo Batman (14) saidldquo To have datainfo on various municipal systems licenses historical data on various going-onsrdquo M (15) mentionedldquoGreat if everything that is not confidential was available Would enable better social science and community involvement services security etcrdquo

Initial Homepage Review

Review the homepage and tell us about your first impressions While you review try not to click on the next page just yet but feel free to scroll up and down the page When reviewing the redesigned homepage for the first time 13 testers had initial positive impressions 2 had critical impressions and 2 had neutral impressions

Mobile users The majority of mobile testers had a positive reaction to the homepage with 5 testers mentioning that it seemed straight-forward 3 testers had negative responses when viewing the homepage on their mobile device 4 out of 11 testers specifically mentioned the categories whereas only 1 tester mentioned the search bar Other things testers mentioned Divvy (3) ldquoService Trackerrdquo (2) clearreadable font (2) specific datasets (2)

Funky Burrito (10) says ldquoWell laid out straightforward Right amount of pictures versus text content Feels easy to navigate based on criteria of informationrdquo

VP (12) thought that the homepage is ldquoWell organized Preview of what is to come is on the banner I assume that I would be able to swipe but you cannot Also lots of scrolling Visually seems very cramped not alignedrdquo

3

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

GPBlight (4) believe that ldquoFlashing promotions are unnecessary It should focus on the top three items people search for not what they are pushing in relation to the Divvy advertisement

Laptop users 4 out of the 6 testers who viewed the homepage mentioned the categories in their initial review while no one mentioned the search bar Most testers had a positive reaction to the homepage while 1 tester specifically said that there was too much information Other things testers mentioned Divvy (1) ldquoChicago appsrdquo (1) ldquoHow to videordquo (1)

Batman (14) says ldquoLooks clean and modern The old one wasnt easy to navigate and this page has broken up categories to find stuff more easily Info is more visualrdquo TJ (5) thought there is ldquoToo much information and stuff on the homepage lsquoDigital Chicagorsquo section should be biggerrdquo

Lhibbard (7) says ldquoNot exactly what I expected Didnt expect to see a map or information about Divvy Thought it would highlight city services Noticed the Map first Looks more fresh and updatedrdquo

What are three actions that you know you can do just from reviewing this first page Here are responses we have heard from testers about actions they can do just from reviewing the homepage (number indicates number of times a response was given as testers sometimes had multiple answers)

6 ndash Browse data catalog by choosing category 6 ndash TutorialsVideos 4 ndash Learn about community 4 ndash Use Chicago apps 4 ndash Request city services (street sweeping garbage pick-up 311 etc) 2 ndash Find events 2 ndash Navigate crime data 2 ndash Click on menu 2 ndash Type information in search box 2 ndash Transportation 2 ndash Find health and human services 1 ndash Environmental data

4

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

1 ndash Review free maps

Thrones (1) mentionedldquoLook at different events coordinate routes and find out whats going on in your communityrdquo Syliva (3) saidldquoClick on menu can type in info in search box or I can choose categories from belowrdquo Lhibbard (7) saidldquoLook up community health and human services look up information about transportationrdquo

Who do you think this website is designed for Why

14 testers mentioned that this Open Data Portal is designed for the general publicresidents to help find city servicesresources (3) government transparency (1) and access data (1) Testers who said it was for residents said so because it is ldquoeasy to use and not laden with jargonrdquo (Gabireads 2) and ldquoto help people find resources in the City of Chicagordquo (Thrones 1) 3 testers specifically said the Open Data Portal is designed with people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis 1 tester said it was built for business owners while another said it was built for developers

After reviewing the homepage what is the first thing you want to do or click

Clicked on a data catalog category 59 (10) Search to find a specific dataset 18 (3) Clicked on a Digital Chicago topic 6 (1) Clicked on a Chicago App 6 (1) Other Clicked on menu on the upper right 6 (1) Other Clicked on Divvy banner 6 (1)

What did you search Or what was the category you chose Category FOIA (3) Education (3) Events (2) Transportation (1) Health amp Human Services (1) Search terms Food (1) Building safety data (1) Community (1)

Tell us more about the first thing you did and why you did it What were you expecting to happen 5 testers expected to find more general information about what the website does or the type of information listed under a category 4 testers had specific topics to query 1 tester expected city service options

5

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Clicked on a data catalog category

Mobile users (6) FOIA 50 (3) Education 33 (2) Events 17 (1)

Laptop users (6) Human Health Services 32 (2) Transportation 17 (1) Community 17 (1) Education 17 (1) Events 17 (1)

TJ (5) chose the transportation category because he ldquowanted to find CTA maps dealing with train transportationrdquo and it ldquomet the expectationrdquo

ldquoSearch to find a specific datasethelliprdquo

Mobile users (1) ldquoFoodrdquo 100 (1) Syliva (3) searched the term ldquofoodrdquo and saidldquoFood because I enjoy fine dining Was expecting to see a list of restaurantsrdquo

Laptop users (1) ldquoBuilding safety datardquo 100 (1)

ldquoOtherrdquo

Mobile users (4) Clicked on a Chicago App OpenGrid 25 (1) Clicked on a Digital Chicago topic 25 (1) Other Clicked on menu on the upper right 25 (1) Other Clicked on Divvy banner 25 (1)

VP (12) clicked on Divvy banner because the user ldquothought it would take me to a list of subcategories but it didnrsquotrdquo

6

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for a dataset youre interested in learning more about How did you search

Chose a data catalog category 59 (10) Search to find a specific dataset box 35 (6) Used search in navigation 6 (1)

Search term (if used) Animal control (1) CPS (1) Crime statistics for Lake View (1) Government (1) Lead testing schools (1) Nursing (1) Potholes (1)

Category (if used) Buildings (2) Community (1) Events (1) FOIA (1) Historic Preservation (1) Sanitation (1) Service Requests (1) Transportation (1)

Tell us more about your search experience Did you find what you were looking for

8 testers had difficulty finding datasets they wanted or said the ldquolanguagerdquo made it hard to find desired results 2 testers expected city services such as employment or 311 service requests 3 testers were still exploring the site generally trying to learn more about what is listed in certain categories 2 testers mentioned that the data appeared outdated while 2 other testers could not distinguish a rational order to the results

Miko (13) typed in ldquopublic transportation and received ldquoOpen Spacesrdquo results which were not relevant to her She mentioned how this site uses ldquoA certain type of lingo for certain type of peoplehellip not geared towards a regular Jane or Joerdquo

Did the search meet your expectations Out of the 7 testers who searched

Yes 29 (2) No 71 (5)

Out of the 10 testers who chose a category

Yes 30 (3) No 70 (7)

Why or why not

When expectations were not met a lot of the time it was because it took too long or too many steps for the tester to find their specific query Some testers again expected city services or

7

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal

Lhibbard (7) did not find what they were looking for and chose the service requests category and saidldquoI found the news about 311 service requests I thought I might be able to make a personal request directly to the aldermanrdquo Thrones (1) searched ldquonursingrdquo and it did not meet expectations as tester was looking to review human resources or employment information

For those that had expectations that were met testers were pleased to find and that the homepage provided more options and details

TJ (5) found what they were looking for and chose a data category They said they were ldquoLooking for libraries and found locationshoursrdquo Batman (14) searched for ldquopotholesrdquo and found what he expected ldquoCurious that they list recent pothole fixes Wanted to see potholes patched recently and got what tester expectedrdquo

Search for the building permits dataset How did you search Buildings Category gt Building Permits dataset 70 (12)

Search to find a specific datasethellip box 24 (4) Buildings Category gt Building Permits data lens 6 (1)

One tester Miko (13) first searched but the results did not load on their mobile device so the tester then clicked on the ldquoBuildingsrdquo category

Search term (if used) Building permits (4)

Anything else to note about this search experience Most testers agreed this search was simple and straightforward but GPBlight (4) suggested that in the data results there ldquoshould be an option for ward which is not on the maprdquo and Batman (14) thought it was ldquoInteresting that some things go back 2 years others 10 years Might be useful if data can be accessed further backrdquo

Next search for a map of crime data How did you search Public Safety Category gt Crimes map 35 (6) Public Safety Category gt Crimes - 2001 to present dataset 23 (4) Public Safety Category gt Crimes - 2001 to present data lens 18 (3)

8

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Responses 17 testers completed the in-person CUTGroup test of the City of Chicagorsquos Open Data Portal homepage at Woodson Regional Library 9525 S Halsted St Chicago IL 60628 on September 28 2016

Background questions

What successes or challenges have you had finding data that you were looking for or using the Open Data Portal Tell us about any of your past experiences 6 out of 17 testers had no previous experience with datasets or using the Open Data Portal 8 testers expressed positive experiences using Chicagorsquos Open Data Portal For example

CommunityResearcher 201601 (11) saidldquoSomewhat successful Used to research a paper a number of years ago [and] was able to find what I needed on the elderly populationrdquo

3 testers shared challenges using Chicagorsquos Open Data Portal

Syliva (3) saidldquoItrsquos not that data is difficult to access but the results are not relevant If Im looking up restaurant data I dont want school data for instance Results are not always relevantrdquo Blessed (6) saidldquoI usually look for crime in the immediate block Theftrobbery person or house [It] wasnrsquot accuraterdquo

We learned from this CUTGroup test that there was some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website Some testers very clearly used the Open Data Portal while a few testers shared experiences finding city services which led us to believe that they were speaking about the official City of Chicago website

What are your expectations of what the City of Chicago Open Data Portal would be useful for Here are some responses from testers about what they expect the City of Chicago Open Data Portal would be useful for (number indicates number of times a response was given as testers sometimes had multiple answers)

6 ndash Search for specific data 3 ndash Analyze data

2

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

3 ndash Find City services 2 ndash Creates transparency 2 ndash Find jobs 1 ndash Find doctors 1 ndash Mapping 1 ndash Develop apps using the data 1 ndash Pay parking tickets 1 ndash Communicate with Aldermen

Gabireads (2) said that the Open Data Portal is useful for ldquoMapping Creating thematic maps Census crime restaurant neighborhood data to make maps and analyze datardquo Batman (14) saidldquo To have datainfo on various municipal systems licenses historical data on various going-onsrdquo M (15) mentionedldquoGreat if everything that is not confidential was available Would enable better social science and community involvement services security etcrdquo

Initial Homepage Review

Review the homepage and tell us about your first impressions While you review try not to click on the next page just yet but feel free to scroll up and down the page When reviewing the redesigned homepage for the first time 13 testers had initial positive impressions 2 had critical impressions and 2 had neutral impressions

Mobile users The majority of mobile testers had a positive reaction to the homepage with 5 testers mentioning that it seemed straight-forward 3 testers had negative responses when viewing the homepage on their mobile device 4 out of 11 testers specifically mentioned the categories whereas only 1 tester mentioned the search bar Other things testers mentioned Divvy (3) ldquoService Trackerrdquo (2) clearreadable font (2) specific datasets (2)

Funky Burrito (10) says ldquoWell laid out straightforward Right amount of pictures versus text content Feels easy to navigate based on criteria of informationrdquo

VP (12) thought that the homepage is ldquoWell organized Preview of what is to come is on the banner I assume that I would be able to swipe but you cannot Also lots of scrolling Visually seems very cramped not alignedrdquo

3

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

GPBlight (4) believe that ldquoFlashing promotions are unnecessary It should focus on the top three items people search for not what they are pushing in relation to the Divvy advertisement

Laptop users 4 out of the 6 testers who viewed the homepage mentioned the categories in their initial review while no one mentioned the search bar Most testers had a positive reaction to the homepage while 1 tester specifically said that there was too much information Other things testers mentioned Divvy (1) ldquoChicago appsrdquo (1) ldquoHow to videordquo (1)

Batman (14) says ldquoLooks clean and modern The old one wasnt easy to navigate and this page has broken up categories to find stuff more easily Info is more visualrdquo TJ (5) thought there is ldquoToo much information and stuff on the homepage lsquoDigital Chicagorsquo section should be biggerrdquo

Lhibbard (7) says ldquoNot exactly what I expected Didnt expect to see a map or information about Divvy Thought it would highlight city services Noticed the Map first Looks more fresh and updatedrdquo

What are three actions that you know you can do just from reviewing this first page Here are responses we have heard from testers about actions they can do just from reviewing the homepage (number indicates number of times a response was given as testers sometimes had multiple answers)

6 ndash Browse data catalog by choosing category 6 ndash TutorialsVideos 4 ndash Learn about community 4 ndash Use Chicago apps 4 ndash Request city services (street sweeping garbage pick-up 311 etc) 2 ndash Find events 2 ndash Navigate crime data 2 ndash Click on menu 2 ndash Type information in search box 2 ndash Transportation 2 ndash Find health and human services 1 ndash Environmental data

4

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

1 ndash Review free maps

Thrones (1) mentionedldquoLook at different events coordinate routes and find out whats going on in your communityrdquo Syliva (3) saidldquoClick on menu can type in info in search box or I can choose categories from belowrdquo Lhibbard (7) saidldquoLook up community health and human services look up information about transportationrdquo

Who do you think this website is designed for Why

14 testers mentioned that this Open Data Portal is designed for the general publicresidents to help find city servicesresources (3) government transparency (1) and access data (1) Testers who said it was for residents said so because it is ldquoeasy to use and not laden with jargonrdquo (Gabireads 2) and ldquoto help people find resources in the City of Chicagordquo (Thrones 1) 3 testers specifically said the Open Data Portal is designed with people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis 1 tester said it was built for business owners while another said it was built for developers

After reviewing the homepage what is the first thing you want to do or click

Clicked on a data catalog category 59 (10) Search to find a specific dataset 18 (3) Clicked on a Digital Chicago topic 6 (1) Clicked on a Chicago App 6 (1) Other Clicked on menu on the upper right 6 (1) Other Clicked on Divvy banner 6 (1)

What did you search Or what was the category you chose Category FOIA (3) Education (3) Events (2) Transportation (1) Health amp Human Services (1) Search terms Food (1) Building safety data (1) Community (1)

Tell us more about the first thing you did and why you did it What were you expecting to happen 5 testers expected to find more general information about what the website does or the type of information listed under a category 4 testers had specific topics to query 1 tester expected city service options

5

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Clicked on a data catalog category

Mobile users (6) FOIA 50 (3) Education 33 (2) Events 17 (1)

Laptop users (6) Human Health Services 32 (2) Transportation 17 (1) Community 17 (1) Education 17 (1) Events 17 (1)

TJ (5) chose the transportation category because he ldquowanted to find CTA maps dealing with train transportationrdquo and it ldquomet the expectationrdquo

ldquoSearch to find a specific datasethelliprdquo

Mobile users (1) ldquoFoodrdquo 100 (1) Syliva (3) searched the term ldquofoodrdquo and saidldquoFood because I enjoy fine dining Was expecting to see a list of restaurantsrdquo

Laptop users (1) ldquoBuilding safety datardquo 100 (1)

ldquoOtherrdquo

Mobile users (4) Clicked on a Chicago App OpenGrid 25 (1) Clicked on a Digital Chicago topic 25 (1) Other Clicked on menu on the upper right 25 (1) Other Clicked on Divvy banner 25 (1)

VP (12) clicked on Divvy banner because the user ldquothought it would take me to a list of subcategories but it didnrsquotrdquo

6

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for a dataset youre interested in learning more about How did you search

Chose a data catalog category 59 (10) Search to find a specific dataset box 35 (6) Used search in navigation 6 (1)

Search term (if used) Animal control (1) CPS (1) Crime statistics for Lake View (1) Government (1) Lead testing schools (1) Nursing (1) Potholes (1)

Category (if used) Buildings (2) Community (1) Events (1) FOIA (1) Historic Preservation (1) Sanitation (1) Service Requests (1) Transportation (1)

Tell us more about your search experience Did you find what you were looking for

8 testers had difficulty finding datasets they wanted or said the ldquolanguagerdquo made it hard to find desired results 2 testers expected city services such as employment or 311 service requests 3 testers were still exploring the site generally trying to learn more about what is listed in certain categories 2 testers mentioned that the data appeared outdated while 2 other testers could not distinguish a rational order to the results

Miko (13) typed in ldquopublic transportation and received ldquoOpen Spacesrdquo results which were not relevant to her She mentioned how this site uses ldquoA certain type of lingo for certain type of peoplehellip not geared towards a regular Jane or Joerdquo

Did the search meet your expectations Out of the 7 testers who searched

Yes 29 (2) No 71 (5)

Out of the 10 testers who chose a category

Yes 30 (3) No 70 (7)

Why or why not

When expectations were not met a lot of the time it was because it took too long or too many steps for the tester to find their specific query Some testers again expected city services or

7

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal

Lhibbard (7) did not find what they were looking for and chose the service requests category and saidldquoI found the news about 311 service requests I thought I might be able to make a personal request directly to the aldermanrdquo Thrones (1) searched ldquonursingrdquo and it did not meet expectations as tester was looking to review human resources or employment information

For those that had expectations that were met testers were pleased to find and that the homepage provided more options and details

TJ (5) found what they were looking for and chose a data category They said they were ldquoLooking for libraries and found locationshoursrdquo Batman (14) searched for ldquopotholesrdquo and found what he expected ldquoCurious that they list recent pothole fixes Wanted to see potholes patched recently and got what tester expectedrdquo

Search for the building permits dataset How did you search Buildings Category gt Building Permits dataset 70 (12)

Search to find a specific datasethellip box 24 (4) Buildings Category gt Building Permits data lens 6 (1)

One tester Miko (13) first searched but the results did not load on their mobile device so the tester then clicked on the ldquoBuildingsrdquo category

Search term (if used) Building permits (4)

Anything else to note about this search experience Most testers agreed this search was simple and straightforward but GPBlight (4) suggested that in the data results there ldquoshould be an option for ward which is not on the maprdquo and Batman (14) thought it was ldquoInteresting that some things go back 2 years others 10 years Might be useful if data can be accessed further backrdquo

Next search for a map of crime data How did you search Public Safety Category gt Crimes map 35 (6) Public Safety Category gt Crimes - 2001 to present dataset 23 (4) Public Safety Category gt Crimes - 2001 to present data lens 18 (3)

8

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

3 ndash Find City services 2 ndash Creates transparency 2 ndash Find jobs 1 ndash Find doctors 1 ndash Mapping 1 ndash Develop apps using the data 1 ndash Pay parking tickets 1 ndash Communicate with Aldermen

Gabireads (2) said that the Open Data Portal is useful for ldquoMapping Creating thematic maps Census crime restaurant neighborhood data to make maps and analyze datardquo Batman (14) saidldquo To have datainfo on various municipal systems licenses historical data on various going-onsrdquo M (15) mentionedldquoGreat if everything that is not confidential was available Would enable better social science and community involvement services security etcrdquo

Initial Homepage Review

Review the homepage and tell us about your first impressions While you review try not to click on the next page just yet but feel free to scroll up and down the page When reviewing the redesigned homepage for the first time 13 testers had initial positive impressions 2 had critical impressions and 2 had neutral impressions

Mobile users The majority of mobile testers had a positive reaction to the homepage with 5 testers mentioning that it seemed straight-forward 3 testers had negative responses when viewing the homepage on their mobile device 4 out of 11 testers specifically mentioned the categories whereas only 1 tester mentioned the search bar Other things testers mentioned Divvy (3) ldquoService Trackerrdquo (2) clearreadable font (2) specific datasets (2)

Funky Burrito (10) says ldquoWell laid out straightforward Right amount of pictures versus text content Feels easy to navigate based on criteria of informationrdquo

VP (12) thought that the homepage is ldquoWell organized Preview of what is to come is on the banner I assume that I would be able to swipe but you cannot Also lots of scrolling Visually seems very cramped not alignedrdquo

3

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

GPBlight (4) believe that ldquoFlashing promotions are unnecessary It should focus on the top three items people search for not what they are pushing in relation to the Divvy advertisement

Laptop users 4 out of the 6 testers who viewed the homepage mentioned the categories in their initial review while no one mentioned the search bar Most testers had a positive reaction to the homepage while 1 tester specifically said that there was too much information Other things testers mentioned Divvy (1) ldquoChicago appsrdquo (1) ldquoHow to videordquo (1)

Batman (14) says ldquoLooks clean and modern The old one wasnt easy to navigate and this page has broken up categories to find stuff more easily Info is more visualrdquo TJ (5) thought there is ldquoToo much information and stuff on the homepage lsquoDigital Chicagorsquo section should be biggerrdquo

Lhibbard (7) says ldquoNot exactly what I expected Didnt expect to see a map or information about Divvy Thought it would highlight city services Noticed the Map first Looks more fresh and updatedrdquo

What are three actions that you know you can do just from reviewing this first page Here are responses we have heard from testers about actions they can do just from reviewing the homepage (number indicates number of times a response was given as testers sometimes had multiple answers)

6 ndash Browse data catalog by choosing category 6 ndash TutorialsVideos 4 ndash Learn about community 4 ndash Use Chicago apps 4 ndash Request city services (street sweeping garbage pick-up 311 etc) 2 ndash Find events 2 ndash Navigate crime data 2 ndash Click on menu 2 ndash Type information in search box 2 ndash Transportation 2 ndash Find health and human services 1 ndash Environmental data

4

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

1 ndash Review free maps

Thrones (1) mentionedldquoLook at different events coordinate routes and find out whats going on in your communityrdquo Syliva (3) saidldquoClick on menu can type in info in search box or I can choose categories from belowrdquo Lhibbard (7) saidldquoLook up community health and human services look up information about transportationrdquo

Who do you think this website is designed for Why

14 testers mentioned that this Open Data Portal is designed for the general publicresidents to help find city servicesresources (3) government transparency (1) and access data (1) Testers who said it was for residents said so because it is ldquoeasy to use and not laden with jargonrdquo (Gabireads 2) and ldquoto help people find resources in the City of Chicagordquo (Thrones 1) 3 testers specifically said the Open Data Portal is designed with people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis 1 tester said it was built for business owners while another said it was built for developers

After reviewing the homepage what is the first thing you want to do or click

Clicked on a data catalog category 59 (10) Search to find a specific dataset 18 (3) Clicked on a Digital Chicago topic 6 (1) Clicked on a Chicago App 6 (1) Other Clicked on menu on the upper right 6 (1) Other Clicked on Divvy banner 6 (1)

What did you search Or what was the category you chose Category FOIA (3) Education (3) Events (2) Transportation (1) Health amp Human Services (1) Search terms Food (1) Building safety data (1) Community (1)

Tell us more about the first thing you did and why you did it What were you expecting to happen 5 testers expected to find more general information about what the website does or the type of information listed under a category 4 testers had specific topics to query 1 tester expected city service options

5

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Clicked on a data catalog category

Mobile users (6) FOIA 50 (3) Education 33 (2) Events 17 (1)

Laptop users (6) Human Health Services 32 (2) Transportation 17 (1) Community 17 (1) Education 17 (1) Events 17 (1)

TJ (5) chose the transportation category because he ldquowanted to find CTA maps dealing with train transportationrdquo and it ldquomet the expectationrdquo

ldquoSearch to find a specific datasethelliprdquo

Mobile users (1) ldquoFoodrdquo 100 (1) Syliva (3) searched the term ldquofoodrdquo and saidldquoFood because I enjoy fine dining Was expecting to see a list of restaurantsrdquo

Laptop users (1) ldquoBuilding safety datardquo 100 (1)

ldquoOtherrdquo

Mobile users (4) Clicked on a Chicago App OpenGrid 25 (1) Clicked on a Digital Chicago topic 25 (1) Other Clicked on menu on the upper right 25 (1) Other Clicked on Divvy banner 25 (1)

VP (12) clicked on Divvy banner because the user ldquothought it would take me to a list of subcategories but it didnrsquotrdquo

6

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for a dataset youre interested in learning more about How did you search

Chose a data catalog category 59 (10) Search to find a specific dataset box 35 (6) Used search in navigation 6 (1)

Search term (if used) Animal control (1) CPS (1) Crime statistics for Lake View (1) Government (1) Lead testing schools (1) Nursing (1) Potholes (1)

Category (if used) Buildings (2) Community (1) Events (1) FOIA (1) Historic Preservation (1) Sanitation (1) Service Requests (1) Transportation (1)

Tell us more about your search experience Did you find what you were looking for

8 testers had difficulty finding datasets they wanted or said the ldquolanguagerdquo made it hard to find desired results 2 testers expected city services such as employment or 311 service requests 3 testers were still exploring the site generally trying to learn more about what is listed in certain categories 2 testers mentioned that the data appeared outdated while 2 other testers could not distinguish a rational order to the results

Miko (13) typed in ldquopublic transportation and received ldquoOpen Spacesrdquo results which were not relevant to her She mentioned how this site uses ldquoA certain type of lingo for certain type of peoplehellip not geared towards a regular Jane or Joerdquo

Did the search meet your expectations Out of the 7 testers who searched

Yes 29 (2) No 71 (5)

Out of the 10 testers who chose a category

Yes 30 (3) No 70 (7)

Why or why not

When expectations were not met a lot of the time it was because it took too long or too many steps for the tester to find their specific query Some testers again expected city services or

7

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal

Lhibbard (7) did not find what they were looking for and chose the service requests category and saidldquoI found the news about 311 service requests I thought I might be able to make a personal request directly to the aldermanrdquo Thrones (1) searched ldquonursingrdquo and it did not meet expectations as tester was looking to review human resources or employment information

For those that had expectations that were met testers were pleased to find and that the homepage provided more options and details

TJ (5) found what they were looking for and chose a data category They said they were ldquoLooking for libraries and found locationshoursrdquo Batman (14) searched for ldquopotholesrdquo and found what he expected ldquoCurious that they list recent pothole fixes Wanted to see potholes patched recently and got what tester expectedrdquo

Search for the building permits dataset How did you search Buildings Category gt Building Permits dataset 70 (12)

Search to find a specific datasethellip box 24 (4) Buildings Category gt Building Permits data lens 6 (1)

One tester Miko (13) first searched but the results did not load on their mobile device so the tester then clicked on the ldquoBuildingsrdquo category

Search term (if used) Building permits (4)

Anything else to note about this search experience Most testers agreed this search was simple and straightforward but GPBlight (4) suggested that in the data results there ldquoshould be an option for ward which is not on the maprdquo and Batman (14) thought it was ldquoInteresting that some things go back 2 years others 10 years Might be useful if data can be accessed further backrdquo

Next search for a map of crime data How did you search Public Safety Category gt Crimes map 35 (6) Public Safety Category gt Crimes - 2001 to present dataset 23 (4) Public Safety Category gt Crimes - 2001 to present data lens 18 (3)

8

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

GPBlight (4) believe that ldquoFlashing promotions are unnecessary It should focus on the top three items people search for not what they are pushing in relation to the Divvy advertisement

Laptop users 4 out of the 6 testers who viewed the homepage mentioned the categories in their initial review while no one mentioned the search bar Most testers had a positive reaction to the homepage while 1 tester specifically said that there was too much information Other things testers mentioned Divvy (1) ldquoChicago appsrdquo (1) ldquoHow to videordquo (1)

Batman (14) says ldquoLooks clean and modern The old one wasnt easy to navigate and this page has broken up categories to find stuff more easily Info is more visualrdquo TJ (5) thought there is ldquoToo much information and stuff on the homepage lsquoDigital Chicagorsquo section should be biggerrdquo

Lhibbard (7) says ldquoNot exactly what I expected Didnt expect to see a map or information about Divvy Thought it would highlight city services Noticed the Map first Looks more fresh and updatedrdquo

What are three actions that you know you can do just from reviewing this first page Here are responses we have heard from testers about actions they can do just from reviewing the homepage (number indicates number of times a response was given as testers sometimes had multiple answers)

6 ndash Browse data catalog by choosing category 6 ndash TutorialsVideos 4 ndash Learn about community 4 ndash Use Chicago apps 4 ndash Request city services (street sweeping garbage pick-up 311 etc) 2 ndash Find events 2 ndash Navigate crime data 2 ndash Click on menu 2 ndash Type information in search box 2 ndash Transportation 2 ndash Find health and human services 1 ndash Environmental data

4

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

1 ndash Review free maps

Thrones (1) mentionedldquoLook at different events coordinate routes and find out whats going on in your communityrdquo Syliva (3) saidldquoClick on menu can type in info in search box or I can choose categories from belowrdquo Lhibbard (7) saidldquoLook up community health and human services look up information about transportationrdquo

Who do you think this website is designed for Why

14 testers mentioned that this Open Data Portal is designed for the general publicresidents to help find city servicesresources (3) government transparency (1) and access data (1) Testers who said it was for residents said so because it is ldquoeasy to use and not laden with jargonrdquo (Gabireads 2) and ldquoto help people find resources in the City of Chicagordquo (Thrones 1) 3 testers specifically said the Open Data Portal is designed with people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis 1 tester said it was built for business owners while another said it was built for developers

After reviewing the homepage what is the first thing you want to do or click

Clicked on a data catalog category 59 (10) Search to find a specific dataset 18 (3) Clicked on a Digital Chicago topic 6 (1) Clicked on a Chicago App 6 (1) Other Clicked on menu on the upper right 6 (1) Other Clicked on Divvy banner 6 (1)

What did you search Or what was the category you chose Category FOIA (3) Education (3) Events (2) Transportation (1) Health amp Human Services (1) Search terms Food (1) Building safety data (1) Community (1)

Tell us more about the first thing you did and why you did it What were you expecting to happen 5 testers expected to find more general information about what the website does or the type of information listed under a category 4 testers had specific topics to query 1 tester expected city service options

5

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Clicked on a data catalog category

Mobile users (6) FOIA 50 (3) Education 33 (2) Events 17 (1)

Laptop users (6) Human Health Services 32 (2) Transportation 17 (1) Community 17 (1) Education 17 (1) Events 17 (1)

TJ (5) chose the transportation category because he ldquowanted to find CTA maps dealing with train transportationrdquo and it ldquomet the expectationrdquo

ldquoSearch to find a specific datasethelliprdquo

Mobile users (1) ldquoFoodrdquo 100 (1) Syliva (3) searched the term ldquofoodrdquo and saidldquoFood because I enjoy fine dining Was expecting to see a list of restaurantsrdquo

Laptop users (1) ldquoBuilding safety datardquo 100 (1)

ldquoOtherrdquo

Mobile users (4) Clicked on a Chicago App OpenGrid 25 (1) Clicked on a Digital Chicago topic 25 (1) Other Clicked on menu on the upper right 25 (1) Other Clicked on Divvy banner 25 (1)

VP (12) clicked on Divvy banner because the user ldquothought it would take me to a list of subcategories but it didnrsquotrdquo

6

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for a dataset youre interested in learning more about How did you search

Chose a data catalog category 59 (10) Search to find a specific dataset box 35 (6) Used search in navigation 6 (1)

Search term (if used) Animal control (1) CPS (1) Crime statistics for Lake View (1) Government (1) Lead testing schools (1) Nursing (1) Potholes (1)

Category (if used) Buildings (2) Community (1) Events (1) FOIA (1) Historic Preservation (1) Sanitation (1) Service Requests (1) Transportation (1)

Tell us more about your search experience Did you find what you were looking for

8 testers had difficulty finding datasets they wanted or said the ldquolanguagerdquo made it hard to find desired results 2 testers expected city services such as employment or 311 service requests 3 testers were still exploring the site generally trying to learn more about what is listed in certain categories 2 testers mentioned that the data appeared outdated while 2 other testers could not distinguish a rational order to the results

Miko (13) typed in ldquopublic transportation and received ldquoOpen Spacesrdquo results which were not relevant to her She mentioned how this site uses ldquoA certain type of lingo for certain type of peoplehellip not geared towards a regular Jane or Joerdquo

Did the search meet your expectations Out of the 7 testers who searched

Yes 29 (2) No 71 (5)

Out of the 10 testers who chose a category

Yes 30 (3) No 70 (7)

Why or why not

When expectations were not met a lot of the time it was because it took too long or too many steps for the tester to find their specific query Some testers again expected city services or

7

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal

Lhibbard (7) did not find what they were looking for and chose the service requests category and saidldquoI found the news about 311 service requests I thought I might be able to make a personal request directly to the aldermanrdquo Thrones (1) searched ldquonursingrdquo and it did not meet expectations as tester was looking to review human resources or employment information

For those that had expectations that were met testers were pleased to find and that the homepage provided more options and details

TJ (5) found what they were looking for and chose a data category They said they were ldquoLooking for libraries and found locationshoursrdquo Batman (14) searched for ldquopotholesrdquo and found what he expected ldquoCurious that they list recent pothole fixes Wanted to see potholes patched recently and got what tester expectedrdquo

Search for the building permits dataset How did you search Buildings Category gt Building Permits dataset 70 (12)

Search to find a specific datasethellip box 24 (4) Buildings Category gt Building Permits data lens 6 (1)

One tester Miko (13) first searched but the results did not load on their mobile device so the tester then clicked on the ldquoBuildingsrdquo category

Search term (if used) Building permits (4)

Anything else to note about this search experience Most testers agreed this search was simple and straightforward but GPBlight (4) suggested that in the data results there ldquoshould be an option for ward which is not on the maprdquo and Batman (14) thought it was ldquoInteresting that some things go back 2 years others 10 years Might be useful if data can be accessed further backrdquo

Next search for a map of crime data How did you search Public Safety Category gt Crimes map 35 (6) Public Safety Category gt Crimes - 2001 to present dataset 23 (4) Public Safety Category gt Crimes - 2001 to present data lens 18 (3)

8

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

1 ndash Review free maps

Thrones (1) mentionedldquoLook at different events coordinate routes and find out whats going on in your communityrdquo Syliva (3) saidldquoClick on menu can type in info in search box or I can choose categories from belowrdquo Lhibbard (7) saidldquoLook up community health and human services look up information about transportationrdquo

Who do you think this website is designed for Why

14 testers mentioned that this Open Data Portal is designed for the general publicresidents to help find city servicesresources (3) government transparency (1) and access data (1) Testers who said it was for residents said so because it is ldquoeasy to use and not laden with jargonrdquo (Gabireads 2) and ldquoto help people find resources in the City of Chicagordquo (Thrones 1) 3 testers specifically said the Open Data Portal is designed with people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis 1 tester said it was built for business owners while another said it was built for developers

After reviewing the homepage what is the first thing you want to do or click

Clicked on a data catalog category 59 (10) Search to find a specific dataset 18 (3) Clicked on a Digital Chicago topic 6 (1) Clicked on a Chicago App 6 (1) Other Clicked on menu on the upper right 6 (1) Other Clicked on Divvy banner 6 (1)

What did you search Or what was the category you chose Category FOIA (3) Education (3) Events (2) Transportation (1) Health amp Human Services (1) Search terms Food (1) Building safety data (1) Community (1)

Tell us more about the first thing you did and why you did it What were you expecting to happen 5 testers expected to find more general information about what the website does or the type of information listed under a category 4 testers had specific topics to query 1 tester expected city service options

5

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Clicked on a data catalog category

Mobile users (6) FOIA 50 (3) Education 33 (2) Events 17 (1)

Laptop users (6) Human Health Services 32 (2) Transportation 17 (1) Community 17 (1) Education 17 (1) Events 17 (1)

TJ (5) chose the transportation category because he ldquowanted to find CTA maps dealing with train transportationrdquo and it ldquomet the expectationrdquo

ldquoSearch to find a specific datasethelliprdquo

Mobile users (1) ldquoFoodrdquo 100 (1) Syliva (3) searched the term ldquofoodrdquo and saidldquoFood because I enjoy fine dining Was expecting to see a list of restaurantsrdquo

Laptop users (1) ldquoBuilding safety datardquo 100 (1)

ldquoOtherrdquo

Mobile users (4) Clicked on a Chicago App OpenGrid 25 (1) Clicked on a Digital Chicago topic 25 (1) Other Clicked on menu on the upper right 25 (1) Other Clicked on Divvy banner 25 (1)

VP (12) clicked on Divvy banner because the user ldquothought it would take me to a list of subcategories but it didnrsquotrdquo

6

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for a dataset youre interested in learning more about How did you search

Chose a data catalog category 59 (10) Search to find a specific dataset box 35 (6) Used search in navigation 6 (1)

Search term (if used) Animal control (1) CPS (1) Crime statistics for Lake View (1) Government (1) Lead testing schools (1) Nursing (1) Potholes (1)

Category (if used) Buildings (2) Community (1) Events (1) FOIA (1) Historic Preservation (1) Sanitation (1) Service Requests (1) Transportation (1)

Tell us more about your search experience Did you find what you were looking for

8 testers had difficulty finding datasets they wanted or said the ldquolanguagerdquo made it hard to find desired results 2 testers expected city services such as employment or 311 service requests 3 testers were still exploring the site generally trying to learn more about what is listed in certain categories 2 testers mentioned that the data appeared outdated while 2 other testers could not distinguish a rational order to the results

Miko (13) typed in ldquopublic transportation and received ldquoOpen Spacesrdquo results which were not relevant to her She mentioned how this site uses ldquoA certain type of lingo for certain type of peoplehellip not geared towards a regular Jane or Joerdquo

Did the search meet your expectations Out of the 7 testers who searched

Yes 29 (2) No 71 (5)

Out of the 10 testers who chose a category

Yes 30 (3) No 70 (7)

Why or why not

When expectations were not met a lot of the time it was because it took too long or too many steps for the tester to find their specific query Some testers again expected city services or

7

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal

Lhibbard (7) did not find what they were looking for and chose the service requests category and saidldquoI found the news about 311 service requests I thought I might be able to make a personal request directly to the aldermanrdquo Thrones (1) searched ldquonursingrdquo and it did not meet expectations as tester was looking to review human resources or employment information

For those that had expectations that were met testers were pleased to find and that the homepage provided more options and details

TJ (5) found what they were looking for and chose a data category They said they were ldquoLooking for libraries and found locationshoursrdquo Batman (14) searched for ldquopotholesrdquo and found what he expected ldquoCurious that they list recent pothole fixes Wanted to see potholes patched recently and got what tester expectedrdquo

Search for the building permits dataset How did you search Buildings Category gt Building Permits dataset 70 (12)

Search to find a specific datasethellip box 24 (4) Buildings Category gt Building Permits data lens 6 (1)

One tester Miko (13) first searched but the results did not load on their mobile device so the tester then clicked on the ldquoBuildingsrdquo category

Search term (if used) Building permits (4)

Anything else to note about this search experience Most testers agreed this search was simple and straightforward but GPBlight (4) suggested that in the data results there ldquoshould be an option for ward which is not on the maprdquo and Batman (14) thought it was ldquoInteresting that some things go back 2 years others 10 years Might be useful if data can be accessed further backrdquo

Next search for a map of crime data How did you search Public Safety Category gt Crimes map 35 (6) Public Safety Category gt Crimes - 2001 to present dataset 23 (4) Public Safety Category gt Crimes - 2001 to present data lens 18 (3)

8

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Clicked on a data catalog category

Mobile users (6) FOIA 50 (3) Education 33 (2) Events 17 (1)

Laptop users (6) Human Health Services 32 (2) Transportation 17 (1) Community 17 (1) Education 17 (1) Events 17 (1)

TJ (5) chose the transportation category because he ldquowanted to find CTA maps dealing with train transportationrdquo and it ldquomet the expectationrdquo

ldquoSearch to find a specific datasethelliprdquo

Mobile users (1) ldquoFoodrdquo 100 (1) Syliva (3) searched the term ldquofoodrdquo and saidldquoFood because I enjoy fine dining Was expecting to see a list of restaurantsrdquo

Laptop users (1) ldquoBuilding safety datardquo 100 (1)

ldquoOtherrdquo

Mobile users (4) Clicked on a Chicago App OpenGrid 25 (1) Clicked on a Digital Chicago topic 25 (1) Other Clicked on menu on the upper right 25 (1) Other Clicked on Divvy banner 25 (1)

VP (12) clicked on Divvy banner because the user ldquothought it would take me to a list of subcategories but it didnrsquotrdquo

6

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for a dataset youre interested in learning more about How did you search

Chose a data catalog category 59 (10) Search to find a specific dataset box 35 (6) Used search in navigation 6 (1)

Search term (if used) Animal control (1) CPS (1) Crime statistics for Lake View (1) Government (1) Lead testing schools (1) Nursing (1) Potholes (1)

Category (if used) Buildings (2) Community (1) Events (1) FOIA (1) Historic Preservation (1) Sanitation (1) Service Requests (1) Transportation (1)

Tell us more about your search experience Did you find what you were looking for

8 testers had difficulty finding datasets they wanted or said the ldquolanguagerdquo made it hard to find desired results 2 testers expected city services such as employment or 311 service requests 3 testers were still exploring the site generally trying to learn more about what is listed in certain categories 2 testers mentioned that the data appeared outdated while 2 other testers could not distinguish a rational order to the results

Miko (13) typed in ldquopublic transportation and received ldquoOpen Spacesrdquo results which were not relevant to her She mentioned how this site uses ldquoA certain type of lingo for certain type of peoplehellip not geared towards a regular Jane or Joerdquo

Did the search meet your expectations Out of the 7 testers who searched

Yes 29 (2) No 71 (5)

Out of the 10 testers who chose a category

Yes 30 (3) No 70 (7)

Why or why not

When expectations were not met a lot of the time it was because it took too long or too many steps for the tester to find their specific query Some testers again expected city services or

7

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal

Lhibbard (7) did not find what they were looking for and chose the service requests category and saidldquoI found the news about 311 service requests I thought I might be able to make a personal request directly to the aldermanrdquo Thrones (1) searched ldquonursingrdquo and it did not meet expectations as tester was looking to review human resources or employment information

For those that had expectations that were met testers were pleased to find and that the homepage provided more options and details

TJ (5) found what they were looking for and chose a data category They said they were ldquoLooking for libraries and found locationshoursrdquo Batman (14) searched for ldquopotholesrdquo and found what he expected ldquoCurious that they list recent pothole fixes Wanted to see potholes patched recently and got what tester expectedrdquo

Search for the building permits dataset How did you search Buildings Category gt Building Permits dataset 70 (12)

Search to find a specific datasethellip box 24 (4) Buildings Category gt Building Permits data lens 6 (1)

One tester Miko (13) first searched but the results did not load on their mobile device so the tester then clicked on the ldquoBuildingsrdquo category

Search term (if used) Building permits (4)

Anything else to note about this search experience Most testers agreed this search was simple and straightforward but GPBlight (4) suggested that in the data results there ldquoshould be an option for ward which is not on the maprdquo and Batman (14) thought it was ldquoInteresting that some things go back 2 years others 10 years Might be useful if data can be accessed further backrdquo

Next search for a map of crime data How did you search Public Safety Category gt Crimes map 35 (6) Public Safety Category gt Crimes - 2001 to present dataset 23 (4) Public Safety Category gt Crimes - 2001 to present data lens 18 (3)

8

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for a dataset youre interested in learning more about How did you search

Chose a data catalog category 59 (10) Search to find a specific dataset box 35 (6) Used search in navigation 6 (1)

Search term (if used) Animal control (1) CPS (1) Crime statistics for Lake View (1) Government (1) Lead testing schools (1) Nursing (1) Potholes (1)

Category (if used) Buildings (2) Community (1) Events (1) FOIA (1) Historic Preservation (1) Sanitation (1) Service Requests (1) Transportation (1)

Tell us more about your search experience Did you find what you were looking for

8 testers had difficulty finding datasets they wanted or said the ldquolanguagerdquo made it hard to find desired results 2 testers expected city services such as employment or 311 service requests 3 testers were still exploring the site generally trying to learn more about what is listed in certain categories 2 testers mentioned that the data appeared outdated while 2 other testers could not distinguish a rational order to the results

Miko (13) typed in ldquopublic transportation and received ldquoOpen Spacesrdquo results which were not relevant to her She mentioned how this site uses ldquoA certain type of lingo for certain type of peoplehellip not geared towards a regular Jane or Joerdquo

Did the search meet your expectations Out of the 7 testers who searched

Yes 29 (2) No 71 (5)

Out of the 10 testers who chose a category

Yes 30 (3) No 70 (7)

Why or why not

When expectations were not met a lot of the time it was because it took too long or too many steps for the tester to find their specific query Some testers again expected city services or

7

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal

Lhibbard (7) did not find what they were looking for and chose the service requests category and saidldquoI found the news about 311 service requests I thought I might be able to make a personal request directly to the aldermanrdquo Thrones (1) searched ldquonursingrdquo and it did not meet expectations as tester was looking to review human resources or employment information

For those that had expectations that were met testers were pleased to find and that the homepage provided more options and details

TJ (5) found what they were looking for and chose a data category They said they were ldquoLooking for libraries and found locationshoursrdquo Batman (14) searched for ldquopotholesrdquo and found what he expected ldquoCurious that they list recent pothole fixes Wanted to see potholes patched recently and got what tester expectedrdquo

Search for the building permits dataset How did you search Buildings Category gt Building Permits dataset 70 (12)

Search to find a specific datasethellip box 24 (4) Buildings Category gt Building Permits data lens 6 (1)

One tester Miko (13) first searched but the results did not load on their mobile device so the tester then clicked on the ldquoBuildingsrdquo category

Search term (if used) Building permits (4)

Anything else to note about this search experience Most testers agreed this search was simple and straightforward but GPBlight (4) suggested that in the data results there ldquoshould be an option for ward which is not on the maprdquo and Batman (14) thought it was ldquoInteresting that some things go back 2 years others 10 years Might be useful if data can be accessed further backrdquo

Next search for a map of crime data How did you search Public Safety Category gt Crimes map 35 (6) Public Safety Category gt Crimes - 2001 to present dataset 23 (4) Public Safety Category gt Crimes - 2001 to present data lens 18 (3)

8

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal

Lhibbard (7) did not find what they were looking for and chose the service requests category and saidldquoI found the news about 311 service requests I thought I might be able to make a personal request directly to the aldermanrdquo Thrones (1) searched ldquonursingrdquo and it did not meet expectations as tester was looking to review human resources or employment information

For those that had expectations that were met testers were pleased to find and that the homepage provided more options and details

TJ (5) found what they were looking for and chose a data category They said they were ldquoLooking for libraries and found locationshoursrdquo Batman (14) searched for ldquopotholesrdquo and found what he expected ldquoCurious that they list recent pothole fixes Wanted to see potholes patched recently and got what tester expectedrdquo

Search for the building permits dataset How did you search Buildings Category gt Building Permits dataset 70 (12)

Search to find a specific datasethellip box 24 (4) Buildings Category gt Building Permits data lens 6 (1)

One tester Miko (13) first searched but the results did not load on their mobile device so the tester then clicked on the ldquoBuildingsrdquo category

Search term (if used) Building permits (4)

Anything else to note about this search experience Most testers agreed this search was simple and straightforward but GPBlight (4) suggested that in the data results there ldquoshould be an option for ward which is not on the maprdquo and Batman (14) thought it was ldquoInteresting that some things go back 2 years others 10 years Might be useful if data can be accessed further backrdquo

Next search for a map of crime data How did you search Public Safety Category gt Crimes map 35 (6) Public Safety Category gt Crimes - 2001 to present dataset 23 (4) Public Safety Category gt Crimes - 2001 to present data lens 18 (3)

8

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

ldquoSearch to find a specific datasethelliprdquo box 18 (3) Chose Crimes - 2001 to present link on header image 6 (1)

Search term (if used) News (1) crime map (1) crime data (1)

Anything else to note about this search experience

6 of 17 testers had little to no difficulty finding the map however a couple of testers commented on the map functions and legend For example Gabireads (2) zoomed in near her home and said that itrsquos not clear what it is a map of nor the time period 7 testers knew that crime data would be listed under the ldquoPublic Safetyrdquo category but chose either the dataset or data lens result which appeared higher in the results list These testers therefore did not find the crime data map as tasked One tester Miko (13) reviewed the categories before typing in ldquoNewsrdquo into the search box This did not come up with relevant results and so she typed in ldquoCrime datardquo in the search box and then chose the data lens option

Next find the potholes that have been recently fixed How did you search ldquoService Requestsrdquo Category gt Potholes Patched Last 7 Days 76 (13) ldquoSearch to find a specific datasethelliprdquo 18 (3) Other ldquoSanitation Categoryrdquo then ldquoSearch to find a specific datasethelliprdquo 6 (1)

6 testers out of the 13 who clicked the ldquoService Requestsrdquo category attempted to find this data under another category first

Search term (if used) Potholes (1) Potholes fixed (1) Pot holes (1)

Anything else to note about this search experience

While almost all of the testers were able to find this dataset a lot of testers had difficulty choosing the correct category at first 6 testers expected that potholes dataset would be either in the Sanitation Environment Public safety or Transportation categories One tester Thornes (1) tried to find this information first in the ldquoSanitationrdquo category then in the ldquoPublic Safetyrdquo category before finally switching to the search functionality and searching for ldquopotholesrdquo

9

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next search for active business licenses How did you search

ldquoCommunity Category gt ldquoBusiness Licenses - Activerdquo dataset 65(11) ldquoSearch to find a specific datasethelliprdquo 35(6)

10 out of 11 testers who eventually found the active business licenses dataset explored other categories first 5 out of the 6 testers who used the search functionality had difficulties with other search terms

Search term (if used) Active business licenses (5) business licenses (1)

Anything else to note about this search experience

15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search 6 testers specifically tried the ldquoAdministration amp Financerdquo category first while 4 other testers suggested the ldquoBusinessrdquo category Brittsky (9) explored the ldquoBuildingsrdquo category before searching ldquoactive business licenserdquo Testers who searched had some difficulties with misspellings in their search terms M (15) first clicked on the ldquoAdministration amp Financerdquo category and then searched ldquolicenserdquo and could not find the correct data Searching ldquoBusiness licensesrdquo led to a successful search

Find the City of Chicagos community boundaries map How did you search

Facilities amp Geo Boundaries Category gt Boundaries - Community Areas (current) map 65 (11) ldquoSearch to find a specific datasethelliprdquo 29 (5) Other Chose a Digital Chicago article 6 (1)

4 out of the 11 testers who selected a category first chose a different category before choosing the ldquoFacilities amp Geo Boundariesrdquo one

Search term (if used) City of Chicago community boundaries map (1)community boundaries map (2) community boundaries (1) community areas (1)

Anything else to note about this search experience

4 testers expected the community boundaries map to be located under the ldquoCommunityrdquo category and not only the ldquoFacilities amp Geo Boundariesrdquo category TRobin312 (17) even pointed out that the hover over help text of the ldquoCommunityrdquo said ldquoboundariesrdquo would be found in this category The correct result however was not apparent in the many search results

10

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

One tester Miko (13) was viewing the site on her mobile device and clicked the ldquoDigital Chicagordquo map image for the article ldquoBuilding Safety Data on the Open Data Portalrdquo because this image appeared to contain a boundary map She tried to search but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience She also attempted searching in the ldquoParks and Recreationrdquo category The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses

Overall how easy was it for you to find these datasets 5 ndash Very easy 35 (6) 4 ndash Easy 35 (6) 3 ndash Neutral 24 (4) 2 ndash Difficult 6 (1) 1 ndash Very difficult 0

Tell us more about your search experiences and how you would improve the site While the majority of testers said it was ldquovery easyrdquo or ldquoeasyrdquo to find datasets there were clear improvements that should be made There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task 2 testers specifically told us improvements that were about city services and not open data

Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function On mobile devices we saw that filters were rarely used and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the ldquoSearch for a Specific Datasethelliprdquo box function

Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices A couple of testers suggested to make the search box bigger and a higher contrast

11

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next explore the Digital Chicago section on the homepage Tell us about the information you find 10 out of 17 testers clicked on the Array of Things article which is the first article under the ldquoDigital Chicagordquo section 4 testers exclaimed that they did not know what ldquoArray of Thingsrdquo means and did not find the descriptive text helpful When reviewing the ldquoDigital Chicago section 2 testers mentioned that that the ldquoMonth in Review July 2016rdquo article was out of date 2 testers tried to click on the ldquoDigital Chicagordquo header but it was not hyperlinked as expected 2 testers also mentioned that they do not know what ldquoDigital Chicagordquo means One tester who was viewing this on her mobile device did not think that ldquoDigital Chicagordquo was a section different from the rest of the page

Lhibbard (7) mentioned that this section ldquoseemed more friendly to city employees than city residentsrdquo

How relevant do you think the Digital Chicago section is to you

5 ndash Very relevant 24 (4) 4 ndash Relevant 24 (4) 3 ndash Neutral 24 (4) 2 ndash Not very relevant 12 (2) 1 ndash Not at all relevant 12 (2) Not applicable 6 (1)

Africa (16) chose a dataset versus reviewing the ldquoDigital Chicagordquo and therefore is listed as ldquonot applicablerdquo in this question

Why or why not Here is what some testers said when describing ldquoDigital Chicagordquo as ldquorelevantrdquo or ldquovery relevantrdquo (817 testers)

TJ (5) felt this sectionldquoProvides valuable info for the public dealing with buildingsetcrdquo Thrones (1) was content and that the section was relevant butldquoCould have more details Kind of shortrdquo

For those saying it was not relevant (417 testers) Lacked descriptive and concise content One user felt it might be more meaningful to city employees than residents Some also felt it might be an interesting read from time to time but not necessary to know

12

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

TRobin312 (17) claimed ldquo[I] donrsquot understand what information Im supposed to get in this sectionrdquo

Next explore the How to Videos section on the homepage Tell us about the information you find Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description 4 testers mentioned that they thought ldquoHow to Videosrdquo are a good way to provide instructions or information another tester would not get information in this way 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation amp Technologyrsquos YouTube page Upon viewing the videos in the playlist 3 testers mentioned that the videos were too long Funky Burrito (10) suggested breaking down the large videos into smaller modules Both Blessed (6) and Funky Burrito (10) were surprised to be directed to YouTube without any notifications after clicking on ldquoMore videosrdquo

How valuable do you think the How to Videos section is to you

5 ndash Very valuable 41 (7) 4 ndash Valuable 24 (4) 3 ndash Neutral 18 (3) 2 ndash Not very valuable 12 (2) 1 ndash Not at all valuable 6 (1)

Why or why not

Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviewstutorials of different subjects Gabireads (2) saidldquo Easy way to get an overview of somethinghellip good way to provide instruction and make robust userdquo

Not valuable Testers that found this section not valuable felt that the videos should be shortened 2 testers said there should have been a notification when leaving the main site for YouTube Blessed (6) also shared that they are ldquoNot interested in [viewing] a video to find one piece of informationrdquo

13

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Next review the Chicago Apps section on the homepage Tell us about the information you find Many testers (11) found out they can find info on when their streets would be plowedswept and could request 311 services which they felt was useful 1 tester on mobile felt ldquoChicago Appsrdquo was not distinctly separated from rest of page 2 testers tried to click on ldquoChicago appsrdquo header

Were you familiar with any of these apps If so which one(s)

15 testers were not familiar with the apps1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it

How likely are you to use these Chicago apps in the future

5 ndash Very likely 59 (10) 4 ndash Likely 175 (3) 3 ndash Neutral 175 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not

Most testers (13) said they would be likely to use these apps in the future as they are convenient faster than calling a city employee and overall useful to have as residents

VP (12) said he was unlikely to use this ldquoChicago appsrdquo section in the future He mentioned that therersquos ldquoNo need Just do a Google searchrdquo GPBlight (4) was neutral becauseldquoIn my neighborhood the data is seriously flawed which makes it inaccurate

Overall how easy or difficult did you think it is to use the new redesigned Data Portal homepage

5 ndash Very easy 41 (7) 4 ndash Easy 47 (8) 3 ndash Neutral 6(1) 2 ndash Difficult 6(1) 1 ndash Very difficult 0

What makes you say this

The majority of testers (15) found the new redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks Some did

14

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

however agree that the search ldquolanguagerdquo and function could be improved to make it give better results when trying to search Categorization could also be improved as many users expected to find one thing in one area and not another

After reviewing the site do you think you are the target audience for the Data Portal

Yes 76 (13) No 12 (2) Not sure 12 (2)

If not who is the target audience Blessed (6) also did not feel they were the target audience and claimed it was for ldquorich people M (15) was unsure and saidldquoBroad target audience At least some parts of the site are intuitive and do not require data science background but if you do have it easy to find datardquo

How likely are you to use the City of Chicagos Open Data Portal in your personal life

5 ndash Very likely 58 (10) 4 ndash Likely 18 (3) 3 ndash Neutral 18 (3) 2 ndash Not very likely 6 (1) 1 ndash Not at all likely 0

Why or why not Out of the 13 testers who are likely to use it in their personal life only 3 testers said they would use it specifically for data purposes 2 out of these 13 testers mentioned that they would use if because of the apps Additionally 2 testers said they would use it in their personal life because they are curious 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services Those who are neutral or unlikely (4) to use in personal life claimed they donrsquot use city services or can just use Google to access what they need

How likely are you to use the City of Chicagos Open Data Portal in your professional life

5 ndash Very likely 53(9)

15

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

4 ndash Likely 29(5) 3 ndash Neutral 12(2) 2 ndash Not very likely 6(1) 1 ndash Not at all likely 0

Why or why not

Many testers (14) are likely to use the data portal in their professional lives as they have a business work a career requiring use of this data such as a librarian or human resources coordinator or will use for educationalresearch purposes

Gabireads (2) is an academic librarian and saidldquoI work a lot with students who need access to data in a range of areasrdquo

Those who are neutralunlikely to use it professionally said the info and data available is nice but donrsquot have a real need for them in their lives or careers

Funky Burrito (10) saidldquoPerhaps not the datasets but the apps I am very likely to userdquo

[Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal From our background question we learned that 11 testers used the Open Data Portal in the past As we previously mentioned we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website

8 testers (73) reacted positively and said it is more user friendly due to the addition of categories and making search easier Testers also shared that the new homepage looks modern and updated 3 testers (27) felt neutral or indifferent about the redesigned portal and mentioned they only care about retrieving data and not design or that it only targets specific people such as people who recently move here or residentialbusiness developers

What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use more valuableimportant to you and more likely for you to use it again In this question 8 testers did not mention any additional improvements 3 testers mentioned to make the search bar more visible 2 testers wanted to see simpler language used and 2 testers reiterated that better categorization was needed 1 tester wanted to see the icon for the ldquoCommunityrdquo changed from a shopping cart to something more relevant Lastly another tester

16

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed All other improvements were shared when completing previous tasks or reviewing content

CUTGroup Review

Was this your first CUTGroup test Yes 41 (7) No 59 (10)

Do you like the CUTGroup Yes 100 (17) No 0

Did you like this CUTGroup test Yes 94 (16) No 6 (1)

Anything else to add about CUTGroup

ldquoLove it look forward to more surveys that are better to the city No matter what age you arerdquo -Blessed (6) ldquoLearned more with this test Thanks for coming to the Southsiderdquo -Brittsky (9) GPBlight (4) asked ldquoWhen are they going to make the data itself user-friendly not for those who use data professionallyrdquo Gabireads (2) saysldquoIt was fantastic Has to remember to keep looking at the site to get to play with these products and sharing them with other people

17

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Results

Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents but many expected access to city services At the end of this CUTGroup test we asked testers if they felt that they are the target audience for the new redesigned Open Data Portal and 13 testers (76) said ldquoyesrdquo Our testers included individuals who had no little or some familiarity with the current data portal and 14 testers mentioned that this site appeared to be designed for the general public and residents 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness 1 tester mentioned it was for people interested in data analysis Another tester said it was built for business owners while another said it was built for developers One concern is that testers even after review of the website still expected more access to city services and resources that could be found on the official City of Chicago website There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website

Identify access points to city services or action steps for residents There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage A second more complex level is at the category level For instance if a user clicks on the ldquoEventsrdquo category there could be a link to the City of Chicago eventsrsquo page before users even begin to explore datasets This level of access would require reviewing all of the categories and understanding if users associate city services with those category types The most complex level is at the dataset level Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information this would require additional user feedback The suggestion is to connect relevant datasets to the appropriate city services Therefore if a user is reviewing data about potholes as one example there could be a link to make a pothole service request

Utilize the header image to clarify purpose about the site ldquoWhatrsquos this sales thing It makes it look like itrsquos advertising stuffrdquo GPBlight (4) said in response to reviewing the header images When the homepage first loaded a bug was

18

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

immediately noticeable because the header images would stack and cause the page to jump upon load While that is a detectable fix we identified a larger opportunity to share information about what the website does Very few testers reacted to the initial ldquodata salerdquo image and those who did react had negative responses The first header image before rotating to either the crime or Divvy data should define the purpose to the site while being as clear and direct as possible The current ldquodata salerdquo language and images confuse some users

Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task We asked testers to search for six different datasets one was open-ended while the rest proctors requested that they find specific datasets building permits map of crime data recently fixed potholes active business licenses and Chicagorsquos community boundaries map We were interested to see how testers would search and find the datasets and what issues they came across during the process While 12 testers (70) indicated that finding these datasets was ldquoeasyrdquo or ldquovery easyrdquo we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant but not requested dataset

Categories were often chosen over search bar but technical bugs and design influenced those choices Only 1 tester out of 17 (6) used the search bar consistently for all of the dataset search tasks whereas 7 testers (41) used categories to complete all dataset search tasks The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find On mobile devices the magnifying glass of the search bar was not working properly Therefore if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results When doing the initial review of the page only 1 tester mentioned the search bar which indicates that it is not a prominent part of the page When asked about what actions they knew they could do from reviewing their page only 2 testers mentioned the search bar When asked to take their first action 3 testers (18) searched whereas 10 (59) clicked on a data catalog category We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes Nonetheless we realized that the search bar was not very prominent for testers Suggested improvements included larger font higher contrast and a more prominent location on the page

19

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible While testers knew where to find building permit and crime data finding recently patched potholes active business licenses and community boundaries was more challenging In one example 10 out of 11 testers who found the active business licenses dataset had explored other categories first Testers expected to find the potholes data in these categories Public Safety Environment and Sanitation Testers expected to find active business licenses in Administration and Finance Lastly testers looking for community boundaries looked in the Community category Some testers did use the help text on the categories to make decisions about which category to choose but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category Testers did not use the filters on the search results page and upon reviewing a few datasets without identifying a relevant one would often rely on search or finding another category Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly We also witnessed that testers did not always distinguish between the type of datandash whether it was a dataset data lens filtered view or map they were choosing This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list A primary search method should be defined for the user Currently users can choose a category or search but both have their challenges Outlined above are results and improvements for the categories since that was a prominent way testers searched For each task 11 or more testers used the categories in completing their search If search was more prominent on the page and functioned better on mobile we could identify if search was a better way for testers to complete these tasks We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience The bug on mobile also caused a stopping point that forced testers to use categories If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources

20

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

Testers liked the ldquoDigital Chicagordquo amp ldquoHow to Videosrdquo resources on the homepage but the ldquoChicago appsrdquo were seen as being the most relevant Testers reviewed the other sections of the homepage including the ldquoDigital Chicagordquo section that showcases recent articles by the City of Chicagorsquos Department of Innovation and Technology ldquoHow to Videosrdquo that showcase how to use the data portal and other tools and ldquoChicago appsrdquo tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker) 48 of testers thought the ldquoDigital Chicagordquo section was ldquorelevantrdquo or ldquovery relevantrdquo but testers felt that ldquoDigital Chicagordquo did not mean much to them (calling this section ldquonewsrdquo could be an improvement) they had questions about the articles and what they meant and this section lacked descriptive content for residents 66 of testers thought the ldquoHow to Videosrdquo section was ldquovaluablerdquo or ldquovery valuablerdquo and a lot of testers liked receiving information via video although there was a consensus that they should be shortened and users should know that they will be directed to YouTube 765 of testers were likely to use the apps under the ldquoChicago appsrdquo section because they saw them as useful to have for residents 15 of our 17 (88) testers were not familiar with these apps prior to visiting the website The ldquoChicago appsrdquo section met expectations from testers who at the beginning searched for or were interested in city services There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future For the resident user we would suggest placing the ldquoChicago appsrdquo section higher on the page because it relates with their needs To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website adding a descriptive sentence about what ldquoChicago appsrdquo is and why itrsquos on this page would be valuable Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives

Conclusion Most testers found the new redesigned Open Data Portal as an accessible resource in finding information about Chicago With different levels of familiarity to the current data portal testers were able to explore datasets and provide feedback to improve the user experience of searching for data Resources like the ldquoDigital Chicagordquo ldquoHow to Videosrdquo and ldquoChicago appsrdquo is a welcomed addition for residents who thought these resources are valuable and relevant to their needs Implementing the recommendations we learned from this CUTGroup test and

21

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

CUTGroup 28 - Chicago Open Data Portal In-person test on September 28 2016

continuing to include residents in the process of building the Open Data Portal will ensure a continued user-centric website

22

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

Make $20 at a test of the open data portal

Hi ltltFirst Namegtgt Weve got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a new redesigned version of the City of Chicagos Open DataPortal homepage Are you available for a 30 shy 45 minute inshyperson test in Chicago on WednesdaySeptember 28 between 400 shy 730 PM If so please complete this form We are in the process of gathering responses so we will be in touch in a couple ofdays to check availability confirm the location and confirm that you are right for thistest We are looking for about 25 testers For your participation you will receive a $20 VISA gift card Youll also help make bettersoftware for Cook County Thanks for being a member of the CUTGroup As usual callor write with questions Remember if you want to be a part of this test please complete this form

shyshy

Sonja Marziano(312) 565shy3274 smarzianocctorg

Subscribe Share Past Issues Translate

Facebook Twitter Website

Creative Commons Attribution 30 Unported License 2016 Smart Chicago Collaborative unsubscribe from this list update subscription preferences

Subscribe Share Past Issues Translate

CUTGroup Test Data Portal RedesignIf you would like to participate in the next CUTGroup test please complete this form We will follow-up to check youravailability and let you know if you qualify for this test Smart Chicago values your online privacy and is committed to safeguarding the data you share with us and youronline privacy Our On-Line Privacy Policy applies to data collected at wwwsmartchicago2012wufoocom domain

Email

Are you available for an in-person test in Chicago on Wednesday

September 28 for 30-45 minutes between 400 and 730 PM

Yes

No

How familiar are you with the City of Chicagos open data portal

5 - Very familiar

4 - Familiar

3 - Neutral

2 - Not very familiar

1 - Not at all familiar

Have you ever used the City of Chicagos open data portal before

Yes

No

Do you work with or analyze datasets in your personal or professional

life

Yes

No

How familiar are you with the City of Chicagos data and technology

programs and initiatives

5 - Very familiar

4 - Familiar

3 - Neutral

2 - Not very familiar

1 - Not at all familiar

Tell us more about the programs you are familiar with

CUTGroup 28 Data Portal RedesignThanks for participating in our test of the City of Chicago Data Portal redesigned homepage

About youFirst some preliminaries Keep in mind that we are going to publish the results of this test but will not associate your name oremail address with the results

Background

First Last

Name

Email

Give yourself a tester profile name

Proctored by

What device are you using for this test

What browser are you using right now

Chrome

Firefox

Internet Explorer

Safari

Other

What successes or challenges have you had finding data that you were

looking for or using the open data portal

Tell us about any of your past experiences

What are your expectations of what the City of Chicago Open Data Portal

would be useful for

Initial homepage review

Review the homepage and tell us about your first impressions

While you review try not to click on the next page just yet but feel free to

scroll up and down the page

What are three actions that you know you can do just from reviewing this

first page

Who do you think this website is designed for Why

After reviewing the homepage what is the first thing you want to do or

click

Search to find a specific dataset

Clicked on a data catalog category

Clicked on a Digital Chicago topic

Clicked on How to Videos link

Clicked on a Chicago App

Clicked Developer Docs in navigation bar

Clicked Feedback in navigation bar

Clicked Data Catalog in navigation bar

Other

What did you search Or what was the category you chose

Tell us more about the first thing you did and why you did it What were

you expecting to happen

Website review

Next search for a dataset youre interested in learning more about How

did you search

Search to find a specific dataset box

Chose a data catalog category

Used search in navigation bar

Other

Search term (if used)

Tell us more about your search experience Did you find what you were

looking for

Did the search meet your expectations

Yes

No

Why or why not

Search for the building permits dataset How did you search

Search to find a specific dataset box

Buildings Category gt Building Permits dataset

Buildings Category gt Building Permits data lens

Data Catalog in navigation bar gt Search

Data Catalog in navigation bar gt Buildings Category

Search box in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next search for a map of crime data How did you search

Search to find a specific dataset box

Public Safety Category gt Crimes - 2001 to present data lens

Public Safety Category gt Crimes - 2001 to present dataset

Public Safety Category gt Crimes map

Chose Crimes - 2001 to present link on header image

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next find the potholes that have been recently fixed How did you search

Search to find a specific dataset box

Service Requests Category gt Potholes Patched Last SevenDays

Service Requests Category gt 311 Service Requests - Pot HolesRequested dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next search for active business licenses How did you search

Search to find a specific dataset box

Community Category gt Business Licenses - Active FilteredView

Community Category gt Business Licenses Dataset

Community Category gt Business Licenses Dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Find the City of Chicagos community boundaries map How did you

search

Search to find a specific dataset box

Facilities amp Geo Boundaries Category gt Boundaries -Community Areas (current) map

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Tell us more about your search experiences and how you would improve

the site

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Next explore the Digital Chicago section on the homepage Tell us about

the information you find

How relevant do you think the Digital Chicago section is to you

5 - Very relevant

4 - Relevant

3 - Neutral

2 - Not very relevant

1 - Not at all relevant

Why or why not

Next explore the How to Videos section on the homepage Tell us about

the information you find

How valuable do you think the How to Videos section is to you

5 - Very valuable

4 - Valuable

3 - Neutral

2 - Not very valuable

1 - Not at all valuable

Why or why not

Next review the Chicago Apps section on the homepage Tell us about

the information you find

Overall Review

Were you familiar with any of these apps If so which one(s)

How likely are you to use these Chicago apps in the future

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

Overall how easy or difficult did you think it is to use the new redesigned

Data Portal homepage

5 - Very easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very difficult

What makes you say this

After reviewing the site do you think you are the target audience for the

Data Portal

Yes

No

Not sure

If not who is the target audience

How likely are you to use the City of Chicagos Open Data Portal in your

personal life

5 - Very likely

4 - Likely

CUTGroup Review

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

How likely are you to use the City of Chicagos Open Data Portal in your

professional life

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

[Only for testers who used the data portal in the past]

How does this redesigned data portal homepage compare to what you

remember of the current portal

What improvements would you make to City of Chicago Open Data portal

homepage to make this site easier to use more valuableimportant to

you and more likely for you to use it again

Was this your first CUTGroup test

Yes

No

Do you like the CUTGroup

Yes

No

Did you like this CUTGroup test

Yes

Yes

No

Anything else to add about CUTGroup

CUTGroup 28 Data Portal RedesignThanks for participating in our test of the City of Chicago Data Portal redesigned homepage

About youFirst some preliminaries Keep in mind that we are going to publish the results of this test but will not associate your name oremail address with the results

Background

First Last

Name

Email

Give yourself a tester profile name

Proctored by

What device are you using for this test

What browser are you using right now

Chrome

Firefox

Internet Explorer

Safari

Other

What successes or challenges have you had finding data that you were

looking for or using the open data portal

Tell us about any of your past experiences

What are your expectations of what the City of Chicago Open Data Portal

would be useful for

Initial homepage review

Review the homepage and tell us about your first impressions

While you review try not to click on the next page just yet but feel free to

scroll up and down the page

What are three actions that you know you can do just from reviewing this

first page

Who do you think this website is designed for Why

After reviewing the homepage what is the first thing you want to do or

click

Search to find a specific dataset

Clicked on a data catalog category

Clicked on a Digital Chicago topic

Clicked on How to Videos link

Clicked on a Chicago App

Clicked Developer Docs in navigation bar

Clicked Feedback in navigation bar

Clicked Data Catalog in navigation bar

Other

What did you search Or what was the category you chose

Tell us more about the first thing you did and why you did it What were

you expecting to happen

Website review

Next search for a dataset youre interested in learning more about How

did you search

Search to find a specific dataset box

Chose a data catalog category

Used search in navigation bar

Other

Search term (if used)

Tell us more about your search experience Did you find what you were

looking for

Did the search meet your expectations

Yes

No

Why or why not

Search for the building permits dataset How did you search

Search to find a specific dataset box

Buildings Category gt Building Permits dataset

Buildings Category gt Building Permits data lens

Data Catalog in navigation bar gt Search

Data Catalog in navigation bar gt Buildings Category

Search box in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next search for a map of crime data How did you search

Search to find a specific dataset box

Public Safety Category gt Crimes - 2001 to present data lens

Public Safety Category gt Crimes - 2001 to present dataset

Public Safety Category gt Crimes map

Chose Crimes - 2001 to present link on header image

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next find the potholes that have been recently fixed How did you search

Search to find a specific dataset box

Service Requests Category gt Potholes Patched Last SevenDays

Service Requests Category gt 311 Service Requests - Pot HolesRequested dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next search for active business licenses How did you search

Search to find a specific dataset box

Community Category gt Business Licenses - Active FilteredView

Community Category gt Business Licenses Dataset

Community Category gt Business Licenses Dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Find the City of Chicagos community boundaries map How did you

search

Search to find a specific dataset box

Facilities amp Geo Boundaries Category gt Boundaries -Community Areas (current) map

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Tell us more about your search experiences and how you would improve

the site

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Next explore the Digital Chicago section on the homepage Tell us about

the information you find

How relevant do you think the Digital Chicago section is to you

5 - Very relevant

4 - Relevant

3 - Neutral

2 - Not very relevant

1 - Not at all relevant

Why or why not

Next explore the How to Videos section on the homepage Tell us about

the information you find

How valuable do you think the How to Videos section is to you

5 - Very valuable

4 - Valuable

3 - Neutral

2 - Not very valuable

1 - Not at all valuable

Why or why not

Next review the Chicago Apps section on the homepage Tell us about

the information you find

Overall Review

Were you familiar with any of these apps If so which one(s)

How likely are you to use these Chicago apps in the future

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

Overall how easy or difficult did you think it is to use the new redesigned

Data Portal homepage

5 - Very easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very difficult

What makes you say this

After reviewing the site do you think you are the target audience for the

Data Portal

Yes

No

Not sure

If not who is the target audience

How likely are you to use the City of Chicagos Open Data Portal in your

personal life

5 - Very likely

4 - Likely

CUTGroup Review

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

How likely are you to use the City of Chicagos Open Data Portal in your

professional life

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

[Only for testers who used the data portal in the past]

How does this redesigned data portal homepage compare to what you

remember of the current portal

What improvements would you make to City of Chicago Open Data portal

homepage to make this site easier to use more valuableimportant to

you and more likely for you to use it again

Was this your first CUTGroup test

Yes

No

Do you like the CUTGroup

Yes

No

Did you like this CUTGroup test

Yes

Yes

No

Anything else to add about CUTGroup

Initial homepage review

Review the homepage and tell us about your first impressions

While you review try not to click on the next page just yet but feel free to

scroll up and down the page

What are three actions that you know you can do just from reviewing this

first page

Who do you think this website is designed for Why

After reviewing the homepage what is the first thing you want to do or

click

Search to find a specific dataset

Clicked on a data catalog category

Clicked on a Digital Chicago topic

Clicked on How to Videos link

Clicked on a Chicago App

Clicked Developer Docs in navigation bar

Clicked Feedback in navigation bar

Clicked Data Catalog in navigation bar

Other

What did you search Or what was the category you chose

Tell us more about the first thing you did and why you did it What were

you expecting to happen

Website review

Next search for a dataset youre interested in learning more about How

did you search

Search to find a specific dataset box

Chose a data catalog category

Used search in navigation bar

Other

Search term (if used)

Tell us more about your search experience Did you find what you were

looking for

Did the search meet your expectations

Yes

No

Why or why not

Search for the building permits dataset How did you search

Search to find a specific dataset box

Buildings Category gt Building Permits dataset

Buildings Category gt Building Permits data lens

Data Catalog in navigation bar gt Search

Data Catalog in navigation bar gt Buildings Category

Search box in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next search for a map of crime data How did you search

Search to find a specific dataset box

Public Safety Category gt Crimes - 2001 to present data lens

Public Safety Category gt Crimes - 2001 to present dataset

Public Safety Category gt Crimes map

Chose Crimes - 2001 to present link on header image

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next find the potholes that have been recently fixed How did you search

Search to find a specific dataset box

Service Requests Category gt Potholes Patched Last SevenDays

Service Requests Category gt 311 Service Requests - Pot HolesRequested dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next search for active business licenses How did you search

Search to find a specific dataset box

Community Category gt Business Licenses - Active FilteredView

Community Category gt Business Licenses Dataset

Community Category gt Business Licenses Dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Find the City of Chicagos community boundaries map How did you

search

Search to find a specific dataset box

Facilities amp Geo Boundaries Category gt Boundaries -Community Areas (current) map

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Tell us more about your search experiences and how you would improve

the site

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Next explore the Digital Chicago section on the homepage Tell us about

the information you find

How relevant do you think the Digital Chicago section is to you

5 - Very relevant

4 - Relevant

3 - Neutral

2 - Not very relevant

1 - Not at all relevant

Why or why not

Next explore the How to Videos section on the homepage Tell us about

the information you find

How valuable do you think the How to Videos section is to you

5 - Very valuable

4 - Valuable

3 - Neutral

2 - Not very valuable

1 - Not at all valuable

Why or why not

Next review the Chicago Apps section on the homepage Tell us about

the information you find

Overall Review

Were you familiar with any of these apps If so which one(s)

How likely are you to use these Chicago apps in the future

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

Overall how easy or difficult did you think it is to use the new redesigned

Data Portal homepage

5 - Very easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very difficult

What makes you say this

After reviewing the site do you think you are the target audience for the

Data Portal

Yes

No

Not sure

If not who is the target audience

How likely are you to use the City of Chicagos Open Data Portal in your

personal life

5 - Very likely

4 - Likely

CUTGroup Review

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

How likely are you to use the City of Chicagos Open Data Portal in your

professional life

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

[Only for testers who used the data portal in the past]

How does this redesigned data portal homepage compare to what you

remember of the current portal

What improvements would you make to City of Chicago Open Data portal

homepage to make this site easier to use more valuableimportant to

you and more likely for you to use it again

Was this your first CUTGroup test

Yes

No

Do you like the CUTGroup

Yes

No

Did you like this CUTGroup test

Yes

Yes

No

Anything else to add about CUTGroup

Website review

Next search for a dataset youre interested in learning more about How

did you search

Search to find a specific dataset box

Chose a data catalog category

Used search in navigation bar

Other

Search term (if used)

Tell us more about your search experience Did you find what you were

looking for

Did the search meet your expectations

Yes

No

Why or why not

Search for the building permits dataset How did you search

Search to find a specific dataset box

Buildings Category gt Building Permits dataset

Buildings Category gt Building Permits data lens

Data Catalog in navigation bar gt Search

Data Catalog in navigation bar gt Buildings Category

Search box in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next search for a map of crime data How did you search

Search to find a specific dataset box

Public Safety Category gt Crimes - 2001 to present data lens

Public Safety Category gt Crimes - 2001 to present dataset

Public Safety Category gt Crimes map

Chose Crimes - 2001 to present link on header image

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next find the potholes that have been recently fixed How did you search

Search to find a specific dataset box

Service Requests Category gt Potholes Patched Last SevenDays

Service Requests Category gt 311 Service Requests - Pot HolesRequested dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next search for active business licenses How did you search

Search to find a specific dataset box

Community Category gt Business Licenses - Active FilteredView

Community Category gt Business Licenses Dataset

Community Category gt Business Licenses Dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Find the City of Chicagos community boundaries map How did you

search

Search to find a specific dataset box

Facilities amp Geo Boundaries Category gt Boundaries -Community Areas (current) map

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Tell us more about your search experiences and how you would improve

the site

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Next explore the Digital Chicago section on the homepage Tell us about

the information you find

How relevant do you think the Digital Chicago section is to you

5 - Very relevant

4 - Relevant

3 - Neutral

2 - Not very relevant

1 - Not at all relevant

Why or why not

Next explore the How to Videos section on the homepage Tell us about

the information you find

How valuable do you think the How to Videos section is to you

5 - Very valuable

4 - Valuable

3 - Neutral

2 - Not very valuable

1 - Not at all valuable

Why or why not

Next review the Chicago Apps section on the homepage Tell us about

the information you find

Overall Review

Were you familiar with any of these apps If so which one(s)

How likely are you to use these Chicago apps in the future

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

Overall how easy or difficult did you think it is to use the new redesigned

Data Portal homepage

5 - Very easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very difficult

What makes you say this

After reviewing the site do you think you are the target audience for the

Data Portal

Yes

No

Not sure

If not who is the target audience

How likely are you to use the City of Chicagos Open Data Portal in your

personal life

5 - Very likely

4 - Likely

CUTGroup Review

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

How likely are you to use the City of Chicagos Open Data Portal in your

professional life

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

[Only for testers who used the data portal in the past]

How does this redesigned data portal homepage compare to what you

remember of the current portal

What improvements would you make to City of Chicago Open Data portal

homepage to make this site easier to use more valuableimportant to

you and more likely for you to use it again

Was this your first CUTGroup test

Yes

No

Do you like the CUTGroup

Yes

No

Did you like this CUTGroup test

Yes

Yes

No

Anything else to add about CUTGroup

Search term (if used)

Anything else to note about this search experience

Next search for a map of crime data How did you search

Search to find a specific dataset box

Public Safety Category gt Crimes - 2001 to present data lens

Public Safety Category gt Crimes - 2001 to present dataset

Public Safety Category gt Crimes map

Chose Crimes - 2001 to present link on header image

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next find the potholes that have been recently fixed How did you search

Search to find a specific dataset box

Service Requests Category gt Potholes Patched Last SevenDays

Service Requests Category gt 311 Service Requests - Pot HolesRequested dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Next search for active business licenses How did you search

Search to find a specific dataset box

Community Category gt Business Licenses - Active FilteredView

Community Category gt Business Licenses Dataset

Community Category gt Business Licenses Dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Find the City of Chicagos community boundaries map How did you

search

Search to find a specific dataset box

Facilities amp Geo Boundaries Category gt Boundaries -Community Areas (current) map

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Tell us more about your search experiences and how you would improve

the site

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Next explore the Digital Chicago section on the homepage Tell us about

the information you find

How relevant do you think the Digital Chicago section is to you

5 - Very relevant

4 - Relevant

3 - Neutral

2 - Not very relevant

1 - Not at all relevant

Why or why not

Next explore the How to Videos section on the homepage Tell us about

the information you find

How valuable do you think the How to Videos section is to you

5 - Very valuable

4 - Valuable

3 - Neutral

2 - Not very valuable

1 - Not at all valuable

Why or why not

Next review the Chicago Apps section on the homepage Tell us about

the information you find

Overall Review

Were you familiar with any of these apps If so which one(s)

How likely are you to use these Chicago apps in the future

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

Overall how easy or difficult did you think it is to use the new redesigned

Data Portal homepage

5 - Very easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very difficult

What makes you say this

After reviewing the site do you think you are the target audience for the

Data Portal

Yes

No

Not sure

If not who is the target audience

How likely are you to use the City of Chicagos Open Data Portal in your

personal life

5 - Very likely

4 - Likely

CUTGroup Review

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

How likely are you to use the City of Chicagos Open Data Portal in your

professional life

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

[Only for testers who used the data portal in the past]

How does this redesigned data portal homepage compare to what you

remember of the current portal

What improvements would you make to City of Chicago Open Data portal

homepage to make this site easier to use more valuableimportant to

you and more likely for you to use it again

Was this your first CUTGroup test

Yes

No

Do you like the CUTGroup

Yes

No

Did you like this CUTGroup test

Yes

Yes

No

Anything else to add about CUTGroup

Community Category gt Business Licenses Dataset

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Find the City of Chicagos community boundaries map How did you

search

Search to find a specific dataset box

Facilities amp Geo Boundaries Category gt Boundaries -Community Areas (current) map

Search in navigation bar

Other

Search term (if used)

Anything else to note about this search experience

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Tell us more about your search experiences and how you would improve

the site

Overall how easy was it for you to find these datasets

5 - Very Easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very Difficult

Next explore the Digital Chicago section on the homepage Tell us about

the information you find

How relevant do you think the Digital Chicago section is to you

5 - Very relevant

4 - Relevant

3 - Neutral

2 - Not very relevant

1 - Not at all relevant

Why or why not

Next explore the How to Videos section on the homepage Tell us about

the information you find

How valuable do you think the How to Videos section is to you

5 - Very valuable

4 - Valuable

3 - Neutral

2 - Not very valuable

1 - Not at all valuable

Why or why not

Next review the Chicago Apps section on the homepage Tell us about

the information you find

Overall Review

Were you familiar with any of these apps If so which one(s)

How likely are you to use these Chicago apps in the future

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

Overall how easy or difficult did you think it is to use the new redesigned

Data Portal homepage

5 - Very easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very difficult

What makes you say this

After reviewing the site do you think you are the target audience for the

Data Portal

Yes

No

Not sure

If not who is the target audience

How likely are you to use the City of Chicagos Open Data Portal in your

personal life

5 - Very likely

4 - Likely

CUTGroup Review

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

How likely are you to use the City of Chicagos Open Data Portal in your

professional life

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

[Only for testers who used the data portal in the past]

How does this redesigned data portal homepage compare to what you

remember of the current portal

What improvements would you make to City of Chicago Open Data portal

homepage to make this site easier to use more valuableimportant to

you and more likely for you to use it again

Was this your first CUTGroup test

Yes

No

Do you like the CUTGroup

Yes

No

Did you like this CUTGroup test

Yes

Yes

No

Anything else to add about CUTGroup

3 - Neutral

2 - Difficult

1 - Very Difficult

Next explore the Digital Chicago section on the homepage Tell us about

the information you find

How relevant do you think the Digital Chicago section is to you

5 - Very relevant

4 - Relevant

3 - Neutral

2 - Not very relevant

1 - Not at all relevant

Why or why not

Next explore the How to Videos section on the homepage Tell us about

the information you find

How valuable do you think the How to Videos section is to you

5 - Very valuable

4 - Valuable

3 - Neutral

2 - Not very valuable

1 - Not at all valuable

Why or why not

Next review the Chicago Apps section on the homepage Tell us about

the information you find

Overall Review

Were you familiar with any of these apps If so which one(s)

How likely are you to use these Chicago apps in the future

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

Overall how easy or difficult did you think it is to use the new redesigned

Data Portal homepage

5 - Very easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very difficult

What makes you say this

After reviewing the site do you think you are the target audience for the

Data Portal

Yes

No

Not sure

If not who is the target audience

How likely are you to use the City of Chicagos Open Data Portal in your

personal life

5 - Very likely

4 - Likely

CUTGroup Review

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

How likely are you to use the City of Chicagos Open Data Portal in your

professional life

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

[Only for testers who used the data portal in the past]

How does this redesigned data portal homepage compare to what you

remember of the current portal

What improvements would you make to City of Chicago Open Data portal

homepage to make this site easier to use more valuableimportant to

you and more likely for you to use it again

Was this your first CUTGroup test

Yes

No

Do you like the CUTGroup

Yes

No

Did you like this CUTGroup test

Yes

Yes

No

Anything else to add about CUTGroup

Overall Review

Were you familiar with any of these apps If so which one(s)

How likely are you to use these Chicago apps in the future

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

Overall how easy or difficult did you think it is to use the new redesigned

Data Portal homepage

5 - Very easy

4 - Easy

3 - Neutral

2 - Difficult

1 - Very difficult

What makes you say this

After reviewing the site do you think you are the target audience for the

Data Portal

Yes

No

Not sure

If not who is the target audience

How likely are you to use the City of Chicagos Open Data Portal in your

personal life

5 - Very likely

4 - Likely

CUTGroup Review

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

How likely are you to use the City of Chicagos Open Data Portal in your

professional life

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

[Only for testers who used the data portal in the past]

How does this redesigned data portal homepage compare to what you

remember of the current portal

What improvements would you make to City of Chicago Open Data portal

homepage to make this site easier to use more valuableimportant to

you and more likely for you to use it again

Was this your first CUTGroup test

Yes

No

Do you like the CUTGroup

Yes

No

Did you like this CUTGroup test

Yes

Yes

No

Anything else to add about CUTGroup

CUTGroup Review

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

How likely are you to use the City of Chicagos Open Data Portal in your

professional life

5 - Very likely

4 - Likely

3 - Neutral

2 - Not very likely

1 - Not at all likely

Why or why not

[Only for testers who used the data portal in the past]

How does this redesigned data portal homepage compare to what you

remember of the current portal

What improvements would you make to City of Chicago Open Data portal

homepage to make this site easier to use more valuableimportant to

you and more likely for you to use it again

Was this your first CUTGroup test

Yes

No

Do you like the CUTGroup

Yes

No

Did you like this CUTGroup test

Yes

Yes

No

Anything else to add about CUTGroup

Yes

No

Anything else to add about CUTGroup