Measuring Success for Federal Websites
How The Office of Extramural Research Utilized ForeSee to Improve User Experience
Branch Chief Web Development and Technology
Office of Extramural ResearchNational Institutes of HealthTook over Branch 4 years ago.
Has worked on other Federal websites like Army.mil and the VA.
Has consulted with Non-Profits and Associations like SHRM and ASAE.
NIH Office of Extramural Research (OER)OER provides the corporate framework for NIH research administration, ensuring scientific integrity, public accountability, and effective stewardship of the NIH extramural research portfolio.
Three Year Modernization2012 - Cold Fusion no CMS
2013-14 Updated Design & Drupal CMS
Describe the three year modernization plan4
2015-16 Foresee Survey Input and Usability Audit
Describe the process with Foresee5
Improved User Interface2012 search for funding
2016 search for funding
UI improvements 6
What Was our Issue?
Investing time and effort into a website while guessing at where time was best spent is not efficient. Knowing what you want your outcome to be from your website makes it easy to measure performance
E-Commerce Websites Make it Easy
ecommerce websites make this easy because revenue is the goal at the end of the day
Non-profit and Member Based.
Non-profit websites can be a bit more complicated but with a bit of digging you can figure out desired outcomes - volunteers, petitions, donations, certifications, membership, etc.
Federal sites well now it becomes interesting since for the most part it is non-transactional
Watching Google Analytics
I have been asked through out my career to provide measurements of success for federal websites to measure our site success and all we can do is provide analytics that for the most part are really just for entertainment and curiosity.
Our First StepWe needed to set a goal.GOAL: To be the main portal for all official information about NIH Extramural Grants and have that information easy to find.
The first step is understanding the goal of your website12
Translate that Goal into Measureable IndicatorsAre our customers finding the information they are looking for?Are they satisfied with their experience?Are our priorities aligned with their interests?Are we missing information that they need from us?
Develop measurable indicators that can support your goal.
1. Are Our Customers Finding the Information They are Looking For?Can you really derive this information from analytics?
Can you really derive this information from analytics? NOYou can try but it really is never a clear answer.The best way to do this is by asking your customer directly when they leave your site so then you have a true answer
Ask Your Customers Directly!how well the features on the site help you find the information you need.
This answer should be your first KPI for success
Monthly KPIDid you find the information you were looking for today?Did you find the information you were looking for today?
31%NO69%YESYesNoOverallResponses64299369%31%Information Browsing683658Look and Feel776874Navigation694361Site Information734564Site Performance878084Satisfaction684260Primary Resource816877Recommend775570Return878185
We can now begin to track on a monthly and annual basis how we are performing.16
Show how the data can be looked into further with open ended survey and also paired up with Foresee usability experience to develop implementation
As you can see this was an overwhelming page with a ton of links. We simplified the approach on our new site.
2. Are They Satisfied with Their Experience?
This is again cannot be derived directly from analytics.
Survey Driven KPI.What is your overall satisfaction with this site?
This answer should be your first KPI for success
Setting Priorities Based on User Satisfaction.Priority MapStatus Quo RequiredMaintain or ImproveMonitor
Top PriorityImpactScoreSite Information69, 1.3
Site Performance85, 0.0
Navigation64, 1.8Look & Feel71, 0.9
Information Browsing61, 1.6
Prioritizing where your effort is best spent to increase your performance21
Understanding what the issues in with those specific areas are22
3. Are Our Priorities Aligned with Their Interests?
This my friends can be done with analytics. you simply compare the % of visits to a specific area of your site to the % of content you have in that area.
Digging Deeper: Utilize Custom Questions to Drill into Key Segments
Visitor expectations often vary based on their role or reason to visit the website.
Digging deeper is easier with Foresee custom questions you can now begin to drill down to specific percentages based on self identification of the user group or satisfaction based on what they decided to visit the site.25
We brought areas of interest for individual audience members26
4. Are We Missing Information that They Need From us?
With search analytics we are able to see where our results are not matching up with what the users are looking for. Is this do to content placement? lack of content? Or search engine optimization.28
We can look deeper into areas there are issues and come to conclusion on how we make the content they are searching for present itself better in search results.29
We begin to insert best bets based on this knowledge or add content to pages where they are looking30
So After all this What Did we Not Look at to Judge Success?
Visits, Hits, Unique Visitors, Page View, and Time on Site!
Visits, hits, unique visitors, page view, time on site!all the info that people love to look at for entertainment purposes but have no real true value in measuring success.