EOSDIS Survey Overview
[email protected] and HDF-EOS WorkshopNov. 4, 2009
Why we survey
NASA’s Earth Observing System Data and Information System provides data products and associated services for interdisciplinary studies to a diverse user community.
Beginning in 2004, the Earth Science Data and Information System (ESDIS) Project has conducted annual surveys to measure user satisfaction using the American Customer Satisfaction Index (ACSI).
The results are one of the ESDIS Project’s metrics sent to the Office of Management and Budget (OMB).
The Easy Part
We start by setting up an Interagency Agreement between NASA and the Department of Interior.
The survey is conducted by CFI Group under contract with the Federal Consulting Group (FCG) of Department of Interior’s National Business Center.
The FCG is Executive Agent in government for the ACSI. They work with OMB to ensure adherence to applicable Federal regulations such as the Privacy Act and the Paperwork Reduction Act.
EOSDIS Data Centers
SEDACHuman Interactions
in Global ChangeLP DAACLand Processes
& Features
NSIDC DAACCryosphere, Polar
Processes
PO.DAACOcean Circulation
Air-Sea Interactions
ASF DAACSea Ice, Polar
Processes
ASDCRadiation Budget,Clouds, Aerosols,Tropo Chemistry
ORNL DAACBiogeochemical
Dynamics, EOS Land Validation
GES DISCAtmos Dynamics, StratoComposition, Hydrology,
Biosphere, Radiance
MODAPSAtmosphere
OBPGOcean Biology &Biogeochemistry
GHRCHydrological Cycle &
Severe Weather
CDDISSolid Earth
Global Hydrology Resource Center—GHRChttp://ghrc.nsstc.nasa.gov/Hydrologic Cycle, Severe Weather Interactions, Lightning, Atmospheric Convection
Alaska Satellite Facility DAAC—ASF DAAChttp://www.asf.alaska.eduSynthetic Aperture Radar (SAR) Products, Sea Ice, Polar Processes, Geophysics
Land Processes DAAC—LPDAAChttp://LPDAAC.usgs.gov Surface Reflectance, Land Cover, Vegetation Indices
Physical Oceanography DAAC— PO.DAAChttp://podaac.jpl.nasa.gov Sea Surface Temperature, Ocean Winds, Circulation and Currents, Topography and Gravity
Oak Ridge National Laboratory DAAC—ORNL DAAChttp://daac.ornl.gov Biogeochemical Dynamics, Ecological Data, Environmental Processes
National Snow and Ice Data Center DAAC—NSIDC DAAC http://nsidc.org Snow and Ice, Cryosphere, Climate Interactions, Sea Ice
EOSDIS Data Centers
Crustal Dynamics Data Information System—CDDIS http://cddis.gsfc.nasa.govSpace Geodesy
Goddard Earth Sciences Data and Information Services Center—GES DISC http://disc.gsfc.nasa.gov Atmospheric Composition, Atmospheric Dynamics, Global Precipitation, Hydrology, Solar Irradiance, Global Modeling, Multi-Sensor Research Products
MODAPS Level-1 Atmospheres Archive and Distribution System— MODAPS LAADShttp://ladsweb.nascom.nasa.govMODIS Level-1 and Atmosphere Data Products
Ocean Biology Processing Grouphttp://oceancolor.gsfc.nasa.govOcean Biology, Sea Surface Temperature, Biogeochemistry
Atmospheric Science Data Center—ASDC LaRChttp://eosweb.larc.nasa.gov Radiation Budget, Clouds, Aerosols, Tropospheric Chemistry
Socioeconomic Data and Applications Center—SEDAC http://sedac.ciesin.columbia.edu Human Interactions, Land Use, Environmental Sustainability, Geospatial Data, Multilateral Environmental Agreements
EOSDIS Data Centers
E-mail addresses are obtained from:•Orders from registered users•Inquiries•Anonymous FTP access•Data center lists
The number of e-mail addresses per data center varies greatly because the data centers vary greatly:
•Discipline-specific•Number or frequency of available products•Size of user communities•Number of anonymous ftp users•Restricted access or cost
Multiple invitations to the same user•A user with multiple e-mail addresses will receive multiple invitations•A user accessing multiple data centers will receive multiple invitations
Who do we survey?
Survey Questions
The survey contains comment fields and several types of questions:• Demographic questions• Questions to aid recall• Rating questions for the ACSI and EOSDIS models• Non-modeled rating questions
The questions were originally based on a previous survey of the members of the User Working Group for each of the data centers.
Formulating the questions is a combined effort of project personnel, EOSDIS User Services Working Group (USWG), and the CFI Group.
The challenge is to keep the survey short enough so users are willing to respond, yet long enough to reflect the complexity of the data and differences in data centers, and more importantly, understand the needs of the diverse user community.
How did EOSDIS Compare in 2008?
The ACSI is the #1 national economic indicator of customer satisfaction. The ACSI is produced by the Stephen M. Ross Business School at the University of Michigan, in partnership with the American Society for Quality (ASQ) and the international consulting firm, CFI Group.
The ACSI is used to measure over 40 industries and 200 organizations covering 45% of the U.S. economy. Over 70 U.S. Federal Government agencies have used the ACSI to measure more than 120 programs/services.
Survey Results 2004-2008
Although the survey contains over 50 questions, the ACSI is based on responses for only three questions.
N= number of responses used (sample size).
See Slide 9 for confidence levels.
The EOSDIS Model
A series of questions are asked for each of the 6 elements of the EOSDIS model:•Product Search•Product Selection and Order•Delivery•Product Quality•Product Documentation•Customer support
CFI’s methodology quantifiably measures and links satisfaction levels to performance and prioritizes actions for improvement.
2008 Results and Priorities
YYR
ACSI 2004 2005 2006 2007 2008
EOSDIS 75 78 74 75 77
Federal Government*
72 71 72 68 69
Overall 74 73 74 75 75
EOSDIS ACSI Score Comparison 2004-Comparisons 2004-2008
Federal Government*Notes:•These numbers are from the end of year results available at thecsi.org.•The charts in our annual presentations may show a slightly different score because the EOSDIS ACSI score is computed before the end of the calendar year.•Our presentations have either the score from a previous year or a quarterly ACSI average of the scores available at the time that our EOSDIS ACSI score was computed.
Top-line Results 2004-2008
2004 2005 2006 2007 2008Customer Satisfaction Index 75 78 74 75 77Likelihood to Recommend 86 89 86 85 86Likelihood to use Services in Future 86 91 88 87 89
Top-line Results 2004-2008 (cont.)
2004 2005 2006 2007 2008Product Search 70 74 70 72 75Product Selection and Order 73 76 72 74 77Delivery 84 85 79 79 81Product Quality* 68 71 71 72 74Product Documentation - - 72 74 75Customer Support 84 84 82 83 84
* Product Quality includes format questions.
2004 HDF/HDF-EOS
In what format were data or products provided?
HDF-EOS 49%HDF 39%NetCDF 5%Binary 14%ASCII 12%GeoTIFF 19%Other 7%
2005 HDF/HDF-EOS
HDF-EOS 39%HDF 30%NetCDF 3%Binary 6%ASCII 7%GeoTIFF 9%Other 3%
HDF-EOS 22%HDF 20%NetCDF 7%Binary 8%ASCII 10%GeoTIFF 25%OGC, Other GIS, Other
6%
Format preferred …Format received …
Q27. In what format were
your data products
provided to you?
Q28. What format would
/ do you prefer?
HDF-EOS 37% 21%HDF 30% 21%NetCDF 3% 8%Binary 6% 7%ASCII 6% 9%TIFF or GeoTIFF 10% 23%JPEG, GIF, PNG 3% 4%OGC Web services 1% 2%Other 3% 5%
In 2005, 9% said products were provided in GeoTIFF and 25% who said it was their preferred method.
2006 HDF/HDF-EOS
2007 HDF/HDF-EOS
In 2006, 67% said products were provided in HDF-EOS and HDF and 42% said they were their preferred method.
Format data or products was
provided*
Format preferred
HDF-EOS/HDF 73% 41%NetCDF 8% 7%
Binary 11% 7%
ASCII 11% 8%
TIFF or GeoTIFF 20% 21%JPEG, GIF, PNG 9% 4%
OGC Web services 0% 0%
GIS 6% 1%
Don't know 4% 8%
Other 2% 3%
*Multiple responses allowed
*
2008 HDF/HDF-EOS
*Multiple responses allowed
Format data products were provided* Format preferredHDF-EOS/HDF 74% HDF-EOS/HDF 41%NetCDF 9% NetCDF 8%Binary 9% Binary 6%ASCII 13% ASCII 8%GeoTIFF 16% GeoTIFF 20%JPEG, GIF, PNG, TIFF 11% JPEG, GIF, PNG, TIFF 5%OGC Web services 1% OGC Web services 1%GIS compatible 4% GIS 6%KLM, KMZ 2% KLM, KMZ 1%Don't know 4% Don't Know 0%Other 2% Other 4%
2009 Format QuestionsWhat format(s) would/do you prefer? (select any that apply)
•HDF-EOS/HDF•NetCDF•Binary•ASCII•GeoTIFF•JPEG, GIF, PNG, TIFF•OGC Web services (WMS, WCS, WFS, etc.)•GIS (e00, shp, etc.)•CEOS•KML, KMZ•Other (Please specify another format or comment on specific version, etc.
In what format(s) were your data products provided to you? (select any that apply)
•HDF-EOS/HDF•NetCDF•Binary•ASCII•GeoTIFF•JPEG, GIF, PNG, TIFF•OGC Web services (WMS, WCS, WFS, etc.)•GIS (e00, shp, etc.)•KML, KMZ•CEOS•Don’t know•Other (please specify and/or comment
Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate…
•Ease of using the data product in the delivered format•Overall quality of the data product•Overall usability of the data product
2009 Documentation Questions
Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate…
•Overall quality of the document (i.e., technical level, organization, clarity)•Extent to which the data documentation helped you use the data
What documentation did you use or were you looking for?
•Instrument specifications•Science algorithm•Product format•Tools•Science applications•Data product description•Production code•Other
Was the documentation•Delivered with the data•Available online•Not found (Skip to Customer Services)
Customer Service
Was it •By phone•By E-mail•Both by phone and e-mail
Have you requested assistance from <Data center name>’s user services office during the past year?
•Yes, continue•No: Go to Closing
Think about the user services staff you interacted with when you requested assistance from <Data center name> user services. On the same scale from 1 to 10 where 1 means “Poor” and 10 means “Excellent,” how would you rate the user services staff on…
•Professionalism•Technical knowledge•Accuracy of information provided•Helpfulness in selecting/finding data or products•Helpfulness in correcting a problem•Timeliness of response
2009 Survey Responses
Data CenterNumber of addresses
Bounced Back
Received Invitation Responded
Response Rate
GHRC 1174 161 1013 82 8%
ASF SAR DAAC 1372 144 1228 127 10%
CDDIS 1597 612 985 143 15%
GES DISC 2248 812 1436 101 7%
SEDAC 2338 184 2154 131 6%
ORNL DAAC 2351 229 2122 164 8%
LaRC ASDC 2386 410 1976 144 7%
PO.DAAC 3770 1425 2345 173 7%
OCDPS-OBPG 3841 381 3460 266 8%
NSIDC DAAC 4673 911 3762 312 8%
MODAPS LAADS 8198 1119 7079 696 10%
LP DAAC 11585 916 10669 1503 14%
Total 45533 7304 38229 3842 10%
Thank you! any questions?
http://www.humanfactors.com
Some thoughts on users …
Some thoughts on web sites …
And just a few more …