Unlocking the Mystery of the Data Quality Act: What it Means for NOAA Research
Jamie Krauk,
Office of Scientific Support
August 6, 2002
Overview Section 515 – Data Quality Act Directives from OMB What is “Information”? The Three Elements of Quality: Utility, Integrity,
and Objectivity NOAA’s Information Standards Compliance Procedures Administrative Mechanism Implications for NOAA Research Discussion
Section 515 – Data Quality Act
Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554):– Directs the Office of Management and Budget
(OMB) to issue government wide guidelines that “provide policy and procedural guidance to Federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information disseminated by Federal agencies”
Directives from OMB
OMB issued guidelines directing all Federal agencies to:– Create and issue agency-specific standards to
ensure Information Quality and develop a pre-dissemination review process
– Establish administrative mechanisms allowing affected persons to seek and obtain correction of information
– Report periodically to OMB on the number and nature of complaints received
What is “Information”?
Any communication or representation of knowledge such as facts or data, in any medium or form, including textual, numerical, cartographic, narrative, or audiovisual forms (includes web dissemination but not hyperlinks to information that others disseminate; does not include clearly stated opinions)
The Three Elements of Quality Utility
– The usefulness of the information to its intended users, including the public
Integrity– The protection and security of information from
unauthorized access or revision to prevent corruption or falsification
Objectivity– The presentation of accurate, reliable, unbiased
information in an accurate, clear, complete and unbiased manner within the proper context
NOAA’s Information Standards
Ensuring Utility NOAA strives to continually improve the
usefulness of its data and information products NOAA interacts with broad range of customers
through workshops, surveys, product reviews to assess and improve product utility and accessibility
NOAA disseminates products in a manner that allows them to be accessible and understandable to a broad range of users via print, diskettes, CD-ROM, internet, and broadcast media using consistent format and attribution
Ensuring Integrity Information is safeguarded from improper access
prior to dissemination to a degree commensurate with the risk associated with the information if it was compromised
All electronic disseminated information complies with “Security of Automated Information Resources”, Computer Security Act, and Government Information Systems Reform Act
Confidentiality of information is protected under the Privacy Act
Additional protections are provided as appropriate
Ensuring ObjectivityA) Original Data
B) Synthesized Products
C) Interpreted Products
D) Hydro-meteorological, Hazardous Chemical Spill, and Space Weather Warnings, Forecasts, and Alerts (Time Sensitive Products)
E) Experimental Products
F) Natural Resource Plans
G) Corporate and General Information
Objectivity – Original Data Data are collected according to documented
procedures or in a manner that reflects standard practices accepted by the relevant scientific and technical communities
Original data undergo quality control prior to being used by the agency, including:– Gross error checks, comparisons with other independent
sources of measurement, examination of statistical summaries, visual inspection
NOAA strives for transparency regarding data collection procedures, level of quality, and limitations
NOAA’s quality control process is ongoing
Objectivity – Synthesized Products Data and information sources are identified or made
available upon request All data is of known quality or from sources
acceptable to the relevant community Products are created using methods in standard
manuals, documented in an accessible format, or generally accepted by the relevant community
All synthesized products, or the procedures used to create them, are reviewed
All methods are included with product or made available upon request
Objectivity – Interpreted Products Data and information sources are properly
referenced or made available upon request Products are created using methods in standard
manuals, documented in an accessible format, or generally accepted by the relevant community
All products are put into context by additional information that demonstrates quality and limitations of products
All products are reviewed ranging from internal peer review to external, independent review
All methods are included with product or made available upon request
Objectivity – Time Sensitive Products
Data of known quality is used to the extent possible under tight time constraints
Data and information sources are identified or made available upon request
To the extent possible, information is produced using methods and techniques that are documented in accessible formats or are generally accepted
Accuracy and skill score of warnings, forecasts, and advisories is tracked as a mechanism for evaluation
Objectivity – Experimental Products Products are either disseminated for experimental
use, evaluation or feedback, or used in urgent, time-critical cases when qualified scientists believe risk will be lessened if the product is used
The best science and supporting studies available are used in accordance with sound and objective practices, evaluated in relevant communities, and peer reviewed if feasible
Provisional documentation of theory and methods are prepared, including assumptions
When products are used, they are accompanied by explicit limitations and used in conjunction with non-experimental methods and tools if possible
Objectivity – Natural Resource Plans Plans are developed according to published
standards and are often used as basis for policy Plans are a composite of several types of scientific
information from both internal and external sources and are based on the best information available
All supporting materials and data are properly referenced
Plans are reviewed by technically competent individuals ranging from internal to external independent review commensurate with importance
Objectivity – Corporate Information This is non-scientific, non-financial, non-
statistical information Sources of materials are noted as appropriate
and information is reliable and accurate to an acceptable degree of error
Product review is incorporated into the normal product formulation process, including:– Personal or supervisory review, process design
and monitoring, use of quality check lists, use of management controls
Compliance Procedures Read and understand all standards carefully and
ensure compliance by October 1, 2002 Identify what research products and information
fall into each category, and the corresponding standard for each
Prepare a current and updateable inventory of all scientific procedures and limitations for each type of information and have available upon request, including:– Methods, materials, assumptions, techniques– Process by which information was last reviewed– Date of last review, names of reviewers, actions taken
Administrative Mechanism Any affected person may request timely correction
of disseminated information Requests must be directed to NOAA Section 515
Officer (NOAA Executive Secretariat) for recording and forwarding to Lab/Program
Requests must contain all relevant information or the request is denied or returned to be amended and resubmitted
If the request is complete the Lab/Program director determines whether the request states a claim
Either way, a response is required within 60 days from original receipt to NOAA Section 515 Officer
Administrative Mechanism A claim is stated if the information was based on
misapplication or non-application of the appropriate standard
All supporting documents pertaining to the information in question must be gathered and filed as potential future administrative record
Lab/Program decides whether the information complies with the applicable standard– If yes, an explanatory response is drafted within 60 days
and requestor has 30 days to file an appeal to DAA– If no, Lab/Program decides whether the information can
be corrected, how, and when and then Lab/Program drafts a response within 60 days as above
Implications for NOAA Research
Paradigm shift in NOAA’s research enterprise?
Opens door for court cases and limits flexibility and scientific freedom
Required to keep impeccable notes Web community needs to be aware and
well-versed in standards in order to be the eyes and ears for NOAA Research
Questions?