May 2013 Mass Report Generation Managing High Volume Reporting

May 2013

Mass Report Generation

Managing High Volume Reporting

Overview of Volume Output Requirements Common Problems Report Resolution Demonstration

Agenda

In 2012 the WorleyParsons Environmental Data Management Team processed:· 6,500 lab EDDs,

· From 4 primary laboratories, accounting for

· 28,000 samples, and

· 1,436,000 results,

Into 3 databases containing 3,000 facilities across Canada

Overview of Volume

Common data table and graph outputs are displayed:· Historically, year over

year

· With guidelines (action levels) comparisons applied

· Grouped by analytes

· From scratch on a regular basis (i.e. quarterly)

Output Requirements

Table 7 of 8, Page 105 of 132 for 1 of 11 sites

Graphing sheet 3 of 800

Staffing:· It takes ~6 months for staff to get up to speed

· After 2.5 years they are burnt out

Timing:· We have several programs that require

reporting to be in the clients hand within 24 hours of sample drop off

Laboratory issues:· Inaccurate or incomplete reporting

· Reporting what we want, not what we need

· Labs sign reports, not data

System Issues:· An x86 32 bit machine has a 2 GB memory

limit

· Large data sets using a modified EQuIS report are time consuming, taking over 45 minutes to run

· Cross tab display uses 5mb/second

· Exporting to Excel 20mb/sec

Common Problems

Does anyone have an analyte group that looks like this?

One analyte, multiple methods.

In EQuIS 5.6 EarthSoft introduced a new Beta analytical report

Allows for a new approach to reporting, using dynamic SQL

Combined with a modified lab reporting methodology WorleyParsons has been able to reduce reporting times significantly

Core features include:· Significant time and memory reduction

− 47,000 record report (pulled in 23 seconds)

− cross-tabbed to 5,785 (37 seconds)

− with guidelines applied (41 seconds)

− Similar data pulled in standard EQuIS report without guideline comparison and no-cross tab (90 seconds)

· Simplified analytical grouping

· Auto Cross-Tab functionality without significant memory loss

· Direct Guideline (Action Level) comparison

· Location Specific Guideline Comparison

· Direct Web GIS Interfacing

Report Resolution

Demonstration (Standard Analytical Report)

Pick Selection

Analytical Report Applied to Web GIS

Analytical Report Pick Interface

Web GIS Report Results

Thank you.