Informix CAC: CompetitiveBenchmarks

Fred Ho, IBM

Content

• AMT/Sybex MDM Benchmark

• Informix-Intel IWA Benchmark

Utility-Scale Smart Meter Deployments, Plans &Proposals*September 2010

© 2010 The Institute for Electric Efficiency

*This map represents smart meter deployments,planned deployments, and proposals by investor-owned utilities and large public power utilities.

http://www.edisonfoundation.net/IEE/

Deployment for>50% of end-users

Deployment for<50% of end-users

Example of Changing Storage Requirements

Monthly MeterReads

Daily MeterReads

15 Minute MeterReads

350.4B

3.65B

120M

Changing Workloads For 10 Million Smart Meters:Today – Each meter read once per monthVery soon – Each meter read once every 15 minutesRegulations – Need to keep data on line for 3 years (PUC) and, perhaps, save for 7 years

# of RecordsPer Year for10M meters

Data for a Utility In California

Frequencyof reads

Keeping up with Smart Meter Data

Large amounts of data causes problems in 2 areas:

• Storage management

• Large storage = Expensive

• Cumbersome to maintain

– Require sophisticated partitioning schemes

– Reorganization often required, which leads to downtime

• Query performance

• Compliance Reports must be completed before the end of eachday

• Customer portal queries must be handled in a timely manner

• Customer billing must be completed each day

100 Million Smart Meter Benchmark: Goals

1. Measure processing times for data collected over a 31 day period for:

100 million meters at 30-minute intervals in an 8 hour day.

2. Demonstrate consistent processing times over the 31 day period.

3. Demonstrate linear storage growth of data stored over the 31 day period.

4. Complete one day’s billing cycle while simultaneously processing and loading meter datafor 100M within an 8 hour time period.

5. Demonstrate all processing can be done using a low-cost combination of commerciallyavailable hardware, storage, and software.

•Operations performed each day

– Load interval and register data for 100M meters at 30 minute intervals (49records/day/meter)

• 49 records/day/meter * 100M = 4.9 Billion records/day

– Perform VEE on the data (validation/estimation/editing)

– Run a daily billing cycle on 6% of the meters

– Gather results for 31 days

100 Million Smart Meter Benchmark: Operations

Perform Validation, Estimation & Editing

Run daily billing cycle

Gather Results

Load & Register Data (100 million meters)

Processed over 30 minute intervals over 31 Days

Proc

essi

ng

100 Million Smart Meter Benchmark: Components

Stor

age

Are

a N

etw

ork

(SA

N)

IBM Power P75032 cores (3.5 GHz) – 16 active500 GB RAM1 GB LAN Fiber dual port adapter - 1 active2 X 8GB FC dual port adapter - 4 active portsof storage

XIV Storage System15 X 2TB storage6 X 6 FC connections @ 4GB

4 X 8GB6 X 4GB

Software Stack

Informix v11.70.xC3 with TimeSeries version 5.0

AMT-Sybex Affinity Meterflow Meter

AIX v7.1

Hardware StackIBM System P Series &Storage

AIX v7.1

Informix 11.70

Monitor & Admin

AMT-Sybex AffinityMeterflow

An “end to end” run of 100 Million Meters at 30 minute intervals was performed for 31 days ofdata

The result: all data was prepared, loaded, validated as well as a billing cycle run in less than 8hours

The average time to do these operations remained consistent over the 31 days– Performance remained constant even as storage increase

The billing cycle completed in less than 5 hours and ran concurrently

100 Million Smart Meter Benchmark: Load Results

Process Avg. ElapsedTime

Avg. ThroughputRatePreparation and Technical

Verification2 hrs 10 min 628,205

records/secData Load 3 hrs 14 min 420,962

records/secValidation, Estimation, andEditing (VEE)

2 hrs 11 min 623,409records/sec

Total Time: 7 hours and 35 minutes!

Tota

l Tim

e-M

inut

es

No. of Days

Total Process Time over 31 Days100 Million Meters @ 30 minute intervals

Load Time over 31 Days100 Million Meters @ 30 minute intervals

Tota

l Tim

e-M

inut

es

No. of Days

Load Performance: Storage Comparison over TimeD

isk

Spac

e in

TB

No. of Days

Storage in TB over 31 Days100 Million Meters @ 30 minute intervals

Why did we choose MDM benchmark?

Informix database has a very strong extensibility story unmatched by anyone inthe industry

Informix’s native handling of time stamped data through TimeSeries extensionsand APIs is already proven to perform orders of magnitude faster takes muchless storage

Helps recruit new partners in the E&U space

Opportunity/potential to grow business by winning new customers in a industrythat is getting transformed

Meet the MDM challenge and solve a key customer problem. Many Energy andUtilities companies are facing scalability issues with competing databasesolutions.

Opportunity leverage on IBM's larger ‘Smarter Planet’ initiative. Gain visibilityand re-vitalize Informix within IBM GBS and S&D teams who are focusing onSmarter Planet solutions

Informix-Intel IWA Benchmark

Tick-Tock Development ModelSustained Xeon® Microprocessor Leadership

Tick Tock Tick Tock Tick Tock Tick Tock

Intel® Core™Microarchitecture

NehalemMicroarchitecture

Sandy BridgeMicroarchitecture

65nm65nm 45nm45nm 32nm32nm

Xeon® 5300

Xeon® 5100 Xeon® 5400

Xeon® 5200Xeon® 5500 Sandy

Bridge-EP/ENXeon® 5600 Ivy Bridge

EP/EN

22nm22nm

Dedicated high-speed bus per CPU

HW-assisted virtualization (VT-x)

Integrated memory controller with DDR3support

Turbo Boost, Intel HT, AES-NI1

End-to-end HW-assisted virtualization (VT-x, -d, -c)

Integrated PCI Express

Turbo Boost 2.0

Intel Advanced Vector Extensions (AVX)

First high-volume server Quad-CoreCPUs

Up to 6 coresand 12MB Cache

Up to 8 coresand 20MB Cache

13

Product Line Descriptions

Intel® Xeon® processor 3000 sequence platforms » Intel® Xeon® processor E3 platformsEconomical (1-way) dependable general purpose 64-bit servers well-suited for small businesses andeducation with features that optimize performance, uptime, and security

Intel® Xeon® processor 5000 sequence platforms » Intel® Xeon® processor E5 platformsVersatile (up to 4-way) servers for all your infrastructure, high-density, workstation and HPCapplications with features that enable optimal performance and power efficiency for the datacenter.

Intel® Itanium® processor 9000 sequence platformsArchitected for mission critical UNIX and mainframe-class reliability & scalability. Targeted at large-scale databases, data warehouses, ERP, business intelligence, and data analytics. Deliversuncompromising scalable performance for today’s most demanding workloads and world-classreliability for uninterrupted real-time business processing and decision support

Intel® Xeon® processor E7 platformsScalable (up to 256-way), reliable, powerful 64-bit multi-core servers offering industry-leadingperformance, expanded memory & I/O capacity, and advanced reliability ideal for the most demandingenterprise and mission critical workloads, large scale virtualization and large-node HPC applications.

Informix and Intel Performance Tests

IWA on Intel Xeon E7 Processors

17

INTEL/IWA: Breakthrough technologies for performance

1

2

34

5

6

7 1

2

34

5

6

7

1. Large memory support64-bit computing; System X with MAX5 supports upto 6TB on a single SMP box; Up to 640GB on eachnode of blade center. IWA: Compress large datasetand keep it in memory; totally avoid IO.

7. Multi-core, multi-node environmentNehalem has 8 cores and Westmere 10 cores. This trend isexpected to continue. IWA: Parallelize the scan, join, groupoperations. Keep copies of dimensions to avoid cross-nodesynchronization.

4. Virtualization PerformanceLower overhead: Core micro-architectureenhancements, EPT, VPID, and End-to-EndHW assist IWA: Helps informix and IWA toseemlessly run and perform in virtualizedenvironment.

5. Hyperthreading2x logical processors; increases processorthroughput and overall performance of threadedsoftware. IWA: Does not exploit this since thesoftware is written to avoid pipeline flushing.

3. Frequency PartitioningIWA: Enabler for the effective parallel accessof the compressed data for scanning.Horizontal and Vertical Partition Elimination.

2. Large on-chip CacheL1 cache 64KB per core, L2 cache is 256KB percore and L3 cache is about 4-12 MB.Additional Translation lookaside buffer (TLB).IWA: New algorithms to avoid pipelineflushing and cache hash tables in L2/L3 cache

6. Single Instruction Multiple DataSpecialized instructions for manipulating128-bit data simultaneously. IWA:Compresses the data into deep columnarfashion optimized to exploit SIMD. Used inparallel predicate evaluation in scans.

Why did we choose Intel benchmark?

IWA takes full advantage of Intel Xeon processor architecture

IWA working with Intel to provide commodity hardware with the bestTCO

Customers can take full advantage of future “chip spin” in processortechnology of Intel with minimal cost

Accommodates terabytes of main memory at little cost

In line with IBM System X product family but compatible with othervendor offerings

Thank you!