View
228
Download
0
Category
Preview:
Citation preview
© 2014 IBM Corporation
Accelerating Oracle with IBM FlashSystem: The Need for Speed
Mike Ault - Oracle FlashSystem Consulting Manager, IBM
January 2014
© 2014 IBM Corporation
Smarter Computing Demands Flash
“The more we have flash for consumer devices... ...the more we need flash for our data centers...”
© 2014 IBM Corporation
Why Flash Storage…….. Timing is Perfect !!
In the last 10 years…
• CPU Speed: Performance increase roughly 8-10x
• DRAM Speed: Performance increase roughly 7-9x
• Network Speed: Performance increase of 100x
• Bus Speed: Performance increased roughly 20x
• Disk speed: Performance increased 1.2x
� IBM FlashSystem™ 50+x 300K+
RPM Disk
© 2014 IBM Corporation
Most Costly & Volatile
Time Consuming, Very Expensive &
Risky
Wasteful, Expensive & Ineffective with Storage Latency
Issues
Expensive & Ineffective for
Storage Performance Issues
Datacenter’s Response to Bridge Disk Performance Gap
Add More Memory
Typical Performance
Mitigation Tactics
HDD Performance Enhancement
Add CPUsTune & Modify
Application
© 2014 IBM Corporation
What if we only reduced Latency ??
Consider Little’s Law of mathematical queue theory as it applies to Application Performance
Now let’s see how FlashSystem alters this equation
Q = Number of parallel threads running in the application
t = Time it takes for an IO request to be serviced (Latency)
R = Result, typically measured in IOPS or Bandwidth
Let’s Assign Values to this equation:
A 50X improvement in application response time by only installing Flash !
© 2014 IBM Corporation
• 38% Lower software license costs
• Due to fewer cores
• Lower software maintenance
• More Efficient Infrastructure
• 13% lower infrastructure software costs
• 35% lower operational support costs
• Server / Storage Admin
• Much better storage utilization
• As much as 50%
• Lower maintenance
• Ease management by 50%
• 17% Fewer Servers
• Fewer cores
• Lower Memory
• Fewer network connections
• Lower maintenance
• Environmentals 74% Lower Cost
• Lower power / cooling
• Less floor space
All Flash is 31% 31% Less Expensive Overall
Economics of All Flash, Performance is Really Just a BONUS!BONUS!
© 2014 IBM Corporation
Microsecond latency maximizes Application CPU utilization
I/O Serviced by Disk
1. Issue I/O request ~ 100 μμμμs
2. Wait for I/O to be serviced ~ 5,000 μμμμs
3. Process I/O ~ 100 μμμμs
• Time to process 1 I/O request = 200 μμμμs + 5,000 μμμμs = 5,200 μμμμs
• CPU Utilization = Wait time / Processing time = 200 / 5,200 = ~4%
Time
Processing~100 µs ~100 µs
Waiting~5,000 µs
1 I/O Request
CPU State
I/O Serviced by IBM FlashSystem
1. Issue I/O request ~ 100 μμμμs
2. Wait for I/O to be serviced ~ 200 μμμμs
3. Process I/O ~ 100 μμμμs
• Time to process 1 I/O request = 200 μμμμs
+ 200 μμμμs = 400 μμμμs
• CPU Utilization = Wait time /
Processing time = 200 / 400 = 50%
Time
Processing~100 µs ~100 µs
Waiting~200 µs
1 I/O Request
CPU State
12X Application benefit by only changing storage latency!
12X Application benefit by only changing storage latency!
© 2013 IBM Corporation
Introduction
� Important applications require:
� High Performance� Queries, reports, and screens must return quickly� Scale to high user loads
� Reliability� 100% uptime� Single system fault can not be fatal� Loss of processing impacts bottom line
� Cost Effectiveness� Effective use of resources� Leverage tech to achieve accelerated performance gains
for the cost� Reliability can not be compromised
© 2014 IBM Corporation
But…Where Does Oracle Need Speed?
© 2013 IBM Corporation
Oracle and Queries -Where does latency matter?
Memory
SGA & PGA
Oracle
Processes
Tables &
Indexes
Logs
Reads - Cache miss
Foreground Waits:
DB file sequential read
DB file scattered read
3-5 ms
User’s Query
Storage latency
READS
© 2013 IBM Corporation
Why Don’t Writes Matter?
� For data and index block writes:
–Uses delayed block cleanout
–Writes when it can’t find clean blocks
–Writes every 3 seconds
–Writes on checkpoints
© 2013 IBM Corporation
Oracle and Insert/update/delete- Where does latency matter?
Memory
SGA & PGA
Oracle
Processes
Tables &
Indexes
LogsLGWR
(foreground)
DBWR (background)
Users Insert
Commit
LOG WRITES
© 2013 IBM Corporation
Where Else?
�Temporary Activity
–Sorts
–Hashes
–Bitmaps
–Global Temporary Tables
�Non-memory Undo activity
�Flash Cache
© 2014 IBM Corporation
FlashSystem 840
© 2014 IBM Corporation
IBM FlashSystem 840: Hardware View
Flash Modules (12)
RAID Controllers (2)
Battery Modules (2)
Power Supplies (2)
Fan Modules (4)
Interface Modules (4)
Management Modules (2)
Canisters (2)
Improved RAS features
�Front/Back accessible Hot-swap Flash Modules, Power Supplies, Batteries, Fans, Controllers w/ interface cards and Canisters�Non-disruptive maintenance and firmware updates (concurrent code load)
Improved RAS features
�Front/Back accessible Hot-swap Flash Modules, Power Supplies, Batteries, Fans, Controllers w/ interface cards and Canisters�Non-disruptive maintenance and firmware updates (concurrent code load)
© 2014 IBM Corporation
Superior Durability:Using the Best Flash
10X
3X
Superior Protection: Beyond Disk RAID Chip/Plane/Die level protection
Self-Recovering Flash Modules
Avoid system rebuilds
Protection Within And Across Flash Modules
Variable Stripe SizesRead Disturb Mitigation
Automatic Read SweeperHigh-Speed Clock Recovery
Advanced Engineering = Less Maintenance
IBM FlashSystem 840: Reliability Ingredients
� SLC Market demand decreasing. � eMLC data protection techniques
delivering more wear life than what market demands
� eMLC delivers best Price/Performance
© 2014 IBM Corporation
Storage Performance Council (SPC-1/e)
© 2014 IBM Corporation
FlashSystem Result Details: IBM MicroLatency™
� Leadership minimum reported latency (SPC-1 LRT™): 0.18 ms
� Single-system latency leadership up to about 85K IOPS, scalable with multiple FlashSystem units
� Nearest latency competitor (HDS) uses 2 racks of equipment, over 2x the flash for storage, plus a massive 1 TB DRAM cache and an additional 1 TB flash cache
� Nearest standard SSDs are ~2x the latency!
© 2014 IBM Corporation
FlashSystem Result Details: Extreme Performance
� Our result shows a single 1U “building block”, not a highly scaled out design like other results
� Maximum aggregate performance of 195,021.70 SPC-1 IOPS™ from a single 1U FlashSystem 820
� Scale IOPS linearly by stacking FlashSystem units
� Strong “performance efficiency”:
– ~200K IOPS per rack unit
– ~50K IOPS per 8 Gbit FC port
– ~250 IOPS/Watt - more than 5x better than last SPC-1/E leader
System
8 Gbit
FC Ext
Ports
Max IOPS/ Ext
Port
$/ASU
GB
Huawei Dorado5100 8 75K $76
Huawei Dorado2100
G28 50K $60
IBM
FlashSystem 8204 49K $25
HDS
HUS 150 (SSDs)4 31K $116
HDS
VSP with HAF32 19K $148
HP StorServ 7400
(SSDs)20 13K $130
SLC + servers = more speed, higher
price! We can do similar with 7xx
products—but do clients need it?
IOPS per external port
“normalizes” aggregate
performance across large
and small-scale results.
© 2014 IBM Corporation
OPERA (Preferred read)IBM FlashSystem Accelerating Disks
© 2014 IBM Corporation
WRITE
S
READS
ASM FG2
ACTIVE
DATA
TRANSITIONAL
DATA
20
TB
5 TB
OPERA Example
21
ARCHIVE DATA
100
TB
ASM FG1
ACTIVE
DATA
TRANSITIONAL
DATA
20
TB
5 TB
ASMASMBoost Performance
Boost Redundancy
- Without Disruption
- Without Risk
- Without Feature
Loss
IBM Flash
System
SANSAN SAN
SAN
DB
Servers
MirrorMirror
© 2014 IBM Corporation
Storwize V7000:
• 36x 300GB 10k disks
• Brocade SAN switch:
• SAN40B-4 8Gbit ports
• Power server(Lpar1&2):
• Power 750 – 8233-E8B
• Each has:
• 8 CPU
• 100 GB memory
• AIX 7.1 TL2 SP2
• 2x 8Gbit FC ports
• FlashSystem 820 20TB
• 4x 8Gbit FC ports
• 8 x 500Gib Luns
Optimal Performance Enhancing Real FlashSystem Architecture
© 2014 IBM Corporation
8,000 Reads / Sec now at extremely low latency
Preferred Read – Acceleration Example
System does 10,000 Writes & IBM FlashSystem does 10,000 Writes &
40,000 Reads
System performance @10,000 IOPS for a given appRead/Write Ratio @ 80% Reads / 20% Writes
Reads: 8,000 / SecWrites: 2,000 / Sec
Introduce IBM FlashSystem as Primary Copy of new mirror
System was 10,000 IOPS� Now 10,000+ Writes / Sec
R/W ratio does not change; No change in the app
= System Accelerated 5x
= System Accelerated 5x
© 2014 IBM Corporation
Swingbench OLTP ResultsIBM FlashSystem verses V7000
© 2014 IBM Corporation
V7000 results
© 2014 IBM Corporation
Acceleration with IBM FlashSystem 820
V7000FlashSystem
820 X Increase
8.06 2.29 2.52
28.43 2.15 12.22
15.26 2.04 6.48
82.42 7.3 10.29
© 2014 IBM Corporation
DWH Swingbench ResultsV7000 and FlashSystem 820
© 2014 IBM Corporation
V7000 results
© 2014 IBM Corporation
FlashSystem 820 Acceleration
V7000Flash
System820 X Increase
1453703 621184 2.34
2885604 718334 4.02
255656 64721 3.95
3705331 1344883 2.76
1660603 857933 1.94
© 2013 IBM Corporation
SLOB (Silly Little Oracle Benchmark) Testing Scenario
�SLOB generates the IO requests via PL/SQL, thus exercising full Oracle IO machinery along with SGA etc.
�SLOB is capable of testing random single block reads, writes andextreme REDO logging. It does all this with no application contention, thus allowing one to measure true maximum IO that can be achieved on a system .
�Workload consisted of 56 users from both RAC nodes generatingdb file sequential reads (Random Physical single block Reads).
�The test started with preferred read set to the disk failure group and continued after preferred read changed to FlashSystemfailure group - ONLINE.
© 2013 IBM Corporation
Acceleration of Database Creation with IBM Flash System
�After swithcing to FlashSystem (05:27 PM) Disk IO wait disappears andwaiting is now on host CPU. This graph shows the effect of the low latency of FlashSystem and how it increases the host CPU utilization.
© 2013 IBM Corporation
SLOB TEST – PR Online Acceleration of Disk by FlashSystem
© 2013 IBM Corporation
SLOB Test Results
Just by Adding a Single FlashSystem Box to an Oracle environment you can get:
�36x acceleration of IOPS!
�30x acceleration of throughput!
�40x Acceleration of Latency! (assuming 0.43 ms, Oracle reports <0.5 as 0)
© 2013 IBM Corporation
Before – Read From Disk
After Acceleration –Read From FlashSystem
The average read response time for the first instance accelerated from 15.33 ms to 0.43 ms and average IOPS accelerated from 3644 to 111831
Preferred Read Acceleration – Comparing AWR logs
© 2013 IBM Corporation
System Configuration
� Linux X86 RHEL Server
� Gen2 XIV disk array
� IBM FlashSystem 820
� ASM used to mirror between XIV and FlashSystem
� Switched preferred read mirror at Instance level
� Gen2/Flash means the Gen2 was PRM
� Flash/Gen2 means FlashSystem was PRM
� With PRM to FlashSystem achieved 75/25 ratio of FS to XIV reads
© 2013 IBM Corporation
Gen2 XIV and FlashSystem PRM Tests
© 2013 IBM Corporation
Gen2 XIV and FlashSystem PRM Tests
© 2013 IBM Corporation
Gen2 XIV and FlashSystem PRM Tests
© 2013 IBM Corporation
Conclusions
� Using ASM PRM achieved near Flash-only levels of performance
� Preferred read mirror using IBM FlashSystem provides dramatic performance boost in read heavy environments.
© 2014 IBM Corporation
How About Some Real World Tests
© 2013 IBM Corporation
ABB
As Is Environment:
� ABB US has ~ 15 major manufacturing plants & 7,500 users
� All plants depend on SAP; The SAP Oracle DB = ~ 3.2 TB
� The PR1 DB is hosted on HACMP-clustered AIX LPARs; The LPARs are clustered between 2x p570 P6+ frames
� The DB resides on 2x DS8700 arrays front-ended by SVCs
� DB LUNs are mirrored between the 2 arrays at the host level
© 2013 IBM Corporation
Business Challenge
� User dissatisfied with SAP performance
� Slow month-end batch reads and reporting
� Dialog response times approaching business SLAs
� Performance concerns causing hesitation to invest in growth
� 100K (USD) per month in SLA fines
© 2013 IBM Corporation
Fixes:
Many alternate solutions tried / considered but limited success:
� Additional CPU => Limited improvements; Additional cost
� SAP dedicated SAN => Too expensive
� DS8700 SSD => Too expensive; Configuration limitations
� SAP / Oracle tuning => Limited changes helped a little; Extensive changes too labor intensive
� SAP HANA => Un-proven; insufficient app team cycles
© 2013 IBM Corporation
Proposed Fix
� Install (2) IBM FlashSystem 810 units, one to accelerate each Pod
– Attach behind SVC; powered by IBM P Series and AIX
– Migrate the database and logs to the SSD and mirror across the Pods via AIX-level mirroring (standard mirroring used today)
© 2013 IBM Corporation
Predicted Acceleration
Predicted CPU Utilization
0
20
40
60
80
100
120
140
Before and After
Perc
ent CPU%
Corrected CPU%
% Increase
0.00%
50.00%
100.00%
150.00%
200.00%
250.00%
Current wait time total 70.84%
New project wait time 2.78%
Total Improvement 213.10%
1
© 2013 IBM Corporation
Results
Actual CPU Utilization
-
20.00
40.00
60.00
80.00
100.00
120.00
140.00
160.00
180.00
Before and After
Perc
en
t CPU%
Corrected CPU%
Percent Increase
© 2013 IBM Corporation
Proof points: Stuttgarter Straßenbahnen AG
47
• SAP ERP Finance• Oracle• AIX• SVC stretched cluster
Several other customer
cases show batch run
time reduction by factor
5 to 10 !
© 2014 IBM Corporation
Non-Oracle Examples
© 2013 IBM Corporation
Tipping Point Demonstration
Highly Scalable & I/O IntensiveOLTP Database Workload
�Compelling EconomicsSignificantly improved workload efficiency
�Extreme CapacityBuy only what you need; add capacity as needed
�Application TransparencyAvoid risk and cost of change as you grow or subside
�Continuous AvailabilityUninterrupted access to data with consistent performance
IBM FlashSystem, IBM Power Systems and DB2
IBM Power 780
(4 nodes, 128 Cores,2TB Memory)
IBM FlashSystem 820
(4-1U units, 20TB Each)
Fibre ChannelNetworking
10 GbE Networking
IBM DB2 v10.5
(10-8 core cluster members)
© 2013 IBM Corporation
Tipping Point Demonstration ResultsIBM FlashSystem, IBM Power Systems and DB2
• 1.3 Million IOPS
• 43K+ Transactions per second
• 13K Updates per second
Normalized $ / IOPS
Energy Space
IBMFlashSystem
2,500Spindles+ 128 SSDs
5,000 Spindles
11x Less
IBMFlashSystem
2,500Spindles+ 128 SSDs
5,000 Spindles
80x Less
IBMFlashSystem
2,500Spindles+ 128 SSDs
5,000 Spindles
26x Less
© 2013 IBM Corporation
All flash Case Study: Life sciences Client
10 TB Flash System 820
SQL cluster
IBM 3650 IBM 3650
Problem•Experiencing pain with JDE BD loads / backups / restores
•Needed better system performance for the end user
Solution•Installed IBM FlashSystem 820 into a a SQL DB, clustered, running Oracle JDE
•Included Oracle OLAP processes
Benefit•Backup Time improved from 5 hours to 42 minutes
•Restore Time improved from 6.5 hours to 1.2 hours
•Batch times went from 7:30 hours to 2:37 and 17:47 to 7:07
© 2013 IBM Corporation
Questions?
Mike Ault
mrault@us.ibm.com
Thanks to :hakany@tr.ibm.com
Ali Fığığığığlalıııı alif@tr.ibm.com
Orçun Budak orcunb@tr.ibm.com
STG Turkey for current results!
Recommended