Upload
eddiesharkey
View
126
Download
2
Tags:
Embed Size (px)
Citation preview
Exadata V2 / Sun Oracle Database Machine and POC results for Market Risk Reporting
Presentation is split into 2 parts Slides 3 26 highlight Market Risk POC results, just completed for another Tier 1 Investment Bank I wanted to provide a very relevant real life example of Oracle Exatadatas performance capabilities Slides 27 - 49 provide an overview of Oracles Exadata / Database Machine
Market Risk Reporting & AnalysisPOC just completed for another Tier 1 Investment bank
Value Proposition and why Oracle We make near real-time / on-demand risk and reporting a reality This demonstrably improves risk management and governance This increases trading revenues, reduces trade losses and reduces operational costs We are not market risk specialists We do not have a pre-built market risk platform We do have game changing technology and experience of building risk platforms
Why are firms doing this now?Most investment banks are unable to view risk on-demand or in near real-time Especially by trade, by instrument, by book, by counter-party, across asset classes or bank wide Why? Trading systems and risk systems are typically siloed Historically, risk has been calculated overnight Recent market events driving need for intra-day risk capabilities More advanced risk calculations e.g. gamma risk for credit derivatives has taken too long to be viable so quants ran simpler risk models not any more! Oracle Sun Exadata now offers Complex Risk Reporting and management of large data volumes in Near Real Time. Oracle Coherence is still the solution for Real Time Data Access needs
Tier 1 Market Risk POC - Test Configuration HP Oracle Database Machine (V1 Hardware) Exadata V2 Software Oracle 11g Release 2 8 Oracle Database servers 64 Intel processor cores Oracle Enterprise Linux
14 Exadata Storage Servers 75.6TB raw storage
InfiniBand switches
Test Scenarios Tests based on actual Market Risk data. Tests carried out on risk data volumes from 3 million (current) to 80 Billion (future) rows. Load tests based on loading from 3 million (current) to 2 Billion (Future) rows four times a day (8B total/day). Query tests based on actual SQL generated from frontend reporting systems. Additional query tests run using Materialised Views and OLAP cubes for comparison. The tests were carried out to provide a comparison between an Oracle Exadata solution and an existing Oracle Data Warehouse solution
Test Results Loading: Current customer solution load takes 6 minutes (3M rows). Tested load on Exadata takes only 4 minutes to load 2 Billion rows
1000 times greater load capacity (Could be improved with tuning). Data loaded in a compressed form (about 7-1 compression) saving large amounts of storage space and improving performance. Queries: All query tests initially carried out against base tables (no indexes, materialised views or other tuning). The current system has reached its performance limit Exadata scaled to data volumes over 25,000 times greater than the current system (and still had lots of room for greater expansion).
Test Results Query Performance Enhancements: Current system uses over 250 Materialised Views for performance reasons. It currently takes 25 minutes to create these views (against 3 million rows). It takes only 15 minutes to create all the views in Exadata (against 2 Billion rows). One OLAP Cube can be used to replace over 250 Materialised Views (and can provide many more dimensions of analysis!) It took only 10 minutes to create the single OLAP cube. Queries from the Cube have sub-second response times.
Test Results A generic Materialised view (can be used in lots of different queries) reduced the query time for one of the sample queries from 1 Min 8 Seconds to 17 seconds and took only 20 seconds to create. A targeted Materialised View (specific to a number of queries including the example above) reduced the query time to subsecond and took less than 2 seconds to create (from the generic materialised view). A single OLAP cube took only 20 minutes to create, replaced all 250+ materialised views and provided many more dimensions of analysis than the MVs it replaced. The same query run against the OLAP cube was also sub-second
Sample Test Results One of the sample queries currently takes 10 Mins 18 Secs to run against the 3M rows of data. The tested Exadata solution took only 2 mins 3 Secs to run against 80B rows of data!
One of the sample queries currently takes 1 Mins 45 Secs to run against the 3M rows of data. The tested Exadata solution took only 47.5 Secs to run against 80B rows of data
Future Testing The Sun Oracle Database Machine is even faster (more than twice as fast). Faster Processors - Intel Nehalem at both the Database and Storage tiers Faster I/O Infiniband Quad Data Rate cards (21 GBytes/Second) Flash Storage (5TB) for data caching Automatic Storage Indexes
Market Risk Platform High Level Target Architecture for Tier 1 bankOracle ComponentsTrading Apps Risk Engines Counterparty Data Reporting Analytics ModellingMDM FX BI
Rates
Intra Day Data
Historical Data
Analytics Modelling
Credit
Risk Analytics powered by Oracle Business IntelligenceData Integration Ad-hoc Analysis Interactive Dashboards Essbase Analytics Reporting & Publishing Proactive Detection and Alerts Disconnected & Mobile Analytics MS Office & Outlook Integration
Common Enterprise Information ModelIntegrated Security, User Management, Personalization Multidimensional Calculation and Integration Engine Intelligent Request Generation and Optimized Data Access Services
OLTP & ODS Systems
Data Warehouse Data Mart
Essbase
SAP, Oracle PeopleSoft, Siebel, Custom Apps
Files Excel XML
Business Process
Daily Summary Example: Alerts
Alerts to show where immediate action is required
Daily Summary Example: Risk Measures
Dashboard display of measures and trends for the day
Daily Summary Example: Shortcuts
Shortcut links to reports
Daily Summary Example: Spatial
Data availability
Daily Summary Example: Summary
Daily Summary Example: Drill to Detail
Risk Reporting Example: Key Risks
Risk Reporting Example: RCSA Dashboard
Risk Reporting Example: CRO Dashboard
Risk Reporting Example: LOB Dashboard
Risk Reporting Example: Market Risk
Comprehensive Heterogeneous Data Integration PlatformBest-in-class Platform for Major Data Integration Requirements
Exadata V2 Sun Oracle Database Machine
Sun Oracle Database Machine Grid is the architecture of the future Highest performance, lowest cost, fault tolerant, scalable on demand
Oracle Database Server Grid 8 compute servers 64 Intel Cores 576 GB DRAM
Exadata Storage Server Grid 14 storage servers 100 TB raw SAS disk storage or 336 TB raw SATA disk storage
InfiniBand Network 40 Gb/sec unified server and storage network Fault Tolerant
5TB flash storage!
Sun Oracle Database Machine Extreme Performance
Oracle Database Server Grid Millions of transactions per minute Tens of millions of queries per minute Billions of rows per minute
Exadata Storage Server Grid 21 GB/sec disk bandwidth 50 GB/sec flash bandwidth 1 million I/Os per second
InfiniBand Network 880 Gb/sec aggregate throughput
Semiconductor Cache HierarchyMassive throughput and IOs through innovative Cache Hierarchy Database DRAM Cache 400GB raw capacity Up to 4TB compressed user data 100 GB/sec
Exadata Smart Flash Cache 5TB raw capacity Up to 50TB compressed user data 50 GB/sec raw scan 1 million IO/sec
Exadata disks 100TB or 336TB raw Up to 500TB compressed user data 21 GB/sec scan 50K IO/sec
Sun Oracle Database Machine Hardware Improvements Same architecture as Exadata V1 Database Machine Same number and type of Servers, CPUs, Disks
Faster
Latest Technologies80% Faster CPUs 100% Faster Networking 50% Faster Disk Throughput 200% Faster Memory Xeon 5500 Nehalem 40 Gb InfiniBand 6 Gb SAS Links DDR3 DRAM 600 GB SAS Disks 2 TB SATA Disks
New
Bigger
33% More SAS Disk Capacity 100% More SATA Disk Capacity 125% More Memory 100% More Ethernet Connectivity
72 GB per DB Node 4 Ethernet links per DB Node
Plus Flash Storage!
Scale Performance and Capacity
Scalable Scales to 8 rack database machine by just adding wires
Redundant and Fault Tolerant Failure of any component is tolerated Data is mirrored across storage servers
More with external InfiniBand switches Scales to hundreds of storage servers
Multi-petabyte databases
Drastically Simplified Deployments Database Machine eliminates the complexity of deploying database systems Months of configuration, troubleshooting, tuning
Database Machine is ready on day one Pre-built, tested, standard, supportable configuration
Runs existing applications unchanged
Extreme performance out of the boxMonths to Days
Best Machine for Data Warehousing
Best Data Warehouse Machine Massively parallel high volume hardware to quickly process vast amounts of data Exadata runs data intensive processing directly in storage
OLAP
Most complete analytic capabilities OLAP, Statistics, Spatial, Data Mining, Real-time transactional ETL, Efficient point queries
ETL
Powerful warehouse specific optimizations Flexible Partitioning, Bitmap Indexing, Join indexing, Materialized Views, Result Cache
Data MiningNew
Dramatic new warehousing capabilities
Exadata Database Processing in Storage Exadata storage servers implement data intensive processing in storage Row filtering based on where predicate Column filtering Join filtering Incremental backup filtering Storage Indexing Scans on encrypted data Data Mining model scoring
New
10x reduction in data sent to DB servers is common No application changes needed Processing is automatic and transparent Even if cell or disk fails during a query
Simple Query ExampleWhat were my sales yesterday? Optimizer Chooses Partitions and Indexes to Access
Exadata Storage Grid
Oracle Database GridSelect sum(sales) where Date=24-Sept
Scan compressed blocks in partitions/indexes Retrieve sales amounts for Sept 24
SUM
10 TB scanned 1 GB returned to servers
Exadata Hybrid Columnar Compression Data is grouped by column and then compressed Query Mode for data warehousing Optimized for speed
New
10X compression typical Scans improve proportionally
Archival Mode for infrequently accessed data Optimized to reduce space 15X compression is typical Up to 50X for some data
Flash60
Query Throughput
Query Throughput with Flash
Flash storage more than doubles scan throughput 50 GB/sec
50 40 30
GB/sec Uncompressed Data
Query Throughput
50
Flash21
Combined with Hybrid Columnar Compression Up to 50 TB of data fits in flash Queries on compressed data run up to 500 GB/sec
20
11.410 0
7.5HITACHIUSP V
10
Disk
TERADATA2550
NETEZZA SUN ORACLE
TwinFin 12 Database Machine
In-Memory Parallel QueriesQphH: 1 TB TPC-H1,166,976 1,018,321
New
One Sun Oracle Database Machine rack 400GB of DRAM usable for caching
Exadata Hybrid Columnar Compression enables 4TB data in DRAM Database release 11.2 introduces parallel query processing on DRAM cached data Harnesses DRAM capacity of entire database cluster for queries Technology for world record benchmarkExasol Oracle Faster than specialized inmemory warehouse databases ParAccel
315,842
DRAM has 100x more bandwidth than Disk
Source: Transaction Processing Council, as of 9/14/2009: Oracle on HP Bladesystem c-Class 128P RAC, 1,166,976 QphH@1000GB, $5.42/QphH@1000GB, available 12/1/09. Exasol on PRIMERGY RX300 S4, 1,018,321 QphH@1000GB, $1.18/QphH@1000GB, available 08/01/08. ParAccel on SunFire X4100 315,842 QphH@1000GB, $4.57 /QphH@1000GB, available 10/29/07.
Exadata Storage IndexTransparent I/O Elimination with No Overhead TableA B C D 1 3 5 5 8 3 Min B = 3 Max B =8
New
Index
Exadata Storage Indexes maintain summary information about table data in memory Store MIN and MAX values of columns Typically one index entry for every MB of disk
Min B = 1 Max B =5 Eliminates disk I/Os if MIN and MAX
can never match where clause of a query Completely automatic and transparent
Select * from Table where B