Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
®
IBM Software Group
© 2007 IBM Corporation
Introducing IBM Balanced Warehouse
Balanced Configuration Units (BCU)
David Cope
EDW Architect – Asia Pacific
IBM Software Group
2
Today’s Market
� Warehouse are larger and more complex than ever
� Consolidation is a key strategy
� The need for business agility is a key strategy. Warehouses are now architected for agility as opposed to performance
� Technology is only one part of the solution – business/operational processes and people process also need to be addressed for a successful single source of truth
� Warehouse become critical business assets with 24 x 7 requirements and DR strategies
� TCO remains an issue but now in the care and feeding of the warehouse, not the acquisition
� In comes the “appliance”………
�Customers of the warehouse perceive delivery as “simplified”
�Extreme cost pressures
�Extreme delivery pressures
IBM Software Group
3
What did the Appliance Really do to the Market?
� Forced a customer requirement of better time to value to be focused on by the
DW vendors
� Highlighted the requirement for managing cost in the warehouse environment
� Accelerated the growth of Linux as an alternative in data warehousing
� Allowed customers to choose commoditized components at reduced costs
� Gave the customers the luxury to focus on the important things
�Doing it yourself is “so out”
�Focus on delivery of information, not integration
IBM Software Group
4
Assisting Customers in Riding the Waves of Change
� On the first round of “appliance” Customers told us:
�Fast time to value
�Totally integrated
�Single source of support
�Tested and validated
�Best Practices included
�Answer = Balanced Warehouse (BCU)
� But we always knew appliances would run into the same issues as traditional
warehouses
�Mixed workload
�High concurrency
�Embedded analytics
�Real-time/Right Time
�Integration of all information (structured or something else)
�This was IBM’s Vision from the start with the Balanced warehouse
� Balanced Warehouse + Dynamic Warehousing = Appliance Plus
IBM Software Group
5
Why we say “Appliance Plus”
“Other DW Appliance Characteristics”
� Fast time to market, no component integration
� Single solution and support
� Proprietary architecture
� Excellent TCO
� Fast loading of data but limited on concurrency
� Great for data mart solutions
� Component redundancy, less need for disaster recovery
� Limited scalability
� Limited bundling and integration of analytics, capability of handling differing data types
Balanced Warehouse
� Fast time to market, no component integration
� Single solution and support
� Industry standard and open architecture
� Excellent TCO
� Great performance on load balanced with the ability to handle mixed workloads for right time enterprises
� Multiple entry points and scaling options –enterprise capable building blocks
� Fully redundant, highly available with optional DR
� Great scalability
� Analytics, workload management, modeling and data integration – in the box!!!
IBM Software Group
6
IBM Data Warehouse “Appliance Plus”
� The introduction of the “Data Warehouse Appliance” by smaller vendors set the
stage for defining the requirement as simple, integrated and proprietary.
�It did NOT address the critical challenges most enterprise customers face
� The Balanced Warehouse is the only “DW Appliance” that was introduced with
the vision of solving more challenges than cost and time to market all on an
open systems architecture
�We continue to work on the “real life” challenges of our customers as they consolidate, manage complexity and deliver information real time
� IBM will continue to address the warehouse market more holistically with a
solution focus and an eye to TCO and time to market
Not just a DW Appliance, but an “Appliance Plus “
IBM Software Group
7
Introducing IBM Balanced WarehouseTM
A fast track to warehousing
Simplicity� Predefined configurations for reduced
complexity
� One number to contact for complete solution support
Flexibility for growth
� Add BCUs to address increasing demands
� Multiple on-ramps for different needs
� Reliable, nonproprietary hardware for reusability
Optimized performance
� Preconfigured and certified for guaranteedperformance
� Based on best practices for reduced risk
Balanced Configuration Unit (BCU)
Preconfigured, pretested allocation of software, storage and hardware to support a specified combination of function and scale
IBM DB2®
Warehouse
SIMPLEFLEXIBLE
OPTIMIZED
Better than an appliance
Balanced Warehouse
IBM Software Group
8
What is the IBM Balanced Warehouse™?
Simple:
Reduced Complexity: Ships configured and ‘Ready to go’
Pre-tested: Validated by IBM
Full integration: High Performance DB2 Warehouse delivered ‘load ready’
Predictable, modular growth through the usage of BCU
One number: Single point of contact for all support issues
Flexible
Modular scalability: Grows with your demands, not your vendor’s
Open and Reusable: Ensures that you are not locked in to proprietary limitations
Spectrum of offerings: Multiple on-ramps to warehousing
Ability to plan for and price the growth of your company’s warehousing needs
Optimized:
IBM Balanced Warehouse solutions are thoroughly tested and tuned for performance
Solutions developed through IBM Best Practices of successful client implementations
Low Risk: Guaranteed performance
IBM Software Group
9
Guiding Design Principles for the Solution Offering
� Simplicity, Simplicity, Simplicity !!
� Price to performance ratio
� Balanced performance
� Scalability
� Stability
� Fault tolerance
� High availability
� Ease of installation and implementation
� Packaging density
� Industry-standard components
� Minimize solution permutations
� Modular growth at a predictable cost
p575
AIX
DB2
Linux
x346, e326
IBM Software Group
10
IBM Balanced Warehouse™ – Solution Classes
The IBM Balanced Warehouse™ can be broken down into 3 different solution classes that each target and
serve a distinct warehousing market segment.
C-Class D-Class E-Class
Enterprise SolutionsGrowth SolutionsApplication Solutions
Class Name
Performance
Highly Scalable Enterprise Data Warehouses
Advanced Departmental Data Marts and Smaller Data Warehouses
Entry-point for Smaller Warehousing Applications and Reporting Tools
Scalability High - BCU Very High - BCULimited
Size 1TB to 5TB* 1TB and Up50GB to 1TB
NOTE: * Can scale higher, yet it is recommended to look to an E-Class solution to maximize performance optimization
Users No Hard LimitsUp to 200 Users
Segment
IBM Software Group
11
IBM Balanced Warehouse™ – Offerings 1H 2007
C1000 C3000 D5000 E6000 E7000 E7050
Enterprise SolutionsGrowth SolutionsApplication Solutions
Scalable Units BCU BCU BCU BCU
Operating System
Scaling Increments1TB 1TB 2TB 2TB
DB2 Warehouse
System
StorageEXP3000 DS4800 DS4800 DS8100
x3650 (I) x3455 (A) p5-575 p5-575
Adv, Base or Ent Base or EnterpriseStarter Intermediate
x3650 (I)x3500 (I)
EXP3000
N/AN/A
~ 250GB~ 50GB
SupportBCU Support BCU Support
Solution Name
Novell / Red HatNovellNovellNovell
Performance
Reseller/Distributor Provided
Internal
IBM Software Group
12
IBM Balanced Warehouse™ - Offering Graph With the addition of Application and Growth Solutions, IBM is uniquely positioned to offer complete data
warehousing solutions across the entire warehousing spectrum.
Warehouse Size
Perf
orm
an
ce
C1000
C3000
D5000
E6000 E7000
E7050
Application Solutions Growth Solutions Enterprise Solutions
1TB Limit on C-Class
50GB to 1TB 1TB to 5TB 5TB and up
IBM Software Group
13
D & E Class Data Warehouse Platforms
D Class Appliance
D5000
�Supports 500 GB upwards
�Dedicated Disk
� Low Price Point
� 2 way Building Blocks
E Class Appliance
Linux BCU
�Aimed at 1-2 TB and upward
�Configured for High availability
�Configured for High end Scalability
� 2 way Building Blocks
�Mid Price Point
P-Series BCU
�Aimed for High Data Volumes
� Larger Rate of Growth (bigger building blocks)
�Aimed for High End User Workloads
�Highly available Hardware Architecture
� 8 way Building Blocks
� Provides same Database Architecture with choice of Platform based on Your Business
Needs
� Provides Scalable Growth Path with a Low Cost entry point upgradeable as your
Business requirements Change
IBM Software Group
14
IBM Balanced Warehouse™
1TB Balanced WarehouseIncludes:1 Data BCU – 2 Dual Core1 Admin BCU
Size: 1TB
Total Size: 1TB
E6000
E7000
Sys
tem
xS
ys
tem
p5 2TB Balanced Warehouse
Includes:1 Data BCU – 8 POWER5+1 Admin BCU
Size: 2TBTotal Size: 2TB
1TB BCUIncludes:1 Data BCU – 2 Dual Core
Size: 1TB
Total Size: 2TB
1TB BCUIncludes:1 Data BCU – 8 POWER5+
Size: 2TBTotal Size: 4TB
Add 1 E6000 BCU
Add 1 E7000 BCU
+
+
Sizing for the System x based Balanced Warehouses follows single increments of 1TB Data BCUs (1,2,3....n)
Sizing for the System p based Balanced Warehouses follows single increments of 2TB Data BCUs (2,4,6....n)
System x Data BCUs have 2 dual core processors (AMD) for a 4-way node. Each balanced piece of storage is 250GB.
4 * 250GB = 1TB per node
System p Data BCUs have 8 processors (POWER5+) for a 8-way node. Each balanced piece of storage is 250GB.
8 * 250GB = 2TB per node
Scaling Example: System x vs. System p
IBM Software Group
15
IBM Balanced Warehouse™
Up to 250/1TB
No
~1400MB/sec
No
2 x Int. SAS @146GB +
2 x EXP3000
24 x SAS @146GB
DWE 9.1.2
SLES10
16GB (4GB/core)
2 x Intel EM64T Dual-core
x3650
YesCall home for storage
YesServer failure HA
Up to 200/800GB
620 MB/sec
DS4800
56 x 73GB 15k RPM FC disk
DWE 9.1.1
SLES9 SP3
16GB (4GB/core)
2 x AMD Opteron Dual Core
x3455
GB per BPU/BCU
Maximum Seq. I/O per BCU
Storage
DB
OS
RAM
Processor
System
E6000D5000
Linux Solution Technical Comparison (D5000 vs. E6000)
IBM Software Group
16
The Balanced Configuration Unit (BCU)
GOAL: Reduce time to business value and total cost of ownership (TCO) through a reusable and validated approach that promotes better consistency and higher quality for data warehouse implementations.
Data Warehouse
…
AIX
AIX
BPU BPU BPUBPU
BPU BPU BPUBPU
Memory
CPUCPU CPUCPU CPUCPU CPUCPU
CPUCPU CPUCPU CPUCPU CPUCPU
AIX
AIX
BPU BPU BPUBPU
BPU BPU BPUBPU
Memory
CPUCPU CPUCPU CPUCPU CPUCPU
CPUCPU CPUCPU CPUCPU CPUCPU
AIX
AIX
BPU BPU BPUBPU
BPU BPU BPUBPU
Memory
CPUCPU CPUCPU CPUCPU CPUCPU
CPUCPU CPUCPU CPUCPU CPUCPU
BCU 1 BCU 2 BCU n
� Minimum replicable HW/SW stack necessary to start or expand the infrastructure of a Business Intelligence (BI) system.
� Provides a scalable performance ratio of disk I/O to memory to CPU to network
� A prescriptive approach for building data warehouse infrastructure
IBM Software Group
17
Guiding design principles for the BCU
� A number of design principles have been used in guiding the design of the BCU, which include:
�Balanced performance
�Scalability
�Stability
�Fault tolerance
�High availability
�Minimize solution permutations
� BCU Touchpoints:
�Database to IO Subsystem
�Database to Memory
�Database to other parts of Database
�Database to Application (ETL)
�Database to End-User Tool
The BCU is configured for an Enterprise BI solution.
As such, price performance is a high priority factor, but not the only
driving factor.
IBM Software Group
18
The Balanced Partition Unit (BPU)
� A BPU is defined as 1 DB2 Data
Partition and the resources
associated with it.
� DB2 Data Partition
� DB2 Agents
� CPUs for DB2 Agents
� Memory for DB2 Agents
� Storage for Partition Data
CPU
I/O Channels
BPU
Memory - Bufferpool
Communication
1 DB2 Data Partition
DB2 Agents
Storage Capacity
IBM Software Group
19
The Balanced Configuration Unit (BCU)
� A single operating system image
provided by an individual server
and the resources associated with
it. A single OS imagine will
support multiple BPUs.
� 8 BPUs
� 32GB memory
� 8 Power5 processors
� Single AIX OS image
� Dedicated Modular Storage
8 CPUs
I/O Channels
Power5 BCU
32GB of memory
Communication
8 BPUs
OS image
Storage Capacity
IBM Software Group
20
AIX Data BCU TypesC
om
mo
n
Co
mm
on
C
hara
cte
risti
cs
Ch
ara
cte
risti
cs
� One p5-575 server
� Eight 1.9 GHz CPUs at 10.4 RBI performance
� Eight DB2 database partitions (data BPUs)
� 32 GB memory (32 DDR1 1GB DIMMs)
� Two Gb Ethernet adapters for DB2 FCM interconnect network
� 2 of the 4 integrated Gb Ethernet ports used for corporate network connectivity
� 2 integrated HMC ports are used for the HMC Console
� 1 of the 2 sides of a Feature Code 5794 Remote I/O drawer with 10 PCI-X slots
Active DW Mixed Workload Larger
Archive
Traditional DW Batch Refresh Query Window
Workload
24,000
12,000
I/O Trans
(IOPS)
≤ 1200
≤ 600
I/O
Thruput
(MB/sec)
1601282BCU BCU
#2#2
80641BCU BCU
#1#1
Usable
Disks
Qty
EXP710 Disk Enclosures
Qty
Fibre
Channel
Adapters
for Disk
DS4800
Storage
Servers
Qty
IBM Software Group
21
Example Metrics for Data BPU Calculation
� Per DB2 Partition / Processor (1 DB2 partition per CPU)
� 150 – 250 GB Raw Data – typically 150GB Active Raw Data
� 450 – 1000GB Useable Disk
� RoT: 3-4 x Raw Data
� Mainstream disk choices; 36Gb, 73Gb
� 146Gb only where large amount of the data is inactive
� 4 GB Memory
� Assuming DB2 v8 and 64 bit
� Database network (FCM) – circa 20 MB/sec
� RoT: 80MB/sec per GB Ethernet port
� Equates to 1 GB Ethernet Adapter per 4 CPUs
� Storage Throughput – circa 80 - 160MB/Sec
� RoT: 10MB/sec random uncached read per disk
� Equates to 1 x 2GB FC card per 2 CPUs or 1 x per SCSI 320 card per 2 CPUs
� Storage Transactions – circa 1200 – 2400 IOPS
� RoT: 150 concurrent random uncached read IOPS per disk.
� Minimum 8 disks at 15k RPM; > 12 is more balanced
� Ensure storage Server/Controller capacity to handle throughput and transactions
Originate from “Typical/Average”
Workloads
IBM Software Group
22