Upload
ira-bates
View
304
Download
8
Embed Size (px)
Citation preview
© 2005 EMC Corporation. All rights reserved.
The Complexity of Information Management The Complexity of Information Management
第一章 信息管理的复杂性
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 2
本章目标及内容 信息,对现代企业管理至关重要,信息不仅仅能够创
造新的商业机会,同时,充分利用信息也能为企业带来市场中的竞争优势。本章从信息的重要性开始讲起,介绍在海量信息管理需求下的存储解决方案与数据中心的基础构架,从而建立起存储的基本概念。
本章内容包括三个部分:1.1 满足现代需求的数据存储 ( Meeting Today’s Data Storage Needs )1.2 数据存储的解决方案 ( Data Storage Solutions )1.3 数据中心的基础构架 ( Data Center Infrastructure )
© 2005 EMC Corporation. All rights reserved.
Meeting Today’s Data Storage Needs Meeting Today’s Data Storage Needs
Module 1.1
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 4
Meeting Today’s Data Storage Needs
Upon completion of this module, you will be able to:
Describe who is creating data and the amount of data being created
Describe the value of data to business
Describe the challenges in data storage and data management
List the solutions available for data storage
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 5
Data Creation
Data is being created at an ever increasing rate– Data creation/generation is growing at a rate in excess of 50% year-
over-year
– The need to store the data over longer periods of time with improved accessibility is also growing
Information Technology (IT) budgets are responding– IT budgets typically have to account for expenditure on Servers,
Networks, Storage, Personnel, etc.
– To keep up with data storage needs, IT expenditure on Storage has increased proportionally
– It is estimated that about 40% of the IT expenditure is Data Storage related
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 6
Data Creation: Individuals
What data is created by Individuals?
Examples include:– Photos
– Documents
– Spreadsheets
– Video
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 7
Data Creation: Individuals
Where is this data stored?
Storage can be in:– Cameras
– MP3 players
– Laptop hard drives
– CDROM/DVDs
– USB drives
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 8
Data Creation: Business
What data is created by a business?
Examples of data created by a business include:– Product data: inventory, description, pricing, availability, sales
numbers and projections
– Customer data: orders, shipping details
– Account data: banking, financial services industry
– Medical data: health care providers, insurance industry, hospitals
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 9
Data Creation: Business
Where is this data stored?
Business data can be stored on:– Employee workstations
– Servers
– Disk arrays
– Tapes
– CDROM/DVDs
– Off-site libraries
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 10
Value of Data to a Business: Information
What do businesses “do” with the data they collect?
They turn it into “information”.
Examples of information include:– Buying habits and patterns of customers
– GPS locations of delivery trucks
– Health history of patients
– Locations where a credit card is used
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 11
Value of Information to a Business
Identifying new business opportunities. For example:– Buying/spending patterns: Internet stores, retail stores, supermarkets.
– Customer satisfaction/service: tracking shipments & deliveries.
Identifying patterns that lead to changes in existing business. For example:– Reduced cost: delivery service optimizing utilization of vehicles and gas.
– New products: MP3 player speaker systems.
– New services: security alerts for “stolen” credit card purchases.
– Targeted marketing campaigns: communicate to bank customers with high checking account balances about a special savings plan.
Creating a competitive advantage!
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 12
Value of Individual Data to a Business
What data, created by individuals, might be valuable to a business?
Examples of business value from individuals’ data include:– On-line resume storage and management service .
– On-line photo storage and organizer.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 13
Information Availability - Downtime is Expensive
Millions of US Dollars per Hour in Lost Revenue
6.5
3.6
2.8
2.6
2.0
1.6
1.6
1.5
1.3
1.2
1.1
Retail brokerage
Point of sale (POS)
Energy
Credit card sales authorization
Telecommunications
Call location
Manufacturing
Financial institutions
Information technology
Insurance
Retail
Source Meta Group, 2005
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 14
Types of Data
Structured
UnstructuredE-mail & Attachments
PDFs
Checks
X-rays
PaperDocumentsRich Media
Web PagesAudio & Video
RecordsInvoices
Manuals
Claims
Contracts Instant Messages
Forms
Images
XML
Rows and Columns
Data can be categorized as either structured or unstructured data.
Over 80% of enterprise information is unstructured. (Fulcrum Research)
What has been the traditional approach to storing all this data?
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 15
Storing Data: An Evolution
Centralized: terminals connected to a Mainframe computer which had connectivity to internal or external storage devices (disks, tapes).
Decentralized: With the advent of Open Systems, business units within an enterprise adopted a Client-Server model.
Centralized: Networked Storage (is the current “best practice” model being used in IT).
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 17
Worldwide Information Growth
~60% Average Growth Rate
>70% in 2005
0%
40%
60%
80%
100%
120%
2001 2002 2003 2004 2005e
Annual Growth of Data Stored on Disk Arrays
Data Source: IDC
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 18
Key Information Management Challenges
Planning for capacity growth
Classifying data
Address data availability
Security
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 19
Summary
Topics covered in this lesson included:
Who is creating data and the amount of data being created
The value of data to business
The challenges in data storage and data management
The solutions available for data storage
© 2005 EMC Corporation. All rights reserved.
Data Storage Solutions Data Storage Solutions
Module 1.2
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 21
Data Storage Solutions
Upon completion of this module, you will be able to:
List the common storage media and solutions.
Describe the three common storage environments.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 22
Lesson: Storage Solution Alternatives
Upon completion of this lesson, you will be able to:
Describe the different media and solutions available to address data storage needs of a business.
Describe the role of each solution in the overall data storage needs.
Describe the advantages of disk arrays.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 23
Common Data Storage Media
Tape Library: A collection of tape drives and tapes
Jukeboxes: A collection of optical disks and drives
Disk Arrays: A collection hard disks
Each solution addresses specific needs for data storage and management.– Tape Library – Backup/Restore; Archival of data
– Jukeboxes – Typically to store non-changing content over long periods of time
– Disk Arrays – To store data that has to be immediately accessible and on-line
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 24
Tape Storage Systems
• Tape based storage is relatively inexpensive, compared to disk arrays.
• They served as primary storage solutions in the early days.
• Tape drives use Read/Write heads to record bits of data onto magnetic material on the tape surface.
• This technology continues to evolve, providing higher storage capacity, greater reliability, and improved performance.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 25
Storing Data on Tape
Data is recorded sequentially from the beginning to end, one byte after another.
Because data is stored linearly along the length of the tape, random access to specific bits of data is slow and time consuming. This severely limits tape as a medium for real-time, rapid access to data.
Tapes cannot be shared among multiple users or applications.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 26
Optical Data Storage
• Popular in small, single-user computing environments.
• Frequently used by individuals to store and share data, or as backup solution.
• Also used as distribution medium for applications, or as a means of transferring small amounts of data from one self-contained system to another.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 27
Disk Based Storage
From a historical perspective, we will explore the following disk based storage solutions:
• DASD: Direct Access Storage Device
• JBOD: “Just a Bunch Of Disks”
• Disk Arrays
• “Intelligent” Disk Arrays
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 28
Types of Disk Systems: DASD
Mainframe
Disk
• Introduced by IBM in 1956.
• The ‘oldest’ of the techniques for accessing disks from a host computer.
• Disks are accessed directly by a single host, historically a mainframe system.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 29
Types of Disk Systems: JBOD
Host
Disk Disk Disk Disk Disk
• Multiple physical disks in an external cabinet.
• Array connects to a single server only.
• Provides higher storage capacity with increased number of drives.
• Data not protected by JBOD
Array
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 30
Types of Disk Systems: Disk Arrays
Host B
Disk 1 Disk 2 Disk 3 Disk 4 Disk 5
Disk ArrayController
Array controllers for optimized I/O operations and RAID (Redundant Array of
Independent Disks) calculations.
Higher speed interconnects between drives than JBODs.
Multiple host I/O channels.
Can be partitioned to allow each host to access its own set of drives.
Host A Host C
Host A Host B Host C
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 31
Types of Disk Systems: “Intelligent” Disk Arrays
Disk 1 Disk 2 Disk 3 Disk 4 Disk 5
Disk ArrayController
Highly optimized for I/O processing.
Cache for improving I/O performance.
Operating environments provide: Intelligence for managing Cache,
Array resource allocation (Logical Unit),
Host access to Array resources,
Connectivity for heterogeneous Hosts
Host BHost A Host C
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 32
Lesson: Summary
Topics covered in this lesson included:
Tape
Optical
Disks – DASD
– JBOD
– Disk Arrays
– Intelligent Disk Arrays
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 33
Lesson: Storage Environment
Upon completion of this lesson, you will be able to:
Describe Direct Attached Storage (DAS) features.
Describe Storage Area Network (SAN) features.
Describe Network Attached Storage (NAS) features.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 34
Direct Attached Storage – DAS
Client 2
Server AApplication A
Server BApplication B
Server CApplication C
Disks for Server A
Disks for Server B
Disks for Server C
Client 3
Client 1
Local Area
Network
SCSI
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 35
Network Attached Storage – NAS
Disks for File System A
Disks for File System B
NAS Device AFile System A
NAS Device BFile System B
Internal/External connectivityto disks or arrays
Server AFile System A
Server BFile System B
Client 1
Client 2
Client 3
Local Area
Network
Linux
Windows
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 36
Storage Area Network – SAN
Client 2
Client 3
Client 1 Server AApplication A
Server BApplication B
Server CApplication C
Disks for Server A
Disks for Server B
Disks for Server C
SANLocal Area
Network
Disk Array
Fibre Channel
FC Switch
Data Block
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 37
Lesson: Summary
Topics covered in this lesson included:
Direct Attached Storage (DAS) features.
Network Attached Storage (NAS) features.
Storage Area Network (SAN) features.
Differences: Connectivity
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 38
Module Summary
Key points covered in this module:
The three types of data storage media.
The three storage environments.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 39
Check Your Knowledge
What are the three most common storage media?
What are the three types of disk-based solutions?
What are the advantages of a disk-based storage solution?
What are the three storage environments?
© 2005 EMC Corporation. All rights reserved.
Data Center InfrastructureData Center Infrastructure
Module 1.3
Computing Center Data Center
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 41
Data Center Infrastructure
Upon completion of this module, you will be able to:
List the five core elements of a Data Center infrastructure.
Describe the role of each element in supporting business activities.
Describe the requirements of storage systems for optimally supporting business activities.
Describe the challenges and activities in managing the storage systems of a data center.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 42
The Core Elements
Applications
Databases – Database Management System (DBMS) and the physical and logical storage of data
Servers/Operating Systems
Networks (between clients and servers or between servers and storage)
Storage Arrays
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 43
An Example: order entry system
Consider an order processing system consisting of:
Application for order entry.
Database Management System (DBMS) to store customer and product information.
Server/Operating System on which the Application and Database programs are run.
Networks that provide– Connectivity between Clients and the Application/Database Server– Connectivity between the Server and the Storage system.
Storage Array.
Local AreaNetwork
Storage Area Network
Storage Array
Client
Server
Application User
InterfaceDatabase
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 44
An Example ..Closer Look
A customer order is entered via the Application User Interface on a client.
Local AreaNetwork
Storage Area Network
Client
Server
Application User
Interface
Storage Array
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 45
An Example ..Closer Look
A customer order is entered via the Application User Interface on a client
The client accesses the Server over a Local Area Network.
Storage Area Network
Client
ServerStorage Array
Local AreaNetwork
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 46
An Example ..Closer Look
A DBMS uses the operating system on the server to read and write this data to the physical location on a disk.
Storage Area Network
Client
Server
O/S and DBMS
Storage Array
Local AreaNetwork
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 47
An Example ..Closer Look
A DBMS uses the operating system on the server to read and write this data to the physical location on disk.
A Network provides the communication link between the server and the storage array, and transports the read/write commands and data between the server and the storage array.
Storage Area Network
Client
ServerStorage Array
Local AreaNetwork
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 48
An Example ..Closer Look
A DBMS uses the operating system on the server to read and write this data to the physical location on disk.
A Network provides the communication link between the client and the server, and transports the read/write commands and data between the server and the storage array.
A storage array receives the read/write commands and data from the server and performs the necessary operations to store the data on the physical disks.
Storage Area Network
Client
ServerStorage Array
Local AreaNetwork
Database
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 49
An Example.. Optimal Order Processing
The Application should be optimized for fast interaction with the DBMS.
The tables in the Database should be constructed with care so that the number of read/write operations can be minimized.
The Server should have sufficient CPU and memory resources to satisfy Application and DBMS needs.
The Networks should provide fast communication between Client and Server, as well as Server and Storage Array.
The Storage Array should service the read/write requests from the Server for optimal performance.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 50
An Example.. A Final Look at Data Access
When the DBMS receives a request from the Application:
It first searches the Server memory. If data is found there, the operation takes, perhaps, a millisecond.
If not, it then uses the Operating System to request the data from the Storage Array.
Dedicated high speed networks transport this request to the Storage Array.
Intelligent Storage Arrays can deliver the requested data within a few milliseconds. They are also typically configured to protect data in the event of drive failures.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 51
Key Requirements of (Intelligent) Storage Systems
AvailabilityAvailability
Data IntegrityData Integrity SecuritySecurity
CapacityCapacity
ScalabilityScalability
PerformancePerformance
ManageabilityManageability
Applicable to all elements of the Data Center Infrastructure.They are qualities that must exist for successful use of data.
Downtime per year99.%=3.7days99.9%=9hours99.99%=53minutes99.999%=5minutes
Authorized users
Physically relocate or logically reassign resources to supporting different critical business needs. (DBMS)
Smoothly increase or decrease resources(apps, DB, server, storage) as needed (business grow).
Meeting user expectations fortimeliness and response toI/O requests.More server, one storage array
I/O chain checks
Flexible to configureand monitor thestorage system.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 53
Some Constraints to Meeting the Requirements
Constraints include:
Cost : the budget
Physical Environment: the site
Maintenance and Support: human resource
Compliance – Regulatory & Legal: the business rule
Hardware and Software infrastructure
Interoperability and Compatibility
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 54
Management Activities
Data Center management activities include:
Provisioning / Capacity Planning / Resource Planning
Monitoring
Reporting
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 55
Monitoring
Performance
Security
Data Protection
Utilization
– Attempts and types of intrusions
– Data transfers or transactions per minute
– Hardware errors that are detected and corrected
– Numbers of users and resource use (CPU, Memory, Storage)
– Network traffic
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 56
Reporting
Utilization
Performance
Reports help in trend analysis. For example, periodic
reporting on disk space utilization enables predicting storage
capacity requirements in the future. Many organizations use
reporting as a chargeback system to recover cost of providing IT
infrastructure services to their internal customers. In turn, the
users may be held accountable for the resources they consume.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 57
Provisioning: provide the hardware, software and resources needed to run the Data Center
Buy hardware
Buy software
Installing hardware and software
Training resources
Performance tuning the interrelated system
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 58
Capacity Planning
Understanding the business model – helps estimate the growth and support needs for the business. Understanding the business will help anticipate the data needs and growth in data capacity requirements.
Understanding the data life cycle for the business – helps identify the various stages of data in the business, requirements for migrating data to archived backup, and anticipate or plan for the increasing capacity needs.
Understanding changes in storage technology – ability to introduce new and more efficient storage methods to meet future capacity needs and manage costs.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 59
Resource Planning
Understanding the procedures and tasks in the data center.
Changes in policy, procedures, or business needs.
Availability of qualified candidates.
Ability to train data center resources.
Sufficient budget to supply staff 24 X 7.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 60
Module Summary
Key points covered in this module:
The five core elements of a Data Center Infrastructure.
The role of each element in supporting business activity was explained with an example of an order entry process.
The importance of an Intelligent Storage Array.
Key requirements of storage systems to support business activities as well as some of the constraints.
Management activities in a data center operation with focus on storage systems.
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 61
Check Your Knowledge
What are the five core technology elements of the Data Center Infrastructure?
What are the seven requirements of storage technology?
What are the common management activities in a Data Center?
© 2005 EMC Corporation. All rights reserved. Storage Systems Architecture - Introduction - 62
Who is the industry leader?
Major companies (IBM, HP, Sun, Microsoft,
Hitachi, EMC, Network appliance, even
Cisco) are competing for storage market!