Embed Size (px)
Citation preview
The Linux Storage People
Simple
Fast
Massively Scalable
Network Storage
CoraidEtherDrive® Storage
The Linux Storage People
• Coraid developing a simple storage protocol to connect low cost disks with low cost networking.
• The protocol is open and now included in the Linux kernel (2.6.11+).
The Linux Storage People
ATA-over-Ethernet (AoE)(A simple storage protocol)
AoE
Ethernet
PHY
IP
TCP
iSCSI
• AoE sits directly on top of Ethernet
• AoE is not “routed” therefore no IP or iSCSI layers are required
• AoE is simple, secure and requires less processing
The Linux Storage People
Network ServicesDrivers
OtherDrivers
OtherDrivers
OtherDrivers
OtherDrivers
LinuxKernel
Daemons&
Services
EthernetDriver
MAC/PHY
Device Drivers
Disks
Block Device Drivers
Memory Management
Process Management
File System Control
Networking
Other Block Devices
AoE InitiatorDriver Disk
DriversOther
Drivers
AoE Target Devices
AoE bridges Block Devices to Ethernet.
The Linux Storage People
File System (EXT3, JFS, XFS, GFS, etc.)
Logical Volume Manager (LVM, LVM2)
RAID Software (RAID tools, MDADM)
Block Devices
AoE Devices
Typical Linux Storage Stack
Optional
Optional
The Linux Storage People
Networked Storage Systems
The Linux Storage People
EtherDriveEtherDrive
Ethernet
Client LAN
AoE
• Simply connect storage and server with Ethernet
• Server and storage communicate using AoE protocol• Server discovers the new EtherDrive block storage devices in its /dev directory
/dev/etherd/e1.4
Shelf #1
Slot #4
Typical System Configurations
SERVERSERVER
The Linux Storage People
Client LAN
• Using standard Ethernet a storage LAN is created
• Storage now has unlimited scalability
Storage LAN
Typical System Configurations
EtherDriveEtherDriveEtherDriveEtherDrive
EtherDriveEtherDriveEtherDriveEtherDrive
EtherDriveEtherDriveEtherDriveEtherDrive
SERVERSERVER
The Linux Storage People
SERVERSERVER
EtherDriveEtherDriveEtherDriveEtherDrive
EtherDriveEtherDrive
SERVERSERVER
Client LAN
Storage LAN
Typical System Configurations
EtherDriveEtherDriveEtherDriveEtherDrive
EtherDriveEtherDriveEtherDriveEtherDrive
EtherDriveEtherDriveEtherDriveEtherDrive
SERVERSERVER• Multiple Server can
share the storage pool
• Storage is not captive inside a single server
• In the event of a server failure, the storage can be accessed by another server
The Linux Storage People
SERVERSERVERSERVERSERVER
AoEStorage
Client LAN
• A single AoE device can be partitioned and shared
• A single storage volume can be shared using a cluster filesystem like GFS
Storage LAN
AoEStorage
Typical System Configurations
SERVERSERVER
The Linux Storage People
EtherDrive® SATA+RAID (SR)Storage Appliance
The Linux Storage PeopleEtherDrive® Storage Appliances
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
CED3U15SS
3U
3U 15 SATA Disk Slots
• 3U Chassis with slots for up to 15 SATA disks
• Accepts any SATA disk size
• Up to 11.25TB with 750GB disk drives• Users may buy their own disks, and save money
• Includes RAID controller
The Linux Storage People
AoE Bridge
SATA Disk Controller
RAID Controller
SA
TA
Dis
ks
SA
TA
Dis
ks
SA
TA
Dis
ks
SA
TA
Dis
ks
Ethernet
• Dual GigE Interface
• AoE Protocol Processing
• Internal RAID Controller
• SATA Interface (4 or 15 Disk)
EtherDrive® Storage Appliances include:
EtherDrive® Storage Appliances
• Command Line Interface1. RS232 Serial Port
RS232 Serial Console Port
KVM Console Port
Coraid Ether Console (CEC)
2. KVM Port
3. Coraid Ether Console (CEC) Linux host application program
The Linux Storage People
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
Simple RAID setup
SATA shelf 2>
Shelf #2
Slot # 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
SATA shelf 2>SATA shelf 2>SATA shelf 2>SATA shelf 2>
make 0 raid5 2.0-9make 1 raid5 2.10-13spare 1.14online 0online 1
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
/dev/etherd/e2.0 /dev/etherd/e2.1
EtherDrive® Storage Appliances
ConsolePort
The Linux Storage People
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
Easily grow volumes
SATA shelf 2>
Shelf #2
Slot # 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
grow 0 raid5 2.10-13
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
SA
TA
Dis
k
/dev/etherd/e2.0/dev/etherd/e2.0
RAID 0, 1, 5, 10 supported
EtherDrive® Storage Appliances
ConsolePort
The Linux Storage PeopleEtherDrive® Storage Appliances
Client LAN
• Alarms are reported to the Server’s syslog
EtherDriveEtherDrive
SERVERSERVER
UDP port 514 alarm messageSATA Shelf 2 > disk failure 2.3, RAID5 2.0 degraded
192.168.1.54
205.185.197.30
Shelf # 2
The Linux Storage PeopleEtherDrive® Storage Appliances
Client LAN
• Volume access can be restricted based upon Server MAC address
EtherDriveEtherDrive
SERVERSERVER00-40-CA-6E-74-55
SERVERSERVER20-F0-15-EF-04-FA
RAID5 2.0 RAID5 2.1
• Port assigned VLAN is also an effective zoning means
The Linux Storage People
Coraid Linux NAS GatewayCLN20
The Linux Storage People
Network Attached Storage (NAS)
• NAS is a great application for EtherDrive® Storage
• NFS or CIFS file sharing protocols transport data over
TCP/IP & UDP between clients and the NAS appliance
• NAS is file sharing & file storage over Ethernet
The Linux Storage People
Typical NAS System
• NAS presents a large storage volume created with RAID
`
NASNAS
SERVERSERVER` ` SERVERSERVER
• NAS stores files via CIFS or NFS• Clients and Servers can share file storage on the network
Bob Sue Bill Email Web
NAS1CIFS or NFS
RAIDBob
Sue
Bill
Web
NAS1
The Linux Storage People
NASNAS
Inside NAS• Typical NAS appliance consist of a processor, memory, RAID controller,
Disks and network interface
RAID Controller
Processor Memory
Network Interface
Disk Disk Disk Disk
LAN
Block Storage Access
CIFS/NFS File Sharing
NASAppliance
• CIFS or NFS file requests are received through the network interface
• Processor services request by accessing the RAID disks as block storage
• Processor assembles blocks of data into files for export to the network
1011 1110 0011 0010
ANYFILE.xxx
The Linux Storage People
Expanding NAS
• When a NAS appliance is full, what can you do?
`
NASNAS
SERVERSERVER` ` SERVERSERVER
• Add another NAS• But directory structure makes finding files more complicated
Bob
Sue
Bill
Web
NAS1
Bob Sue Bill Email Web
NAS1
Bob
Sue
Bill
Web
NAS2NASNAS
NAS2
FULL
The Linux Storage People
RAID Controller
Disk Disk Disk Disk
A Better NAS• Remove the storage from the NAS
Processor Memory
Network Interface
LAN
Block Storage Access
CIFS/NFS File Sharing NASGateway
• Access storage with ATA-over-Ethernet (AoE) protocol
AoEStorage
Block Storage Access via AoE
RAID Controller
Disk Disk Disk Disk
NASAppliance
AoE
Network Interface
Network Interface
The Linux Storage People
EtherDrive® Storage with NAS
• NAS with scaleable storage
` SERVERSERVER` ` SERVERSERVER
• NAS Gateway provides CIFS & NFS service• Storage expands without complicating directory
Bob
Sue
Bill
Web
NAS
Bob Sue Bill Email Web
NAS
EtherDriveEtherDrive
NAS GATEWAYNAS GATEWAY
CIFS or NFS
EtherDriveEtherDrive
Bob
Sue
Bill
Web
AoE
The Linux Storage People
• A more reliable NAS with EtherDrive® Storage
` SERVERSERVER` ` SERVERSERVER
• Fail-Over NAS Gateway provides protection• EtherDrive® Storage is not captive to a single NAS head
EtherDriveEtherDrive
NAS GATEWAYNAS GATEWAY
EtherDriveEtherDrive
NAS GATEWAYNAS GATEWAY
EtherDriveEtherDrive
• Efficient high speed Backup and Restore is possible
DATA DATA COPY
EtherDrive® Storage with NAS
The Linux Storage People
Simple
Fast
Massively Scalable
Network Storage
EtherDrive® Storage
