53
1 IBM BladeCenter Fundamentals Introduction Unit objectives After completing this unit, you should be able to: List the major elements common to the IBM BladeCenter Describe the key aspects of compatibility between BladeCenter models Identify the components providing redundancy in the BladeCenter chassis Match the power components necessary to support varying BladeCenter resource configurations List the power input requirements for the BladeCenter models Describe the common cooling components used in the BladeCenter chassis Describe the supported disk configurations for the BladeCenter S 2 Dynamic infrastructure: Key points Enables visibility, control, and automation across all business and IT assets Transforms assets into higher value services Highly optimized to achieve more with less Addresses the information challenges Leverages flexible sourcing like clouds 3 Dynamic infrastructure: IBM BladeCenter Reduce cost x86 industry-leading performance, virtualization, energy efficiency, and scalability Manage risk Resilient architectures and management tools for security and high availability Improve service Performance to drive new demanding application workloads 4 BladeCenter chassis components BladeCenter chassis BladeCenter chassis midplane Power Cooling Unit summary 5 IBM BladeCenter family Common blades, common switches, common management IBM BladeCenter E Highest density, super power efficient IBM BladeCenter H Ultra high performance, and I/O flexibility BladeCenter chassis by model and type IBM BladeCenter HT Highly rugged, Telco, AC/DC, NEBS, air filtration IBM BladeCenter T Highly rugged, Telco, AC/DC, NEBS, air filtration 7U design Up to 14 blade bays Up to 4 switch fabrics Low cost Low power Support 10 GB Uplinks Support 8 Gb FC 9U design Up to 14 blade bays Up to 10 GB midplane I/O flexibility up to 8 switch bays Support 3 0mm blades w/ up to 8 ports Support 10 GB Ethernet Support 8 Gb FC Support 4x InfiniBand 8U design Up to 8 blade bays Up to 4 switch fabrics AC or DC models NEBS compliant Rugged Support 10 GB uplinks Support 8 Gb FC Telco, military, dirty floor 12U design Up to 12 blade bays AC or DC models I/O flexibility up to 8 switch bays NEBS compliant Rugged Up to 10 GB midplane Support 10 GB Ethernet Support 8 Gb FC Support 4x InfiniBand Telco, military, dirty floor IBM BladeCenter S Extending the benefits of BladeCenter outside the datacenter 7U design Up to 6 blade bays Integrated storage Up to 3 switch fabrics Lowest cost Lowest power (100 - 240v) 950w/1450 AC auto- sensing Support 10 GB Uplinks Support 4 GB FC 6

IBM Bladecenter

  • Upload
    dsunte

  • View
    179

  • Download
    1

Embed Size (px)

DESCRIPTION

IBM Bladecenter ppt slides

Citation preview

Page 1: IBM Bladecenter

1

5.3

IBM BladeCenter Fundamentals

Introduction

Unit objectives

After completing this unit, you should be able to:

• List the major elements common to the IBM BladeCenter

• Describe the key aspects of compatibility between BladeCenter models

• Identify the components providing redundancy in the BladeCenter chassis

• Match the power components necessary to support varying BladeCenter resource

configurations

• List the power input requirements for the BladeCenter models

• Describe the common cooling components used in the BladeCenter chassis

• Describe the supported disk configurations for the BladeCenter S

2

Dynamic infrastructure: Key points

• Enables visibility, control, and automation across all business and IT assets

• Transforms assets into higher value services

• Highly optimized to achieve more with less

• Addresses the information challenges

• Leverages flexible sourcing like clouds

3

Dynamic infrastructure: IBM BladeCenter

• Reduce cost

– x86 industry-leading performance, virtualization, energy efficiency, and scalability

• Manage risk

– Resilient architectures and management tools for security and high availability

• Improve service

– Performance to drive new demanding application workloads

4

BladeCenter chassis components

• BladeCenter chassis

• BladeCenter chassis midplane

• Power

• Cooling

• Unit summary

5

IBM BladeCenter family

Common blades, common switches, common management

IBM BladeCenter E

Highest density, super power

efficient

IBM BladeCenter H

Ultra high performance, and I/O

flexibility

BladeCenter chassis by model and type

IBM BladeCenter HT

Highly rugged, Telco, AC/DC, NEBS, air

filtration

IBM BladeCenter T

Highly rugged, Telco, AC/DC, NEBS, air

filtration

7U design

Up to 14 blade bays

Up to 4 switch fabrics

Low cost

Low power

Support 10 GB

Uplinks

Support 8 Gb FC

9U design

Up to 14 blade bays

Up to 10 GB midplane

I/O flexibility up to 8

switch bays

Support 3 0mm blades

w/ up to 8 ports

Support 10 GB

Ethernet

Support 8 Gb FC

Support 4x InfiniBand

8U design

Up to 8 blade bays

Up to 4 switch fabrics

AC or DC models

NEBS compliant

Rugged

Support 10 GB

uplinks

Support 8 Gb FC

Telco, military, dirty

floor

12U design

Up to 12 blade bays

AC or DC models

I/O flexibility up to 8

switch bays

NEBS compliant

Rugged

Up to 10 GB midplane

Support 10 GB Ethernet

Support 8 Gb FC

Support 4x InfiniBand

Telco, military, dirty floor

IBM BladeCenter S

Extending the benefits of BladeCenter outside

the datacenter

7U design

Up to 6 blade bays

Integrated storage

Up to 3 switch fabrics

Lowest cost

Lowest power (100 -

240v)

950w/1450 AC auto-

sensing

Support 10 GB Uplinks

Support 4 GB FC

6

Page 2: IBM Bladecenter

2

Chassis: IBM BladeCenter E

• 7U rack based mechanical

• One Midplane board

– Interface for major system components

– Divided into top and bottom halves

– Each half of the midplane is part of a redundant pair

• Front of chassis

– 14 hot-swap blade server bays

– One front control panel/media tray

• LED panel

• Media tray

– One USB port

– One DVD-RW

• Rear of chassis

– Four switch module bays

– Two management module bays

– Four hot-swap power supply module bays

– Two hot swap blowers

7

IBM BladeCenter E: Front view

CD-ROM drive Diskette drive

Blade server

Location

Over-Temp Information System error

USB portPower DVD-RW drive

Recess for chassis

service label

Front panel

LEDs and USB

port

7U

Blade server filler

8

IBM Calibrated Vectored Cooling

Hot-Swap Redundant Blowers with

PFA

Hot-Swap Management

Module

(Optional) Hot-Swap

Redundant Management

Module

Power Module

Bay 2 with

PFA

Power Module

Bay 4 with PFA

Switch Module Bay 3

Switch Module Bay 4

Switch

Module Bay 2

LED

Panel

Power Module Bay 1 with PFA

Power Module Bay 3 with

PFA

Switch Module Bay 1

IBM BladeCenter E: Rear view

9

IBM BladeCenter E: Rear view (new chassis)

IBM BladeCenter E with 2320W Power Supply Modules

10

Chassis: IBM BladeCenter T

• 8U and 28 inches deep

• Supports up to eight blade servers

– Up to four double slot blade servers

• Hot swappable media tray

– 24X DVD/CD drive

– Telco alarm panel

– Two USB front inputs

• Supports up to two management modules

• Four front load balancing and failover 1300W AC or DC power supplies

– Each power supply contains two fans and a LED panel

• Four rear I/O modules

• Four rear blower modules

• KVM module

• LAN module

11

IBM BladeCenter T: Front view

Power Module 2

Power Module 4

Power Module 3

Power Module 1

Management Module 1 Management Module bay 2

Front panel Media tray ESD connector

From top to bottom

Blade servers 1-8

12

Page 3: IBM Bladecenter

3

IBM BladeCenter T: Rear view

Blower Module 2

Blower Module 4

Blower Module 1

Blower Module 3

I/O Module 2 I/O Module 1

I/O Module 4 I/O Module 3

AC-power connectors

(8730) or DC-power

connectors (8720)

ESD connector

KVM Module LAN Module

13

Chassis: IBM BladeCenter H

• 9U and 28 inches deep

• Supports up to 14 30 mm blades

• Customer serviceable, hot swappable

media tray

• 9.5 mm combo drive (CD/DVD)

– Two USB front inputs

– Full light path diagnostics panel

• Rack mounted on rails

• Four front load 2900W power supplies

– Each power supply includes a replaceable fan

pack with three fans

14

Hot-Swap Media Tray with DVD, 2

USB, and LightPath Diagnostics Panel

Two bays for optional Hot-Swap Redundant

2900 Watt Power Supplies with PFA

and three replaceable fans each

Two bays for Hot-Swap Redundant

2900 Watt Power Supply with PFA and

three replaceable fans each

Support for up to 14 Blades

9U

IBM BladeCenter H: Front view

15

Redundant Power Input Redundant Power InputHigh Speed Switch Module Bays

Switch Module Bay 1 (dedicated Ethernet)

High Speed Switch Module Bays

Dedicated Bridge

Module Slot

IBM Calibrated Vectored Cooling

Hot-Swap Redundant Blowers with PFA

Hot-Swap Advanced

Management Module (AMM)

Switch Module Bay

4 (or Bridge Slot)

Switch Module Bay

3 (or Bridge Slot)

(Optional) Hot-Swap Redundant

Advanced Management

Module (AMM)

Switch Module Bay 2 (dedicated Ethernet)

Dedicated Bridge

Module Slot

LightPath

Diagnostics Panel

Aggregated Serial

Connector

IBM BladeCenter H: Rear view

16

Chassis: IBM BladeCenter HT

• 12U and 27.8 inches deep

• Supports up to 12 blade servers

– Up to six double slot blade servers

• Support up to two media trays

– Full Light Path Diagnostic panel

– Two USB front inputs

• Support up to four front 3160W AC or DC power supplies with loadbalancing and failover

– Each power supply contains a replaceable fan pack with three fans

• Four I/O module bays

• Four high-speed I/O module bays

• Four rear blower modules

• Supports up to two multiplexer expansion module

17

IBM BladeCenter HT: Front view

Power Module 2 Power Module bay 4

Power Module bay 3 Power Module 1

Management Module 1

Management Module 2

Media tray bay 1

Media tray bay 2

ESD connector

I/O Module bay 2

I/O Module bay 1

I/O Module bay 4

I/O Module bay 3

High-speed I/O

Module bay 7

High-speed I/O

Module bay 8

High-speed I/O

Module bay 9

High-speed I/O

Module bay 10

From left to right

Blade servers 1-12

18

Page 4: IBM Bladecenter

4

IBM BladeCenter HT: Rear view

Power connector 3

Fan Module 3

Power connector 2 Power connector 4

Power connector 1

Fan Module 4

Fan Module 1

Fan Module 2

Multiplexer

expansion module 2

Multiplexer

expansion module 1

Network-clock bay 1

Serial connector

Alarm panel

module

Alarm panel interface connector

Network-clock bay 2

19

Chassis: IBM BladeCenter S

• Supports up to six 30 mm blades

• Hot swappable media tray

– Light Path Diagnostics panel

– Combo drive (CD/DVD)

– Two optional battery backup units

– Two USB 2.0 ports

• Two 950-watt/1450 auto-sensing

power supplies

– Express model comes with four 950-

watt/1450 power supplies

• Four hot-swap fan packs

• Two optional integrated Disk Storage

Modules (DSM)

– Up to twelve 3.5-inch HS HDDs per

shuttle

• One Advanced Management Module

20

Ideal for business-in-a-box configurations, legacy & future servers supported

Ideal for tiered workloads; SAS and SATA disks can be mixed in BladeCenter S using RAID 0, 1, and 10

Easy access to

shared USB portsand UltraBay

Battery Backup Units for use with IBM BladeCenter SAS Connectivity Module

7U

IBM BladeCenter S: Front view

21

IBM BladeCenter S: Rear view

Power Supplies 3 and 4

are optional, Auto-

sensing b/w

950W 100/110V or

1450W 220/240v

Four hot-swap fan

pack standard

Power Supplies 3 and 4

are included in Express,

Auto-sensing b/w 950W

100/110V or 1450W

220/240v

Top and bottom right: I/O modulesBottom left: For future use

BladeCenter S Express Model

AMM standard

Serial module option

22

IBM BladeCenter chassis: Comparison

BladeCenter E

(M/T 8677)

BladeCenter T

(M/T 8720 -DC)

(M/T 8730 -AC)

BladeCenter H

(M/T 8852)

BladeCenter HT

(M/T 8740 -DC)

(M/T 8750 -AC)

BladeCenter S

(M/T 8886)

Chassis height 7U / 12.0” (305mm) 8U / 13.75” (349mm)9U / 15.75”

(400mm)

12U / 21.0”

(/528mm)

7U / 12.0”

(306.3mm)

Chassis depth 28 “ (711mm) 20.0” (508mm) 28.0” (711mm) 27.8” (706.0 mm) 28.9” (733.4mm)

Max number of blade

servers14 8 14 12 6

Max number of I/O

Modules

(Switch, and bridge)

4 4

10 (4 x high

speed, 4 x

standard, 2 x

bridge)

8 (4 x high speed, 4

x standard or 2 x

bridge)

3

Power supply size

(standard)

2000 Watts AC

(2320 Watts

optional)

1300W DC, -38V to -

75V (-48V nominal);

1300W AC

2900 Watts AC

2535W DC (60A)

3160W DC (75A)

2800 Watts AC

950W/1450W AC

auto-sensing

Number of Power

Supplies (standard/

maximum)

2/ 4 2/ 4 2/ 4 2/ 4 2/ 4

Number of Blowers

(standard/ maximum)2/ 2 4/ 4 2/ 2 4/ 4 4/ 4

Diskette Drives

(standard)

None (early models

contain 1.44 MB

drive)

None None None None

DVD/CD drives

standard

1x DVD-ROM (in

Media Tray)

1x DVD-ROM (in

Media Tray)

1x DVD-ROM (in

Media Tray)None

1x DVD-ROM (in

Media Tray)

23

BladeCenter chassis components

• BladeCenter chassis

• BladeCenter chassis midplane

• Power

• Cooling

• Unit summary

24

Page 5: IBM Bladecenter

5

BladeCenter: Interior view

Lower midplane

half

Upper midplane

half

Upper

Processor

blade

connectors

Lower

BladeCenter H example

25

Media tray

connector

Power

module connector

Blade power

connector

Power

module connector

Power

module connector

Power

module connector

Blade power

connector

BladeCenter H example

BladeCenter: Midplane

26

BladeCenter chassis components

• BladeCenter chassis

• BladeCenter chassis midplane

• Power

• Cooling

• Unit summary

BC-E Power Module

BC-H Power Module

27

Right Redundant Pair

Standard with chassis Operate as redundant pair Provides redundant power for

chassis modules and blades 1-6 Load balancing power supplies

Left Redundant Pair

Optional Operate as redundant pair Provides redundant power for

blades 7-14 Load balancing power supplies

Power supply:

• 2000 W or 2320 W AC/DC hot-swap redundant power modules– Maximum of four power

modules supported

– AC and DC LEDS on each power module

• 200-240 V AC (range: 180-265 V AC)– Load balancing across all

power supplies

– Built-in overload and surge protection

– Cooling is provided by the host system

Power Supplies

1 and 2

Optional Power

Supplies3 and 4

IBM BladeCenter E: Power components

28

IBM BladeCenter E: Specifics and considerations

• Two power supplies in bays 1 and 2 standard

• Two 9ft IEC 320-C13 to C14 or two IEC 320-C19 to C20 cables for

intra-rack power distribution)

• Max: four hot-swap power supplies

• Rear access

• 2000 or 2320 watts at 220v

• Both redundant (N+N redundancy) / two std modules supply power to

all modules and blade bays 1 through 6

• Power for bays 7 through 14 requires two additional power modules in

bays 3 and 4 (redundant to each other)

• Voltage-sensing

• Auto-restart

• No fans power in

module(s)

IBM BladeCenter E Power Supply C19 to C20 Power Cord

29

BladeCenter H: Power components

Power

Connector 2

Power

Connector 1

BladeCenter H power supplies

• Each vertical pair is

N+N redundant

30

Page 6: IBM Bladecenter

6

• Power module bays– Maximum of four per chassis

– Two ship standard with the chassis

BladeCenter H: Power modules

31

IBM BladeCenter T: Power components

• Two power supplies in bays 1 and 2 standard

• Max: Four hot-swap power supplies

• Front access

• Two IEC320 for the M/T 8720 (DC power)

– Input voltage range -48V - 60V (Min/Max) at 50/60Hz

– IBM BladeCenter T 8720 should be attached to the appropriate DC wiring

• Two IEC320 for the M/T for the 8730 (AC power)

– Input voltage range 180VAC - 264VAC (Min/Max) at 50/60Hz

– IBM BladeCenter T 8730 should be attached to high-voltage PDUs

• C14 male appliance connector

• 1300 watts

• Two standard modules (in slots 1 and 2) supply power to switch bays 1 and 2, both management modules, blade bays 1 - 4 and media tray

• Two additional power modules provide full system redundant power and power to switch bays 3 and 4, and blade bays 5 - 8 All: Load balancing / voltage-sensing / auto-restart

IBM BladeCenter T AC power supply

IBM BladeCenter T DC power supply

32

IBM BladeCenter HT: Power components

• Two standard power supplies ships with the BladeCenter

• Max: four hot-swap power supplies

• Front access

• IBM BladeCenter HT 8740 (DC Power Input voltage range -48VDC to -60VDC

(Min/Max)

• IBM BladeCenter HT 8750 (AC Power input voltage range -48VDC to -60VDC

(Min/Max) at 50/60Hz

• Redundant within pair (N+N redundancy) / power modules 1 and 2 support blade

bays 1- 6, all legacy switch and bridge modules, upper and lower media trays, both

advanced management modules, multiplexer expansion modules and alarm panel

module

• Power modules 3 and 4 support blade bays 7-12 and all high speed switch modules

• Auto-restart

• Three fans per power module

IBM BladeCenter HT Power Supply

33

IBM BladeCenter S: Power components

• Two standard power supplies ships with the

BladeCenter

• Maximum four hot-swap

• Rear access

• 1450 watts at 220V

• 950 watts at 110V

• Redundant within pair (N+N

redundancy)

• Voltage-sensing

• Auto-restart

• Two fans per power module

• Modules 1 and 2 standard

• Modules 3 and 4 required with 2nd

Storage module installed

34

BladeCenter chassis components: Cooling

• BladeCenter chassis

• BladeCenter chassis midplane

• Power

• Cooling

• Unit summary

35

BladeCenter E: Cooling subsystem

• Two blowers

– Capable of 325 cubic feet per minute (CFM) each

– 150 CFM each in standard operation

• Hot swap, redundant

• Air flow is from the front to rear

• Fan speed control

• Predictive blower failure by monitoring the blower RPM

• Back flow dampers shall be incorporated to prevent air short circuiting if one blower fails

36

Page 7: IBM Bladecenter

7

IBM BladeCenter H: Cooling subsystem

Hot-swap customer serviceable

fan packs

Hot-swap customer serviceable

AC blower units

BladeCenter H front chassis BladeCenter H rear chassis

IBM BladeCenter H blowerHot-swap customer serviceable fan pack

37

IBM BladeCenter T: Cooling subsystem

• Four blowers comes standard

• Maximum four

• 3+1 redundancy configuration

• Calibrated vectored cooling

• Hot-swap

• Redundant

• Variable speed

• Rear access

• 330 cubic ft per min (CFM)

• Front to back airflow; filtered air intake

BC T Blower Unit (1) of (4)

BC T Rear Chassis

38

IBM BladeCenter HT: Cooling subsystem

• Four hot-swap blowers standard/Maximum

• N+1 cooling redundancy

• Speed controlled according to ambient air temperature

• Rear access

• Front to back airflow

39

IBM BladeCenter S: Cooling subsystem

• Four hot-swap fans standard

• N+1 redundant cooling airflow

• Throttled according to incoming air temperature

• Rear access

• Front to back airflow

40

Key words

• Advanced Management Module (AMM)

• Concurrent Keyboard, Video and Mouse (cKVM)

• Disk Storage Module (DSM)

• Dynamic Infrastructure (DI)

• Keyboard, Video, Mouse (KVM)

• IBM BladeCenter E (Enterprise)

• IBM BladeCenter H (High Performance)

• IBM BladeCenter HT (High Performance Telco)

• IBM BladeCenter S (Simplification)

• IBM BladeCenter T (Telco)

• Local Area Network (LAN)

• Network Equipment Building System (NEBS)

41

Checkpoint (1 of 3)

1. The IBM BladeCenter contains a redundant circuit path to connect all blades

servers to the AMM and switching fabric through which of the following:

a. Blade servers are interconnected by a system of shielded cabling

b. A Midplane, with redundant upper and lower halves, connects all blade servers and provides

a path to the switching modules

c. Each blades server is independently connected to the AMM and switching modules

2. True/False: The Midplane for the BladeCenter H contains only the new high-

speed 4x signaling fabric connectors.

3. The IBM BladeCenter E Chassis supports up to 14 Blade Servers. With Power

Supplies 1 and 2 installed, which blade servers, switch modules and management

modules will receive power?

a. All fourteen Blade Server slots, all four switch module slots, the two blowers, the two

management module slots and the media tray.

b. Blade Server slots 1 through 6, all four switch module slots, the two blowers, the two

management module slots and the media tray.

c. Blade Server slots 1 through 6, switch module slots one and two, one management module

slot and the media tray.

d. All fourteen Blade Server slots and the management module slots, the two blowers, and the

media tray.

42

Page 8: IBM Bladecenter

8

Checkpoint solutions (1 of 3)

1. The IBM BladeCenter contains a redundant circuit path to connect all blades servers to the AMM and switching fabric through which of the following:

a. Blade servers are interconnected by a system of shielded cabling

b. A Midplane, with redundant upper and lower halves, connects all blade servers and provides a path to the switching modules

c. Each blades server is independently connected to the AMM and switching modules

Answer: b

2. True/False: The Midplane for the BladeCenter H contains only the new high-speed 4x signaling fabric connectors.

Answer: False

3. The IBM BladeCenter E Chassis supports up to 14 Blade Servers. With Power Supplies 1 and 2 installed, which blade servers, switch modules and management modules will receive power?

a. All fourteen Blade Server slots, all four switch module slots, the two blowers, the two management module slots and the media tray.

b. Blade Server slots 1 through 6, all four switch module slots, the two blowers, the two management module slots and the media tray.

c. Blade Server slots 1 through 6, switch module slots one and two, one management module slot and the media tray.

d. All fourteen Blade Server slots and the management module slots, the two blowers, and the media tray.

Answer: b

43

Checkpoint (2 of 3)

4. True/False: All models of the BladeCenter implement common power, cooling and signaling paths for multiple Blade Servers.

5. True/False: In addition to component cooling, each BladeCenter provides general chassis cooling through blowers.

6. Select the correct statement regarding power input to the BladeCenter S chassis.a. The BladeCenter S chassis requires 220/240V power

b. The BladeCenter S can be ordered with 110V input as an option

c. The BladeCenter S supports (2) power supplies maximum at 100/110V

d. The BladeCenter S supports either 100/110V or 220/240V input

7. Operator information indicating warnings for over temperature and system error is viewed on the BladeCenter E chassis through which of the following methods?a. The front panel of each blade server

b. On the Light Path panel on the media tray

c. On the Light Path display on the lower rear of the chassis

d. On the AMM module

e. Both b and c

8. Which statement is correct regarding BladeCenter S power modules?a. Standard power supplies 1 and 2 support 3 blade servers

b. Four power modules are shipped with the base BladeCenter S chassis

c. Optional power supplies 3 and 4 are required for the second Disk Storage Module

d. Optional power supplies 3 and 4 are required for I/O bays 1 and 3

44

Checkpoint solutions (2 of 3)

4. True/False: All models of the BladeCenter implement common power, cooling and signaling paths for multiple Blade Servers.

Answer: True

5. True/False: In addition to component cooling, each BladeCenter provides general chassis cooling through blowers.

Answer: True

6. Select the correct statement regarding power input to the BladeCenter S chassis.

a. The BladeCenter S chassis requires 220/240V power

b. The BladeCenter S can be ordered with 110V input as an option

c. The BladeCenter S supports (2) power supplies maximum at 100/110V

d. The BladeCenter S supports either 100/110V or 220/240V inputAnswer: d

7. Operator information indicating warnings for over temperature and system error is viewed on the BladeCenter E chassis through which of the following methods?

a. The front panel of each blade server

b. On the Light Path panel on the media tray

c. On the Light Path display on the lower rear of the chassis

d. On the AMM module

e. Both b and cAnswer: e

8. Which statement is correct regarding BladeCenter S power modules?

a. Standard power supplies 1 and 2 support 3 blade servers

b. Four power modules are shipped with the base BladeCenter S chassis

c. Optional power supplies 3 and 4 are required for the second Disk Storage Module

d. Optional power supplies 3 and 4 are required for I/O bays 1 and 3 Answer: c

45

Checkpoint (3 of 3)

9. True/False: The BladeCenter S Disk Subsystem supports a combination of SAS and SATA drives.

10. True/False: IBM BladeCenter E contains 10 I/O module bays and supports I/O network switching for storage switches,

pass through devices, traditional fabrics, and high-speed fabrics.

11. True/False: For Telco maintenance, the BladeCenter HT chassis supports up to 4 hot-swap and redundant DC or AC

power supply modules with load-balancing and failover capabilities located in the rear of the chassis

12. Which chassis was designed specifically for telecommunications network infrastructures to support a highly rugged

environments using air filtration?

a. BladeCenter S and BladeCenter E

b. BladeCenter T and BladeCenter HT

c. BladeCenter E and BladeCenter H

d. BladeCenter H and BladeCenter HT

46

Checkpoint solutions (3 of 3)

9. True/False: The BladeCenter S Disk Subsystem supports a combination of SAS

and SATA drives.

Answer: True

10. True/False: IBM BladeCenter E contains 10 I/O module bays and supports I/O

network switching for storage switches, pass through devices, traditional fabrics,

and high-speed fabrics.

Answer: False

11. True/False: For Telco maintenance, the BladeCenter HT chassis supports up to 4

hot-swap and redundant DC or AC power supply modules with load-balancing and

failover capabilities located in the rear of the chassis

Answer: False

12. Which chassis was designed specifically for telecommunications network

infrastructures to support a highly rugged environments using air filtration?

a. BladeCenter S and BladeCenter E

b. BladeCenter T and BladeCenter HT

c. BladeCenter E and BladeCenter H

d. BladeCenter H and BladeCenter HT

Answer: b

47

Unit summary

Having completing this unit, you should be able to:

• List the major elements common to the IBM BladeCenter

• Describe the key aspects of compatibility between BladeCenter models

• Identify the components providing redundancy in the BladeCenter chassis

• Match the power components necessary to support varying BladeCenter resource

configurations

• List the power input requirements for the BladeCenter models

• Describe the common cooling components used in the BladeCenter chassis

• Describe the supported disk configurations for the BladeCenter S

48

Page 9: IBM Bladecenter

9

5.3

Blade server technology

Unit objectives

After completing this unit, you should be able to:

• Identify and describe the components implemented on a blade server

• List the methods used for blade server communications through the chassis

• Identify the procedures used to access the blade server console

• Select and describe the features enabling blade server scalability

• List the hard drive options available in the blade server models

• Select the appropriate high speed expansion card options by chassis model

50

• A blade server contains the core components of a server

• Each blade server has:

– Processor(s)

– Memory

– Internal storage (optional)

– Network Interface Cards (NIC)

– Optional plug-in components

• The blade server plugs into the midplane of a chassis that provides common functions:

– Management console access (KVM)

– Power supplies

– Cooling fans

– Network connectivity (LAN, SAN, NAS, and HPC)

– Shared media devices (CD-ROM and diskette drives)

– Optional modules to support additional functionalities

What is a blade server?

51

IBM Blade Server

• IBM Blade Server overview

• Common components

• Expansion cards

• Enhanced features

• Specialized function

• Model comparison

• Expansion blades

52

IBM Blade Server overview: Front panel

• Controls and Indicators– Power on/off

– KVM and media

– Reset• Hard drives (selected models) Light Path

Diagnostic Panel

HS22 Front ViewHS21 XM Front View

53

IBM Blade Server overview:

Interior components (1 of 2)

HS22 Blade Server Internal View (externally accessible hard drives)

54

Page 10: IBM Bladecenter

10

IBM Blade Server overview:

Interior components (2 of 2)

KVM connector

Light PathDiagnostics Panel

DIMM socket 7

DIMM socket 5

DIMM socket 3

DIMM socket 1

DIMM socket 8

DIMM socket 6

DIMM socket 4

DIMM socket 2

3V lithium battery Modular flash drive connector

Microprocessor 1

Microprocessor 2

SAS hard disk drive and

connector

I/O

Expansion

option

connectors

HS21 XM Blade Server Internal View

55

IBM Blade Server overview:

Power and signal paths

Signaling (redundant)

Power input (redundant)

Blade Server

Midplane

56

IBM Blade Server: Common components

• IBM Blade Server overview

• Common components

• Expansion cards

• Enhanced features

• Specialized function

• Model comparison

• Expansion blades

57

Common components: Processor

• Processor architectures:

– x86-64

– IBM POWER

– IBM PowerXCell 8i processor

• Single to four socket

implementations

• Single to quad-core

processors

58

Common components: Memory

• System board mounted

DIMMs

• Speed and capacity

– Up to PC2-6400 memory

speed

– System board capacity to 128

GB

• Error correction and

redundancy

– ECC detection and correction

– Chipkill technology

59

Common components: Storage

• Hard drives

– SAS

– SATA

– SCSI

• Hot swap

• Internal and front panel accessible

• RAID (0,1)

• Solid state disk (SSD)

• Modular flash drive

Solid State Disk (SSD)

Hard DiskModular Flash Drive

60

Page 11: IBM Bladecenter

11

Common components: Storage configurations

Modular flash drive

Two internal hard drives, non-hot swapTwo internal hard drives, front-mounted hot swap

61

Common components: Communications

• Blade servers implement two NICs– Broadcom 1 Gb chipset

– First NIC attached to Ethernet I/O Module 1

– Second NIC attached to Ethernet I/O Module 2

• Optional expansion card– Provides two additional NICs

NIC 1 on Blade

NIC 2 on Blade

Optional Expansion Card

62

Common components: Blade server access

• AMM KVM functions

– Access to blade server onboard graphics (remote control)

– Connects blade servers to keyboard, mouse, and media tray

DVD drive and USB ports

• BladeCenter media tray

– Provides DVD drive and USB ports

• Serial over LAN (SOL)

• Concurrent KVM (cKVM) option cardKVM and Media Tray

Ownership Control

Concurrent KVM (cKVM) card Power

On/Off

Reset

63

IBM Blade Server: Expansion cards

• IBM Blade Server overview

• Common components

• Expansion cards

• Enhanced features

• Specialized function

• Model comparison

• Expansion blades

64

Expansion cards: Form factor

Standard Form Factor (StFF) Small Form Factor (SFF) Combination Form Factor vertical (CFFv)

Combination Form Factor horizontal (CFFh) High Speed Form Factor (HSFF)

Combination I/O vertical (CIOv)

65

Expansion cards: Connections

I/O connector

I/O Expansion card (PCI-X )

Blade Server PCI-X ConnectorBlade Server PCI-E Connector

Midplane PCI-

E connectors

Midplane PCI-

X connectors

BC-H Midplane with PCI-X and PCI-E Connectors

66

Page 12: IBM Bladecenter

12

BladeCenter E chassis: I/O topology

IBM BladeCenter E I/O Topology

67

BladeCenter E chassis: Expansion card routing

1. IBM BladeCenter HS22 or other supported Blade server

2. Ethernet controller on the system board

3. Compatible CFFv or CIOv expansion card

4. BladeCenter chassis

5. BNT Ethernet switch module(s)

6. BNT Ethernet switch module(s) routing signals from the system board integrated controller

68

BladeCenter H chassis: I/O topology

IBM BladeCenter H I/O Topology

69

BladeCenter H chassis: Expansion card routing

1. IBM BladeCenter HS22 or other supported Blade server

2. Ethernet controller on the system board

3. Compatible CFFv or CIOv expansion card

4. 2/4 port Ethernet expansion card (CFFh)

5. BladeCenter H chassis

6. BNT Ethernet switch modules routing signals from the system board integrated controller

7. BNT Ethernet switch modules routing signals from the CFFv or CIOv card

8. BNT Ethernet switch modules routing signals from the CFFh card

9. Multi-switch Interconnect module

70

IBM Blade Server: Enhanced features

• IBM Blade Server overview

• Common components

• Expansion cards

• Enhanced features

• Specialized function

• Model comparison

• Expansion blades

71

Enhanced features: Scalability

• IBM LS42 Blade Server

– Quad-core AMD Opteron

processor (8000 series)

– Optional Multiprocessor

Expansion (MPE) unit

– Scalable to 8-way processing

– 128 GB memory capacityBlade Interconnection

LS42 Base Blade

LS42 MPE

72

Page 13: IBM Bladecenter

13

Enhanced features: Embedded hypervisor (1 of 2)

• Hypervisor

– A virtualization platform that allows multiple operating systems to run on the same physical hardware at

the same time.

73

Enhanced features: Embedded hypervisor (2 of 2)

• HS21 XM supports Modular Flash Hypervisor storage

• LS22 and LS42 support USB Hypervisor storage

Hypervisor USB Flash Device

LS42 with Hypervisor

HS21 XM with Hypervisor4 GB Modular Flash Drive

74

Enhanced features: Advanced management

• HS22 advanced features:

– Integrated Management Module (IMM)

• Functions of BMC

• RSA II

• Remote presence/cKVM

– Unified Extensible Firmware Interface (UEFI)

• Event logs

• Adapter configuration

• Out-of-band ASU

75

IBM Blade Server: Specialized function

• IBM Blade Server overview

• Common components

• Expansion cards

• Enhanced features

• Specialized function

• Model comparison

• Expansion blades

76

Specialized blade servers: Specialized function

• QS22 blade server

– High performance computing

applications

• Image and signal processing

• Scientific research

• Financial algorithms

– IBM PowerXCell 8i processor

• PN41 Deep Packet Inspection

Blade

– Deep Packet Inspection (DPI) and

real-time analysis of network traffic

– Network security solution

– Optimization of DNS processing

– Specialized operating system

– Intel IXP2805 network processor

QS22 CellBE Blade

PN41 Deep Packet Inspection (DPI Blade

77

IBM Blade Server: Model comparison

• IBM Blade Server overview

• Common components

• Expansion cards

• Enhanced features

• Specialized function

• Model comparison

• Expansion blades

78

Page 14: IBM Bladecenter

14

IBM Blade Server: Model comparison

79

IBM Blade Server: Expansion blades

• IBM Blade Server overview

• Common components

• Expansion cards

• Enhanced features

• Specialized function

• Model comparison

• Expansion blades

80

HS21 Blade

Memory and I/O

Expansion Blade

Two additional Gigabit Ethernet

TOE controllers (total of four)

Four additional DIMM sockets

(total of eight and up to 32GB with

online spare and memory mirroring)

One additional I/O

expansion slot

(total of two)

IBM expansion blades: Memory and I/O (MIO)

Note: The MIO Expansion Blade is supported on the HS21 (8853) only

81

Three SAS hot-swap

hard drive bays

Blade expansion

connector

IBM expansion blades: Storage and I/O

IBM Blade (HS21, LS21, or LS41)

Storage and

I/O Expansion

Blade

Storage and I/O Expansion Blade Assembly

RAID 5 (optional battery cache)

I/O Expansion slot I/O Expansion slot

82

IBM expansion blades: PCI express and PCI-X I/O

• PCI-E and PCI-X models

• Provides PCI-X and PCI-E attachment and access

PCI-E Expansion Blade

PCI-E Card Connectors PCI-E Card External Access

83

Key words

• AMD

• Intel

• Cell Broadband Engine

• Dual-Core

• High Performance Computing (HPC)

• Local Area Network (LAN)

• Memory and I/O Expansion Unit (MIO)

• Modular Flash DriveNetwork Interface Cards (NIC)

• PCI Expansion Unit

• Power6

• Quad-core

• Redundant Connection

• Serial Over LAN (SOL)

• Single-core

• Storage and I/O Expansion Unit (SIO)

• Uni-processor

84

Page 15: IBM Bladecenter

15

Checkpoint (1 of 2)

1. Select the correct statement regarding the components implemented on a blade server.

a. Each Blade server contains processor, memory, power supply and graphics components

b. A Blade server contains processor, memory, storage and communications components

c. All Blade servers contain a minimum of two processors

d. In order to implement Ethernet, each Blade server must have an expansion card installed

2. Access to the blade server console is accomplished by which of the following?

a. An externally attached KVM

b. Through a shared KVM facility through the Advanced Management Module or through SOL

c. Only through an Ethernet connection from a switch module

d. Only after an operating system has been loaded on a blade

3. A blade server communicates through the BladeCenter chassis through which of the following?

a. High density cabling within the BladeCenter chassis

b. Blade server to blade server over external bus

c. Through the BladeCenter midplane

d. A blade server expansion card is required

4. Select the correct statement regarding blade server scalability.

a. Blade server scaling to 4 processors is done by adding processors 3 and 4 to the system board

b. Blade servers are limited to implementing 2 processors maximum

c. Scaling a blade server requires the addition of scalability cables

d. The LS42 blade server can be scaled to 4 processors by adding the MPE expansion blade

85

Checkpoint solutions (1 of 2)

1. Select the correct statement regarding the components implemented on a blade server.a. Each Blade server contains processor, memory, power supply and graphics components

b. A Blade server contains processor, memory, storage and communications components

c. All Blade servers contain a minimum of two processors

d. In order to implement Ethernet, each blade server must have an expansion card installed

Answer: b

2. Access to the blade server console is accomplished by which of the following?a. An externally attached KVM

b. Through a shared KVM facility through the Advanced Management Module or through SOL

c. Only through an Ethernet connection from a switch module

d. Only after an operating system has been loaded on a blade

Answer: b

3. A blade server communicates through the BladeCenter chassis through which of the following?a. High density cabling within the BladeCenter chassis

b. Blade server to blade server over external bus

c. Through the BladeCenter midplane

d. A blade server expansion card is required

Answer: c

4. Select the correct statement regarding blade server scalability.a. Blade server scaling to 4 processors is done by adding processors 3 and 4 to the system board

b. Blade servers are limited to implementing 2 processors maximum

c. Scaling a blade server requires the addition of scalability cables

d. The LS42 blade server can be scaled to 4 processors by adding the MPE expansion blade

Answer: d

86

Checkpoint (2 of 2)

5. True/False: In order to implement hot swap hard drive capability and expansion blade must be added to the HS22 blade server.

6. True/False: The CFFh form factor Ethernet PCI-E Expansion card will function on a blade server in the BladeCenter H, HT, and S chassis.

7. Select the correct statement regarding the memory and I/O expansion (MMIO) blade.a. The MMIO expansion blade supports 2 internal hard disk drives

b. An MMIO expansion blade cannot be attached to any another expansion blade

c. The MMIO can be attached to any model blade server

d. The MMIO contains four DIMM sockets and two 1 Gb Ethernet connections

8. Which statement is correct regarding the QS22 blade server?a. The QS22 is used primarily for applications requiring large disk storage capacity

b. The QS22 is optimized for applications requiring parallel processing and streaming capabilities

c. Processing in the QS22 is accomplished with one POWER6 processor and one numeric co-processor

d. Implementation of the QS22 requires the attachment of an MMIO expansion blade

87

Checkpoint solutions (2 of 2)

5. True/False: In order to implement hot swap hard drive capability and expansion blade must be added to the HS22 blade server.

Answer: False

6. True/False: The CFFh form factor Ethernet PCI-E expansion card will function on a blade server in the BladeCenter H, HT, and S chassis.

Answer: True

7. Select the correct statement regarding the memory and I/O expansion (MMIO) blade.

a. The MMIO expansion blade supports 2 internal hard disk drives

b. An MMIO expansion blade cannot be attached to any another expansion blade

c. The MMIO can be attached to any model blade server

d. The MMIO contains four DIMM sockets and two 1 Gb Ethernet connections

Answer: d

8. Which statement is correct regarding the QS22 blade server?

a. The QS22 is used primarily for applications requiring large disk storage capacity

b. The QS22 is optimized for applications requiring parallel processing and streaming capabilities

c. Processing in the QS22 is accomplished with one POWER6 processor and one numeric co-processor

d. Implementation of the QS22 requires the attachment of an MMIO expansion blade

Answer: b

88

Unit summary

Having completed this unit, you should be able to:

• Identify and describe the components implemented on a blade server

• List the methods used for blade server communications through the chassis

• Identify the procedures used to access the blade server console

• Select and describe the features enabling blade server scalability

• List the hard drive options available in the blade server models

• Select the appropriate high speed expansion card options by chassis model

89 5.3

Ethernet connectivity and options

Page 16: IBM Bladecenter

16

Unit objectives

After completing this unit, you should be able to:

• List the BladeCenter components required for Ethernet connectivity

• Name the management components supporting Ethernet switch modules

• Select the Ethernet I/O switch module to meet a specific requirement

• List the components required to provide additional Ethernet connectivity to a blade server

• Identify the method used to provide blade server Ethernet connection redundancy

91

BladeCenter chassis I/O overview

• BladeCenter chassis I/O overview

• Internal chassis traffic flow

• Switch I/O module management connectivity

• BNT (Nortel) Ethernet switch I/O modules

• Server connectivity module

• Pass-thru module

• High speed I/O modules overview

• Ethernet expansion cards

92

BladeCenter chassis overview: Switching path

BladeCenter Chassis Physical Components

93

Chassis overview: Gigabit Ethernet routing

Midplane (Upper Section)

Midplane (Lower Section)

SW Module 1

1………..14

SW Module 3

1………..14

1………..14

SW Module 2

1………..14

SW Module 4

Network

Interface

Expansion

Card

SERDES

Ethernet

SERDES

Ethernet

Line 1

Line 3

Line 4

Line 2

Processor blade

#1

94

Chassis overview: Ethernet port assignments (1 of 2)

1

B

S

1

2

1

B

S

1

4

2

1

B

S

2

2

1

B

S

3

2

1

B

S

4

2

1

B

S

5

2

1

B

S

6

2

1

B

S

7

2

1

B

S

8

2

1

B

S

9

2

1

B

S

1

0

2

1

B

S

1

1

2

1

B

S

1

2

2

1

B

S

1

3

2

Bottom ESM (Bay 2 )

MM 1

MM 2

Top ESM (Bay 1 )

1 2 3 4 5 6 7 8 9 10 11 12 13 14 Mgt 1 Mgt 2

ESMExternal uplinks

ESMExternal uplinks

MM 1Uplink

MM 2Uplink

BladeCenter Chassis

Eth 1

Eth 1

Eth 0

Eth 0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 Mgt 1 Mgt 2

95

Chassis overview: Ethernet port assignments (2 of 2)

1

B

S

1

2

1

B

S

2

2

1

B

S

3

2

1

B

S

4

2

1

B

S

5

2

1

B

S

6

2

MM 1

Top ESM (Bay 1 )

1 2 3 4 5 6 8 9 10 11 12 13 Mgt 1

ESMExternal uplinks

MM 1Uplink

BladeCenter S Chassis

Eth 1

Eth 0

96

Page 17: IBM Bladecenter

17

Internal chassis traffic flow

• BladeCenter chassis Ethernet overview

• Internal chassis traffic flow

• Switch I/O module management connectivity

• BNT (Nortel) Ethernet switch I/O modules

• Server connectivity module

• Pass-Thru module

• High speed I/O modules overview

• Ethernet expansion cards

97

Ethernet connectivity: Layer 2 traffic flow

14 Internal Blade ports

Management

Module

Ports

CPU

Hidden internal filter that blocks traffic flow between the External ports and Mgmt Mod ports.

Prevents STP loops between ESM uplinks and MM uplinks (as well as between ESMs)

Ethernet

Switch

Module

(ESM)

MM1

MM2

BladeCenter Chassis

Upstream connectionsUpstream connections

Red = Ethernet

External ports

98

Traffic flow: NIC teaming and trunk failover

1 2 3 4 5 6 1 2 3 4 5 6

ES

M

ES

M

NIC

1

NIC

2

X X

Teamed

Active/Standby

VLAN X

x

Blade Server

Logical NIC

Interface

If failure anywhere on the

link toward the upstream

switch, the NIC on blade

server does not know

about the failure and may

continue to send traffic

toward the top switch,

which will discard the

traffic. The Trunk

Failover feature

addresses this issue.

If the switch fails in such

a way that the link toward

the Blade server goes

down, or NIC fails, Blade

server can sense this

and redirect traffic out

the other NIC toward the

bottom switch. NIC

Teaming can take care

of this without the need

for trunk failover. 99

Traffic flow: Trunk failover example 1

• If Po1 goes down, trunk failover takes over and shuts

down internal downstream defined port(s)

– This alerts NIC Teaming to an upstream failure at which point

Teaming switches to the other path out of the server

– This example shows a single VLAN to Teamed NICs, It is

possible to also carry multiple VLANs to the Teamed NICs

1 2 3 4 5 6 1 2 3 4 5 6

ES

M

ES

M

NIC

1

NIC

2

Teamed

Active/Standby

VLAN X

Blade Server

Logical NIC

Interface

VLAN X

Must be carried

between ESMs via an external path

•Link state group1

Upstream

•Link state group1

Upstream

•Link state group

Downstream

•Link state group

Downstream

VLAN X Must

be carried on

all uplinks

from ESM

•NIC

Teaming

•Po1 •Po1

•Po1

•Po2 •Po2

100

Traffic flow: Trunk failover example 2

• If only one port channel goes down, spanning tree allows backup path to other port channel

• If both port channels go down, trunk failover takes

over and shuts down internal downstream defined

port(s)

– This alerts NIC Teaming to an upstream failure at which

point it switches to the other CIGESM

– This example shows a single VLAN to Teamed NICs, It is

possible to also carry multiple VLANs to the Teamed NICs

1 2 3 4 5 6 1 2 3 4 5 6

ES

M

ES

M

NIC

1

NIC

2

Teamed

Active/Standby

VLAN X

Blade Server

Logical NIC

Interface

•Link state group1

Downstream

•Link state group1

Downstream

VLAN X Must

be carried on

all uplinks

from ESM

•NIC

Teaming

•Po2 •Po1

•Po1•Po2 •Po2

•Link state group1

Upstream

•Link state group1

Upstream

•Po1 •Po2

•Po3 •Po3

101

Switch I/O module management connectivity

• BladeCenter chassis Ethernet overview

• Internal chassis traffic flow

• Switch I/O module management connectivity

• BNT (Nortel) Ethernet switch I/O modules

• Server connectivity module

• Pass-Thru module

• High speed I/O modules overview

• Ethernet expansion cards

102

Page 18: IBM Bladecenter

18

Management connectivity for the ESM

Management Module

Ethernet

Switch

Module

ManagementNetwork

Routed ProductionNetwork

2

1 3

Web Interface

1B

1A

2

3 4

4

Legend

Ethernet

I2C path

Serial

Serial path

5

Management

Workstation

Blade

Server

I2C interface

Ethernet path

103

BNT (Nortel) Ethernet switch I/O modules

• BladeCenter chassis Ethernet overview

• Internal chassis traffic flow

• Switch I/O module management connectivity

• BNT (Nortel) Ethernet switch I/O modules

• Server connectivity module

• Pass-Thru module

• High speed I/O modules overview

• Ethernet expansion cards

104

BNT GbESM switch I/O modules: Layer 2-3

• Layer 2/3 copper GbESM

– 14 internal 1 Gb FDX

connections

– 6 external 1 Gb FDX

connections

– 2 10/100 Mb FDX

management ports

• Layer 2/3 Fibre GbESM

– 14 internal 1 Gb FDX

connections

– 6 external 1 Gb FDX fibre (SX)

connections

– 2 10/100 FDX management

ports

BNT Copper and Fibre Layer 2-3 Switch I/O Modules

105

BNT L2/3 GbESM: Simplified block diagram

Gigabit

Ethernet

Accelerator

Ap

plic

ati

on

Sw

itc

h

co

nn

ec

tor

Management

Processor

Memory

Control

Data

6 external 10/100/1000 Mbps Copper Ports

(Auto-negotiation enabled)

Or

6 external 10/100/1000 Mbps Copper Ports

(Auto-negotiation enabled)

And

14 internal 1000 Mbps server links

2 internal 100 Mbps management links

106

BNT L2/3 GbESM: Network interfacesB

lad

e 1

Bla

de

2

Bla

de

3

Bla

de 1

2

Bla

de

13

Bla

de

14

Int1

(1)Int2

(2)

Int3

(3)

Int12

(12)

Int13

(13)

Int14

(14)

Ext1

(17)

Ext2

(18)

Ext5

(21)

Ext6

(22)

Mgt1

(15)

Mgt2

(16)

Management

Module 1

Management

Module 2Cu

sto

mer

Netw

ork

Default VLAN

VLAN 1

Management VLAN

VLAN 4095

IP Interface 128

107

BNT Nortel layer 2-7 GbESM

• The Layer 2-7 GbESM

features:

• Support for up to

300,000 simultaneous

Layer 2-7 sessions

with full wire-speed

packet forwarding for

all connections

• Link aggregation on

the four external ports

• Layer 4 load balancing

for computing

optimization

• Virtual router

redundancy to

facilitate high

application availability

in the event of

application, operating

system or hardware

failures

Layer 2/3Layer 4Layer 5-7

108

Page 19: IBM Bladecenter

19

BNT (Nortel) Layer 2-7 GbESM: Additional function

109

BNT 1/10 Gb uplink Ethernet switch module

• Combined 1 Gb and 10 Gb external ports

• Three slots for 10 Gb Ethernet SFP+ modules

• Six 1 Gb RJ45 ports

• Fourteen internal 1 Gb ports

110

Server connectivity module

• BladeCenter chassis Ethernet overview

• Internal chassis traffic flow

• Switch I/O module management connectivity

• BNT (Nortel) Ethernet switch I/O modules

• Server connectivity module

• Pass-Thru module

• High speed I/O modules overview

• Expansion cards

111

Server connectivity module: Product highlights

• Low-cost Layer 2 device

• Simplified user interface to enable server administrator control of device

• Ideal for SMB customers and applications

• Function summary

– Basic Layer 2 connectivity within chassis

– “Non-networking”, server admin setup and control

– Simple, GUI/CLI interface

• Create blade groups (simple VLANs)

• Create external port groups (auto Etherchannel)

• Enable trunk failover (redundancy option)

• Set external port speed

• Assign tagged VLANs to blade groups

112

Pass-Thru module

• BladeCenter chassis Ethernet overview

• Internal chassis traffic flow

• Switch I/O module management connectivity

• BNT (Nortel) Ethernet switch I/O modules

• Server connectivity module

• Pass-Thru module

• High speed I/O modules overview

• Expansion cards

113

Intelligent copper pass-thru module: Product highlights

• 14 external RJ-45 copper ports for making connections to the external network.

– The external ports can operate at 100 or 1000 Mbps.

• 14 internal bi-directional ports, connected to each of the blade servers

• One-to-one mapping of internal Ethernet ports on the blade to external copper RJ-45

• GUI based operator interface

114

Page 20: IBM Bladecenter

20

High speed I/O modules overview

• BladeCenter chassis Ethernet overview

• Internal chassis traffic flow

• Switch I/O module management connectivity

• BNT (Nortel) Ethernet switch I/O modules

• Server connectivity module

• Pass-Thru module

• High speed I/O modules overview

• Expansion cards

115

High speed I/O modules: Overview

• BNT 10Gb Ethernet Switch I/O Module

– BladeCenter H and HT High Speed Module

– 10 External Ports

– 14 Internal Ports

• BNT 10Gb Ethernet Pass-Thru I/O Module– BladeCenter H and HT High Speed Module

– 14 External Ports

– 14 Internal Ports

BNT 10Gb Ethernet Switch I/O Module

BNT 10Gb Ethernet Pass-Thru I/O Module

116

Expansion cards

• BladeCenter chassis Ethernet overview

• Internal chassis traffic flow

• Switch I/O module management connectivity

• BNT (Nortel) Ethernet switch I/O modules

• Server connectivity module

• Pass-Thru module

• High speed I/O modules overview

• Expansion cards

117

Ethernet expansion cards: Combination I/O vertical (CIOv)

• PCI Express host interface

• Broadcom BCM5709S communication module

• BladeCenter Open Fabric Manager (BOFM) support

• Connection to 1000BASE-X environments using BladeCenter Ethernet switches

IBM HS22 Blade Server with Ethernet CIOv card installed CIOv Ethernet Expansion Card

118

Ethernet expansion cards: Combination form factor vertical (CFFv)

• Based on Broadcom 5704 chipset

• Two 1 Gb ports per blade server

IBM HS22 Blade Server with Ethernet CFFv card installed Ethernet CFFv Expansion Card

119

Ethernet expansion cards: QLogic iSCSI

• Dual ports

• TCP/IP Offload Engine

– TC/IP stack implemented in firmware

• Provides iSCSI diskless boot capability

QLogic iSCSI StFF Expansion CardQLogic iSCSI Management Utility

120

Page 21: IBM Bladecenter

21

Ethernet expansion cards: High speed overview

• High Speed Form horizontal (CFFh) expansion cards

– Interface to Horizontally mounted I/O modules

– BladeCenter H and HT chassis

• PCI-E bus connections

QLogic 2-Port 10Gb Converged Network

Adapter CFFh

Broadcom 10Gb 4-Port CFFh card

IBM BladeCenter 2/4-Port CFFh card

NetXen 10Gb 2-Port CFFh card

121

Key words

• Device Manager

• Failover

• Fibre Ethernet

• Graphical user interface (GUI)

• Local Area Network (LAN)

• Layer 2 – 7

• Media Access Control (MAC) Address

• Power On Self Test (POST)

• Spanning Tree

• Serializer/deserializer (SERDES)

• Server Connectivity Module (SCM)

• Open Shortest Path First (OSPF)

• Uplink

• Virtual Local Area Network (VLAN) tagging

• Virtual Router Redundancy Protocol (VRRP)

• Trunking

122

Checkpoint (1 of 2)

1. True/False: In order to provide Ethernet connectivity, each blade server requires an Ethernet expansion card.

2. A customer requires policy management capability in the application through Transport layers of the OSI model. Select the switch module appropriate for these requirements.

a. BNT 10Gb Pass-thru Module

b. BNT L2/3 Copper GbESM

c. IBM Server Connectivity Module

d. BNT L2-7 GbESM

3. True/False: The Ethernet switch I/O modules can be managed through the AMM management network, or through the switch I/O module external ports

123

Checkpoint solutions (1 of 2)

1. True/False: In order to provide Ethernet connectivity, each blade server requires an Ethernet expansion card.

Answer: False

2. A customer requires policy management capability in the application through Transport layers of the OSI model. Select the switch module appropriate for these requirements.

a. BNT 10Gb Pass-thru Module

b. BNT L2/3 Copper GbESM

c. IBM Server Connectivity Module

d. BNT L2-7 GbESM

Answer: d

3. True/False: The Ethernet switch I/O modules can be managed through the AMM management network, or through the switch I/O module external ports

Answer: True

124

Checkpoint (2 of 2)

4. In order to support 4 Ethernet connections from a Blade server, which of the following is correct?

a. An expansion blade is required

b. Add an Ethernet expansion card and switch module in Bay 2

c. Add an Ethernet expansion card and switch modules in Bays 3 and 4

d. Add an IBM Server connectivity module

5. True/False: NIC Teaming can be used to provide redundancy, utilizing the paths from the blade server onboard NICs through the midplane to Ethernet switch I/O modules in Bays 1 and 2 of the chassis.

125

Checkpoint solutions (2 of 2)

4. In order to support 4 Ethernet connections from a blade server, which of the following is correct?

a. An expansion blade is required

b. Add an Ethernet expansion card and switch module in Bay 2

c. Add an Ethernet expansion card and switch module in Bays 3 and 4

d. Add an IBM Server connectivity module

Answer: c

5. True/False: NIC Teaming can be used to provide redundancy, utilizing the paths from the blade server onboard NICs through the midplane to Ethernet switch I/O modules in Bays 1 and 2 of the chassis.

Answer: True

126

Page 22: IBM Bladecenter

22

Unit summary

Having completing this unit, you should be able to:

• List the BladeCenter components required for Ethernet connectivity

• Name the management components supporting Ethernet switch modules

• Select the Ethernet I/O switch module to meet a specific requirement

• List the components required to provide additional Ethernet connectivity to a blade server

• Identify the method used to provide blade server Ethernet connection redundancy

127 5.3

Storage connectivity and options

Unit objectives

After completing this unit, you should be able to:

• List the internal storage options for the blade server

• Select the appropriate I/O switch module for external storage access

• List the components required for Fibre Channel connectivity

• Select the components required for Fibre Channel boot from SAN

• Describe the functions implemented on the iSCSI expansion card

129

Storage options for BladeCenter: Internal storage

• Internal storage

• Fibre Channel

• iSCSI

• SAS

• Boot from SAN

130

Common components: Storage

• Hard drives

– SAS

– SATA

– SCSI

• Hot swap on selected blade servers

• RAID (0,1)

• Solid state disk (SSD)

• Modular flash drive

Solid State Disk (SSD)

Hard DiskModular Flash Drive

131

Common components: Storage configurations

Modular Flash Drive

Two internal hard drives, non-hot swapTwo internal hard drives, front-mounted hot swap

132

Page 23: IBM Bladecenter

23

IBM expansion blades: Storage and I/O

Three SAS hot-swap

hard drive bays

Blade expansion

connector IBM Blade (HS21, LS21, or LS41)

Storage and

I/O

Expansion

Blade

Storage and I/O Expansion Blade Assembly

RAID 5 (optional battery

cache)

I/O Expansion slot I/O Expansion slot

133

Storage options for BladeCenter: Fibre Channel

• Internal storage

• Fibre Channel

• iSCSI

• SAS

• Boot from SAN

134

Fibre Channel components: Expansion cards

1. Blade Server

2. Fibre Channel Expansion Card

• IBM BladeCenter Fibre Channel expansion cards

– Provide blade server connectivity to remote storage

– BIOS support for boot from SAN

– Provide multi-port redundancy

135

Fibre Channel expansion cards: QLogic

• QLogic Fibre Channel expansion

card models

– 8 Gb CIOv form factor

– 4 Gb CIOv form factor

– 4 Gb SFF form factor

– 4 Gb CFFv form factor

• QLogic combination Fibre Channel and Ethernet models

– 8 Gb CFFh form factor

– 4 Gb CFFh form factor

QLogic 8Gb FC CIOv Expansion Card QLogic 4Gb FC CIOv Expansion Card QLogic 4Gb FC CFFv Expansion Card

QLogic 4Gb Ethernet and FC CFFh Card

QLogic 8Gb Ethernet and FC CFFh Card

136

Fibre Channel expansion cards: Emulex

• Emulex Fibre Channel expansion

card models

– Emulex 8 Gb Fibre Channel expansion card

• Two 8 Gb ports

• CIOv form factor

– Emulex 4 Gb Fibre Channel expansion card

• Two 4 Gb ports

• CFFv form factor

Emulex 8 Gb FC Expansion Card (CIOv) Emulex 4 Gb FC Expansion Card (CFFv)

137

Fibre Channel storage components: Switch I/O modules

3. BladeCenter Chassis

4. Fibre Channel Switch I/O Modules

• IBM BladeCenter Fibre Channel Switch I/O Modules– QLogic 20-port 8 Gb SAN Switch Module

– QLogic 10-port and 20-port 4Gb SAN Switch Module

– Brocade 10 and 20 port SAN Switch Module

• IBM BladeCenter Fibre Channel Pass-Thru I/O Modules

– QLogic Intelligent 8 Gb Pass-Thru Fibre Channel Module

– QLogic Intelligent 4 Gb Pass-Thru Fibre Channel Module

138

Page 24: IBM Bladecenter

24

Definition of 10-port and 20-port

For the 10-port offerings, each switch module has three

external ports enabled and seven internal ports enabled

For the 20-port offerings, each switch module has all six external

ports enabled and all fourteen internal ports enabled

Blade

Bay

1

Blade

Bay

2

Blade

Bay

3

Blade

Bay

4

Blade

Bay

5

Blade

Bay

6

Blade

Bay

7

Blade

Bay

8

Blade

Bay

9

Blade

Bay

10

Blade

Bay

11

Blade

Bay

12

Blade

Bay

13

Blade

Bay

14

Blade

Bay

1

Blade

Bay

2

Blade

Bay

3

Blade

Bay

4

Blade

Bay

5

Blade

Bay

6

Blade

Bay

7

Internal Parts

Internal Parts

139

QLogic 10 and 20 Port FC switch modules

• QLogic 8 Gb SAN Switch I/O Module

– 20 Ports (14 internal, 6 external

– Supports 2, 4 and 8 Gb speeds

• QLogic 4 Gb SAN Switch I/O Module

– Affordable 10-port and 20-port offerings available - ideal for Small, Medium and Enterprise business needs

– Easy-to-use software upgrade doubles 10-port switch connectivity to 20-ports for on demand scalability

– Included with every switch, QLogic's SANsurfer Management Suite eases installation, configuration and management of your SAN infrastructure all from one GUI

140

QLogic intelligent pass-thru Fibre Channel modules

• QLogic 8 Gb and 4 Gb Intelligent Pass-Thru Modules

– 20 port modules provide standards-based NPIV interface

– Zoning and LUN Masking

– Alternative to traditional optical pass-thru

– 6 External auto-sensing ports

– 14 Internal auto-sensing ports

141

Brocade 10 and 20 port Fibre Channel switch module

• 1, 2 and 4 Gigabits per second bandwidth

• Automatic failover support

• 10-port and 20-port

• Non-disruptive upgrade from 10-port switch connectivity to 20-ports

• Utilizes Brocades Silkworm and interoperable with Silkworm and IBM TotalStorage b-type SAN switches

142

Managing the Fibre Channel switch modules

143

Interaction with the management module

• The switch interacts with the Blade Center Management Module via an Inter-Integrated Circuit (I2C)* device to transfer Vital Product Data (VPD)

• User can perform “write erase” using the “reset to factory defaults” control on the MM’s GUI.

• User can disable ALL external ports via the MM GUI.

• The switch can be power cycled via the MM’s GUI.

• The switch provides some information to the MM.

– Temperature alarms

– Switch name

– Port License State

– SFP information

– POST status

– URL to launch the Device manager

144

Page 25: IBM Bladecenter

25

BladeCenter AMM Interface: Admin/Power/Restart

145

BladeCenter AMM interface: Configuration

146

Establishing a remote session

The default IP addresses and

commands to open a Telnet command-

prompt session open are:

• For the switch module in I/O bay 3:

– telnet 192.168.70.129

• For the switch module in I/O bay 4:

– telnet 192.168.70.130

147

SAN Browser for the QLogic switch module

• SAN Browser graphical (Java) user interface:– Allows for viewing and

configuration Ports, Zoning and Network (including SNMP alerting) settings.

– Troubleshooting through Diagnostic functionalities

– Multiplatform support:

– Requires 128 MB RAM, 150 MB of disk space for installation, 300 MHz CPU and either Microsoft Internet Explorer or Netscape Navigator

• ANSI Command Line Interface

– Telnet directly onto IP address of the module

– Functionally equivalent to the Java application

148

Fabric Tree

Switch Icons appear in

the Graphic Window

Data Window

Data Window Tabs

for different displays

SAN Browser topological view

149

SAN Browser faceplate

Switch Name Status

Data Window Tabs

for different displays

150

Page 26: IBM Bladecenter

26

SAN Browser zoning

• Zoning can be accomplished from Enterprise Fabric Connectivity Manager (EFCM) for the Switch Modules

• Since Zoning is a Fabric Service, it can also be managed from EFCM

151

SAN Browser functions

152

Establishing a Web interface session for Brocade

To establish a Web-interface session through a Web browser, open a supported Web browser on the network-management workstation and type one of the following default IP addresses in the Web address field and press Enter.

• For I/O bay 3: http://192.168.70.129

• For I/O bay 4: http://192.168.70.130

The I/O-module Advanced Web Tools interface window will then open.

153

Brocade switch administration

To access the Switch Admin

window:

• Select the desired switch from

the Fabric Tree.

The selected switch appears in

the Switch view.

• Click the Admin icon from the

Switch view.

The login dialog box displays.

Enter the admin user name

and password.

• Click the desired tab.

154

Brocade switch zone administration

To Create a Zone• Click Create

• The Create New Config dialog

box appears

• Enter a name for the new

configuration and click OK

• Click on “+” signs in the

Member Selection List to

view the nested elements

• Highlight an element in the

Member Selection List that

you want to include

• Click Add Member to add

configuration members.

155

Configuring the expansion card

• Boot the blade server.

• Press Ctrl+Q to enter

the BIOS configuration

utility.

• Select the first Fibre

Channel adapter port

(port 1 correlates to the

top Fibre Channel

Switch in slot 3 of the

BladeCenter chassis).

• Select Configuration

Settings and Host

Adapter Settings.

156

Page 27: IBM Bladecenter

27

Fibre Channel storage components: Disk systems

• IBM BladeCenter Boot Disk System

• IBM System Storage DS3400

• IBM System Storage DS3200

157

Storage options for BladeCenter: iSCSI

• Internal storage

• Fibre Channel

• iSCSI

• SAS

• Boot from SAN

158

What is iSCSI?

• iSCSI: Internet SCSI (Small Computer System Interface), SCSI

commands sent across a network in

TCP/IP packets. It was developed as a storage networking standard for

linking data storage facilities

• Initiator: System making the iSCSI requests for data

• Target: System receiving the iSCSI requests for data

159

iSCSI expansion card overview

The iSCSI expansion card offers a connection to an iSCSI storage device, via iSCSI host connections, to leverage the available standard Ethernet infrastructure to offer storage area network (SAN) solutions

iSCSI Storage Device

Blade Server

Via TCP

160

Storage options for BladeCenter: SAS

• Internal storage

• Fibre Channel

• iSCSI

• SAS

• Boot from SAN

161

IBM BladeCenter storage: SAS expansion cards

• SAS Expansion Card

– Configurable SAS device

– Form factor CFFv card

– Enables Boot from IBM BladeCenter Boot Disk System

– Connects through SAS I/O Modules in module bays 3 and 4

• SAS Connectivity Card

– Uses SAS controller ports on Blade server

– Form factor CFFv

– Connects through SAS I/O Modules in module bays 3 and 4

SAS Expansion Card SAS Connectivity Card

162

Page 28: IBM Bladecenter

28

IBM BladeCenter storage: SAS connectivity module

• Provides SAS connectivity from Blade server SAS expansion card to external SAS

storage

– 14 Internal connections to BladeServer blade bays

– 4 External connections to SAS infrastructure

Sample connection using dual SAS Connectivity Modules IBM BladeCenter SAS Connectivity Module

163

IBM BladeCenter storage: DS3200 SAS controller

• 3 Gbps Serial Attached SCSI (SAS) host

interface and drive

expansion technology

• Combination of 12 SAS or SATA 3.5" drives per

enclosure

• Scalable to 5.4 TB of storage

IBM DS3200

IBM DS3200 Rear Chassis, Dual SAS Controllers

164

Storage options for BladeCenter: Boot from SAN

• Internal storage

• Fibre Channel

• iSCSI

• SAS

• Boot from SAN

165

In this example, we connect the BladeCenter to an IBM TotalStorage DS4500 through two Fibre Channel Switches. Both FC Switch modules are in use.

Booting from SAN

Boot LUN

Blade Server

166

Benefits to booting from SAN

• Improving disaster tolerance If a server becomes faulty, unavailable or compromised, it can

be swapped out for a new server with the same configuration, without having to:

– Reconstruct the booting, operating and application environments for the new machine when it is most

urgently needed.

– Swap out hard drives, reconfigure arrays, and restore data and applications from backup.

– The ability to replace and add servers in minutes provides a significant return on investment (ROI) by

preventing lengthy downtime of core servers.

• Enabling centralized administration

• Reducing TCO through diskless servers

• Using high availability (HA) storage ensuring business continuance

167

Implementation considerations of booting from SAN

• The server must have an FC HBA equipped with a boot BIOS that can support booting from the storage device attached to the SAN

• Typically the host needs to have a boot order of CD-ROM, diskette, and then drive 0

• The SAN must be installed and configured, and the storage visible to the host

• The storage system must have at least one available LUN for booting the server(s)

• The storage must be configured to allow the server exclusive access to the LUN; access is typically assigned via the FC HBA’s WorldWide Port Name (WWPN) on the storage

• To minimize data interruptions between hosts, it is best to keep hosts separated from each other by way of zoning to minimize the Registered State Change Notification (RSCN) interruptions

168

Page 29: IBM Bladecenter

29

Booting from SAN: Creating the LUNs

18 GB 18 GB 18 GB

Array A (RAID 5)

9 GB Logical Drive

1. Create an array

2. Create a logical drive

(LUN)

3. From the blade,

broadcast the WWPN

4. Map the LUN to the

host via the WWPN

5. Assign the Boot LUN to

HBA Port A

6. Install the operating

system

169

Booting from SAN: Zoning the switch

• SAN Zoning provides partitioning of resources

• Zoning is done at the fabric switch level

• Zones can be created at port level

– HBA

– Switch

– SAN Controller

• Zones can also be created at the

WWN level

– HBA

– Switch

– SAN Controller

Zoning by WWN

Zoning by Port

170

Configuring the expansion card BIOS

• Boot the blade server.

• Press Ctrl+Q to enter

the BIOS configuration

utility.

• Select the first Fibre

Channel adapter port

(port 1 correlates to the

top Fibre Channel

Switch in slot 3 of the

BladeCenter chassis).

• Select Configuration

Settings and Host

Adapter Settings.

171

Broadcasting the world wide name (WWN)

In order for the storage

device to see the HBA

ports, you need to open

the port by doing a Scan

Fibre Devices. Do this

for both ports.

172

Mapping the logical drive to the WWN

173

Assign the boot LUN to an HBA port

We want to assign

the boot LUN to port

B also, but we will

disable this path

during the initial OS

installation

174

Page 30: IBM Bladecenter

30

Installing the operating system

• You must use the Fibre card Driver diskette during the installation phase (press F6 for

boot disk during install)

• During the POST sequence notice the following text:

Drive letter C: is moved to the Drive letter D:

Loop ID 129,0 is installed as Drive C:

• This indicates that the blade is now booting from the LUN 0 associated to the first HBA

adapter

175

Key words

• Access Control List (ACL)

• Basic Input/Output System (BIOS)

• Boot from SAN

• Fibre Channel

• Fabric

• Just a bunch of disks (JBOD)

• InfiniBand

• Internet Small Computer System Interface (iSCSI)

• Logical Unit Number (LUN)

• Serial-attached SCSI (SAS)

• Small Computer System Interface (SCSI)

• Small Form Factor (SmFF)

• Standard Form Factor (StFF)

• Storage Area Network (SAN)

• Zoning

• LUN masking

• World Wide Name (WWN)

176

Checkpoint (1 of 2)

1. True/False: Blade server local disk options include hot-swap capability.

2. To enable fibre channel (FC) SAN connectivity, which of the following

statements is true?

a.The BladeCenter chassis must have a fibre channel controller

installed in I/O bay 2

b. Internal cabling must be performed from the Blade server to the

fibre channel switch

c. The Blade server fibre channel expansion card (HBA) World Wide

Name (WWN) must be known to the external fibre channel

controller

d.The BladeCenter Ethernet Switch I/O module must be connected to

the external fibre channel controller

3. True/False: The QLogic iSCSI TOE Expansion card contains an iSCSI

Initiator

177

Checkpoint solutions (1 of 2)

1. True/False: Blade server local disk options include hot-swap capability.

Answer: True

2. To enable fibre channel (FC) SAN connectivity, which of the following statements is true?

a. The BladeCenter chassis must have a fibre channel controller installed in I/O bay 2

b. Internal cabling must be performed from the Blade server to the fibre channel switch

c. The Blade server fibre channel expansion card (HBA) World Wide Name (WWN) must be known to the external fibre channel controller

d. The BladeCenter Ethernet Switch I/O module must be connected to the external fibre channel controller

Answer: c

3. True/False: The QLogic iSCSI TOE Expansion card contains an iSCSI Initiator

Answer: True

178

Checkpoint (2 of 2)

4. Select the correct statement regarding Boot from SAN.

a. The storage controller requires a minimum of (4) available boot LUNs

b. A parallel Ethernet connection must exist for each Blade server using fibre channel

c. The I/O Module Advanced Configuration page in the AMM is used to enable the Blade server fibre channel expansion card BIOS

d. The components required for Boot from SAN include: Blade server fibre channel expansion card, fibre channel switch I/O module and external fibre channel controller

5. True/False: To connect from the BladeCenter chassis to an external Fibre Channel fabric requires either a Fibre Channel switch module or an SAS Connectivity module.

6. True/False: Boot from SAN required steps include: Configuring the fibre channel expansion card BIOS and configuring a LUN at the fibre channel storage controller.

179

Checkpoint solutions (2 of 2)

4. Select the correct statement regarding Boot from SAN.

a. The storage controller requires a minimum of (4) available boot LUNs

b. A parallel Ethernet connection must exist for each Blade server using fibre channel

c. The I/O Module Advanced Configuration page in the AMM is used to enable the Blade server fibre channel expansion card BIOS

d. The components required for Boot from SAN include: Blade server fibre channel expansion card, fibre channel switch I/O module and external fibre channel controller

Answer: d

5. True/False: To connect from the BladeCenter chassis to an external Fibre Channel fabric requires either a Fibre Channel switch module or an SAS Connectivity module.

Answer: False

6. True/False: Boot from SAN required steps include: Configuring the fibre channel expansion card BIOS and configuring a LUN at the fibre channel storage controller.

Answer: True

180

Page 31: IBM Bladecenter

31

Unit summary

Having completed this unit, you should be able to:

• List the internal storage options for the blade server

• Select the appropriate I/O switch module for external storage access

• List the components required for Fibre Channel connectivity

• Select the components required for Fibre Channel boot from SAN

• Describe the functions implemented on the iSCSI expansion card

181 5.3

Installation and troubleshooting

overview

Unit objectives

After completing this unit, you should be able to:

• Identify the BladeCenter components used to provide PD information

• List the planning elements required for the BladeCenter management network

• Select the functions available to modify firmware settings

• List the blade server indicators and Light Path Components

• Select the steps appropriate in diagnosing blade server hardware failures

• Identify the utility to use in displaying BladeCenter component health

183

Best practices

• Best practices

• Troubleshooting and problem determination

• BladeCenter management interfaces

• Firmware updates and settings

• Information gathering

• IBM BladeCenter support resources

184

BladeCenter chassis questions: Requirements

• Given your specific needs, what is the best BladeCenter solution (in terms of components) necessary to

meet your requirements?

• Define the networking and SAN requirements for your BladeCenter environment based on your existing

infrastructure, including fault tolerance, throughput and interoperability.

• Do you plan on having a separate Management LAN and production LAN? What is the

advantage/disadvantage of this environment?

• Are all of the components being installed in the BladeCenter chassis on the ServerProven list?

• Is this BladeCenter chassis to be deployed locally

or in a remote location?

185

Blade server considerations: Questions

• Is the blade server at the latest firmware level?

If not, what method of applying the latest

firmware updates are you going to implement?

• Besides the BIOS, what other firmware

updates are needed for the blade server?

• What operating system are you going to put on

the blade server. How do I find out if this OS is

supported on the blade server?

• What are the different deployment methods for

operating system installations, and which

method makes the most sense in my

environment?

• What performance requirements are needed

out of my blade server? Based upon these

requirements, which model best fits my

business needs?

186

Page 32: IBM Bladecenter

32

BladeCenter chassis questions: Power

• Do you understand the necessary power requirements for a given BladeCenter solution?

• Will your BladeCenter chassis be connected to either a front-end or high-density front-end

rack PDU?

• How many blade servers are in the chassis and will that impact oversubscription of the

power domains?

• Do you have the correct electrical connectors to power your new BladeCenters and their

PDUs?

187

Cooling questions

• Are the systems on a raised floor?

• How many BTUs am I generating when my installation is complete?

• What are the power requirements for the new systems?

• Are there plans to grow in the future?

188

Troubleshooting and problem determination

• Best practices

• Troubleshooting and problem determination

• BladeCenter management interfaces

• Firmware updates and settings

• Information gathering

• IBM BladeCenter support resources

189

Problem determination: Information gathering

• Due to the variety of hardware and software combinations that can be encountered, use the following information to assist you in problem determination. If possible, have this information available when requesting assistance from Service Support and Engineering functions.

– Machine type and model

– Microprocessor or hard disk upgrades

– Failure symptom

• Do diagnostics fail?

• What, when, where, single, or multiple systems?

• Is the failure repeatable?

• Has this configuration ever worked?

• If it has been working, what changes were made prior to it failing?

• Is this the original reported failure?

– Diagnostics version — type and version level

– Hardware configuration

• Print (print screen) configuration currently in use

• BIOS level

– Operating system software — type and version level

190

Blade servers: Diagnostics tools

• Light Path Diagnostics

• Standalone diagnostics

• Diagnostics by PC Doctor

– Test results are stored in a test log

– Management Module event logs contain system status messages from the

blade server service processor and can be:

• Viewed

• Saved to diskette

• Printed

• Attached to e-mail alerts

– Standard log is a summary of tests

– Press <Tab> while viewing the test log

• Power On Self Test (POST) beep codes

• Unified Extensible Firmware Interface (UEFI)

– Elimination of Beep Codes

– Advanced logging and firmware control

• Command-line interface (CLI)

191

IBM Blade Server: Front panel LEDs HS22 example

IBM HS22 Blade Server Front

Panel indicators and controls

HS22 Blade Server Front Panel

192

Page 33: IBM Bladecenter

33

IBM Blade Server: System board diagnostic indicators

HS22 example

IBM Blade Server HS22 System Board Indicators

HS22 System Board Light Path Panel

• IBM HS22 Blade server system board example

– Memory, processor, and disk Indicators

– Light Path Panel

193

LS22 Blade Server Front Panel Controls and Indicators

IBM LS22 Blade Server Front Panel

IBM Blade Server: Front panel LEDs LS22 example

194

LS22 Blade Server System Board Light Path Panel

IBM LS22 Blade

Server System Board

IBM Blade Server: System board diagnostic indicators LS22 example

195

IBM Blade Server: Diagnostics tools

• Light Path Diagnostics

• Press F2 at POST to invoke standalone diagnostics

• Diagnostics by PC Doctor

– Test results are stored in a test log

– Management Module event logs contain system status messages from the blade server service processor and can be:

• Viewed

• Saved to diskette

• Printed

• Attached to e-mail alerts

– Standard log is a summary of tests

– Press <Tab> while viewing the test log

• Power On Self Test (POST) beep codes

• Real time diagnostics

• Command-line interface (CLI)

196

Blade server: Basic input/output system (BIOS)

• Blade server BIOS

– Menu-driven setup

– Settings for configuration and performance

– Set, change, delete (IRQ, date and time, and Passwords)

– Advanced settings for specific needs (for example, memory, CPU, PCI bus and BMC)

– BIOS defaults

• Flash diskette

• BIOS updates for host and devices CD-ROM BIOS/firmware updates and configuration for

host and devices

• BIOS system board jumpers or switches

– BIOS boot selection

– Password override

– Wake on LAN enablement

197

UEFI: Unified Extensible Firmware Interface (1 of 3)

• The next generation of BIOS

• Allows OSs to take full advantage of the hardware

– Architecture independent

– Modular

• 64-bit code architecture

• 16 TB of memory can be addressed

• More functionality

– Adapter vendors can add more features in their options (for example, IPv6)

– Design allows faster updates as new features are introduced

– More adaptors can be installed and used simultaneously

– Fully backwards compatible with legacy BIOS

• Better user interface

– Replaces ctrl key sequences with a more intuitive human interface

– Moves adaptor and iSCSI configuration into F1 setup

– Creates human readable event logs

• Easier management

– Eliminates “beep” codes; all errors can now be covered by Light Path

– Reduces the number of error messages and eliminates out-dated errors

– Can be managed both in-band and out of band

198

Page 34: IBM Bladecenter

34

UEFI: Unified Extensible Firmware Interface (2 of 3)

BMC

RSAII

Diags BIOS

xFlash

ASU

Configuration

&

IMMPb

DSAUEFI

Configuration

&xFlash

ASU

Today’s update

and configuration

on systems

Tomorrow’s update

and configuration

on systems

199

UEFI: Unified Extensible Firmware Interface (3 of 3)

UEFI BIOS64 bit code architecture: 16 TB of memory can be

addressed

16 bit code architecture: Only 1MB of memory can be

addressed.

Eliminates Code Space Constraints. Adapter Option ROMs

can be loaded anywhere in memory with no size

restrictions.

Adapter Vendors must fit all option code into a shared

128K. Limits the number of adapters that can be

effectively installed.

Adapter vendors are free to add function. i.e. IPV6 Vendors are limited in the function they can provide in the

option ROM.

UEFI defines a Human Interface that is being extended to

Adapter Vendors.

Cryptic Ctrl Key sequences required for configuring

Adapters.

iSCSI Configuration is in F1 Setup and consolidated in to

ASU.

iSCSI Configuration requires separate tool.

Elimination of Beep Codes – All Errors covered by Light

Path. Reduction in Number of Error Messages.

Multiple Beep Codes for fundamental failures.

Adapter Configuration can move into F1 Setup. Eliminates

Ctrl Key sequences for configuring Adapters.

Advanced Settings Utility (ASU) has partial coverage of F1

Settings

In & Out of Band UEFI Updates. Settings accessed Out of

Band via ASU and the IMM.

In-Band only updates via DOS, wFlash, or lFlash.

UEFI Event codes available out of band. Human readable

Event logs in F1 Setup

Numerous Legacy POST Errors.

UEFI versus BIOS

200

Blade server: Integrated Management Module (IMM)

• Integrated Management Module (IMM)

– Replacement for BMC

– LAN over USB

– OS drivers included in Windows and Linux

201

Blade server six system states

System State Data Gathering Data Analysis

1 There is no AC Visual PDSG

2 There is AC power but no DC

Advanced Management Module

(AMM) & (IMM)

Light Path

System event log

3There is AC and DC power but the system fails

to complete post

Checkpoint codes

F1 and F2

Beep codes (prior to UEFI)

Adapter BIOS messages

PDSG

Retain tips

IBM Support Web site

4

There is AC and DC power, the system

completes POST but the NOS fails to start

loading

F2 diagnostics PDSG

Retain tips

5

There is AC and DC power, the system

completes POST but the NOS fails to complete

loading

NOS boot messages

'Blue Screen'

'Safe' mode

NOS Vendor messages

6

There is AC and DC power, the system

completes POST and the NOS completes loading

but stops during operation

DSA

NOS event logs DSA

202

Advanced Management Modules (AMM): Overview

Video Connector

Power-on LEDS

Serial Console

Connector

RJ45

10/100 Ethernet

Connector

RJ45

USB Dual Stack

Pin-hole Reset

Release handle

MAC

Address

Activity LEDS Error LEDS

Port Link LED

Port Activity LED

Advanced Management Module LEDS

• The Management Module stores all event and error information for the BladeCenter

• The Management Module configuration data is stored both in itself and on the midplane– To reset the IP address back to the default settings, press and hold the IP reset button for 3

seconds or less

203

Recovering Management Module TCP/IP address

• MM configuration data is stored in the midplane

– To reset a TCP/IP address only:

• Remove the cable from the MM Ethernet port

• Press and hold the IP reset button for 3 seconds or less

– TCP/IP address will reset to 192.168.70.125/255.255.255.0

– Simply replacing the MM will cause the replacement MM to adopt the same values as the original MM

• PERFORM ALL RESET STEPS BEFORE REPLACING THE MM

204

Page 35: IBM Bladecenter

35

Management Module full reset: Factory defaults

• MM configuration data is stored in the midplane

– To force a complete MM reset (including password):

• Remove the cable from the MM ethernet port

• Press and hold the IP reset button for 5 seconds

• Release the IP reset button for 5 seconds

• Press and hold the IP reset button for 10 seconds

– TCP/IP address will be reset to 192.168.70.125/255.255.255.0

– All IDs and passwords will be deleted (except USERID/PASSW0RD)

– Simply replacing the MM will cause the replacement MM to adopt the same values as the original MM

• PERFORM ALL RESET STEPS BEFOIRE REPLACING THE MM

205

Advanced management event log

206

Problem determination: Blade server example

• Example of a memory DIMM problem

– Display of BladeCenter Front Panel LEDs

Management Module web interface indicating error LEDs

207

Problem determination: Blade server example

• Example of a memory DIMM problem

– Display of the Blade server front panel LEDs

Advanced Management Module Blade server LEDs

208

Problem determination: Blade server example

• Example of a memory DIMM problem

– Display of the BladeCenter Event Log

Advanced Management Module Event Log

209

Problem determination: Blade server example

• Using the IBM Problem Determination guide - IBM BladeCenter HS21

– Locate the error symptom code in the log (in this example: 289)

– Match the table entry to the code

Check POST error

log

for error message

289:

210

Page 36: IBM Bladecenter

36

Problem determination: Blade server example

• Consult the IBM Installation Guide for the HS21

– Proper DIMM installation procedure

HS21 DIMM Installation slot and order

211

Problem determination: Blade server example

• Verifying fix and proper operation

AMM Status Display and Event Log

212

Problem determination: Blade servers

• What do you do if:

– Blade server powered down for no apparent reason

– Blade server does not power on, the system-error LED on the BladeCenter system-LED panel is lit, the

blade error LED on the blade server LED panel is lit, and the system-error log contains the following

message: ″CPUs Mismatched″

– Some components do not report environmental status (temperature, voltage)

– Switching KVM control between blade servers gives USB device error

213

Ethernet switch modules: Addressing issues

• What do you do if:

– You have duplicate IP address reported on the ESM

– You have duplicate IP address reported on the blade server

– You have a native VLAN mismatch reported on the ESM

– There are connection problems to the blade servers

– The DHCP server uses up all IP addresses and the blade server

still cannot get an address

214

Problem determination: Ethernet switch I/O modules

• Hardware failures

• Not very common

– On MM, look under I/O Module Tasks ->

Power/Restart to see diagnostic code after reboot.

Also look at fault LED on the Ethernet Switch

Module

• Software Failures

– Not very common

– As with all products, software bugs do exist

– Reference the latest code readme file for a list of

resolved bugs with each release of code

• Misconfiguration of Ethernet Switch Module or

other component

– This is the most common issue encountered

– Often requires close cooperation between different

administrative groups to resolve

215

Ethernet switch modules: Configuration issues

• Most common issue encountered

– May be with the Ethernet Switch Module, a device upstream or the server within the BladeCenter

– May also be misconfiguration on the Management Module

• Same tools used to troubleshoot configuration issues can also be used to help isolate broken

hardware and software bugs

• Usually requires close cooperation between network administrators and server administrators

• Often helps to have special tools (for example, network sniffer) to understand and resolve

problem

216

Page 37: IBM Bladecenter

37

Ethernet switch modules: Basic rules

• Do not attach cables to the ESM until both sides of the connection are configured

• Do not put the blade servers on the VLAN that the ESM uses for its

management VLAN interface

• Make sure the ESM firmware (IOS) code is upgraded

• Decide the ESM management path (via Management Module or ESM uplinks)

and configure for it

217

BladeCenter management interfaces

• Best practices

• Troubleshooting and problem determination

• BladeCenter management interfaces

• Firmware updates and settings

• Information gathering

• IBM BladeCenter support resources

218

BladeCenter AMM: System status screen

Navigation

menu

Main

information window

219

System Event Log (SEL) screen

• This screen shows event history of the BladeCenter

220

Hardware Vital Product Data (VPD)

• This screen shows information relating to the hardware in the BladeCenter

221

Rules for I/O module management

• In-band management

– Use the AMM path to an I/O module

• Provides centralized management of all I/O modules

– All activities and reporting is through a single Ethernet port

– Makes LAN configuration easier

• Requires MM and all I/O modules to be on the same IP subnet

• Out-of-band management

– Requires enablement of external management over all ports

• May require management VLAN configuration

• Access will involve many Ethernet ports

• I/O module need not be on the same IP subnet as the MM

– If subnets are different, AMM path to I/O module is unavailable

222

Page 38: IBM Bladecenter

38

I/O module tasks: Close up

223

I/O module tasks: Advanced switch management

224

Ethernet switch I/O module Web interface

225

CIGESM Web interface

226

Nortel ESM Web interface

227

Fibre Channel switch module Web interface

• SAN Utility (QLogic)

– Full Function GUI

• SAN Browser (Qlogic)

– Limited functionality

• Switch Explorer (Brocade)

– Limited functionality

228

Page 39: IBM Bladecenter

39

Firmware updates and settings

• Best practices

• Troubleshooting and problem determination

• BladeCenter management interfaces

• Firmware updates and settings

• Information gathering

• IBM BladeCenter support resources

229

UpdateXpress CD-ROM package

• UpdateXpress

– Bootable CD-ROM

– Supports maintenance of system firmware and Windows device drivers

• Automatically detects current device-driver and firmware levels

• Gives the option of selecting specific upgrades or allowing UpdateXpress to update all of the system levels it detected as

needing upgrades

• Can be installed using local DVD or over network using the AMM

230

UpdateXpress firmware update scripts

• UpdateXpress Firmware Update Scripts for BladeCenter (UXBC)

– Process that enables firmware updates to be run in a remote, unattended fashion

• Requires a management station and supporting software

– Windows or Linux OS

– FTP and TFTP servers somewhere on the management LAN

– UXBC discovery and deployment components

– For more information, see

– http://www-03.ibm.com/systems/management/uxs.html

231

IBM preboot dynamic system analysis

• Provides problem isolation,

configuration analysis, error

log collection

– Collects information about:

• System configuration

• Network interfaces and settings

• Installed hardware

• Light path diagnostics status

• Service processor status and configuration

• Vital product data, firmware, and UEFI configuration

• Hard disk drive health

232

Advanced settings utility

• Enables the user to modify firmware settings from the command line

– Supported on multiple operating system platforms

– Enables remote changes to POST and BIOS settings

• Does not require F1 access to a console session

– Supports scripting through a batch processing mode

– Does not update any of the firmware code

– For more information, see

– http://www-

304.ibm.com/systems/support/supportsite.wss/docdisplay?brandind=5000008&lndocid=MIGR-55021

233

Information gathering

• Best practices

• Troubleshooting and problem determination

• BladeCenter management interfaces

• Firmware updates and settings

• Information gathering

• IBM BladeCenter support resources

234

Page 40: IBM Bladecenter

40

Data gathering

• Read the BladeCenter data collection guide

– Contains details of what logs and information are needed for escalations

– Contains a step-by-step guide on how the logs are collected

– For more information, see

– http://www-304.ibm.com/systems/support/supportsite.wss/docdisplay?lndocid=SERV-

BLADE&brandind=5000008

235

Gathering information from blade servers

• Blade server logs can be gathered within the operating system

– Use the following table to determine what utility to use

Type of blade server Operating system Type of gathering utility:

HS Series Windows Dynamic System Analysis

HS Series Linux Dynamic System Analysis

LS Series Windows Dynamic System Analysis

LS Series Linux Dynamic System

Analyses

JS Series Linux SNAP

JS Series AIX SNAP

SNAP is built into AIX and SNAP for Linux on Power can be found at:

http://techsupport.services.ibm.com/server/lopdiags.

236

Gathering information from I/O switch modules

• Logs from a Brocade, Cisco, BNT or QLogic switch module

can be captured within the switch interface

– Enable capture text/console logging within the telnet application

– Login to the switch using telnet

– Issue the command from the table below

Type of switch: Command:

Brocade showSupport

Cisco show tech-support

Nortel maint/tsdmp

Qlogic support show

237

IBM BladeCenter support resources

• Best practices

• Troubleshooting and problem determination

• BladeCenter management interfaces

• Firmware updates and settings

• Information gathering

• IBM BladeCenter support resources

238

IBM support Web site

• New central Web site for all server products:

http://www-304.ibm.com/systems/support/

– Select BladeCenter from the drop-down menu

239

Documentation

• Hardware Maintenance Manual

– Available electronically (Adobe Acrobat .PDF format) from the IBM support Web site

• Primary support document for diagnostics and troubleshooting

• User’s Guide, Installation Guide

– System documentation that ships with the BladeCenter and with options such as blade servers and

switch modules

• Useful for confirming shipping group contents (missing parts, and so on) and initial customer setup

240

Page 41: IBM Bladecenter

41

IBM Blade Server references

• IBM BladeCenter Products and Technology

– http://www.redbooks.ibm.com/cgi-bin/searchsite.cgi?query=bladecenter

• IBM ServerProven – Compatibility for BladeCenter Products

– http://www-03.ibm.com/servers/eserver/serverproven/compat/us/

• System x Reference (xREF)

– http://www.redbooks.ibm.com/xref/usxref.pdf

• Intel Products

– http://www.intel.com/products/server/processors/index.htm

• AMD Products

– http://www.amd.com/us/products/server/Pages/server.aspx

241

Key words

• Advanced Management Module (AMM)

• Alternating Current (AC)

• Basic Input/Output System (BIOS)

• British thermal unit (BTU)

• Central Processing Unit (CPU)

• Cisco Intelligent Gigabit Ethernet Switch Module (CIGESM)

• Command-line interface (CLI)

• Compact Disc Read-Only Memory (CD-ROM)

• Dynamic Host Configuration Protocol (DHCP)

• Ethernet switch modules (ESM)

• Fibre Channel Switch Module (FSCM)

• File Transfer Protocol (FTP)

• Graphical User Interface (GUI)

• IBM BladeCenter E (Enterprise)

• IBM BladeCenter H (High Performance)

• IBM BladeCenter HT (High Performance Telco)

• IBM BladeCenter S (Simplification)

• IBM BladeCenter T (Telco)

• Integrated Management Module (IMM)

• Input-output (I/O)

• Internet Protocol (IP)

• Interrupt Request (IRQ)

• Jumper (J)

• Keyboard, Video, and Mouse (KVM)

• Local-Area Network (LAN)

• Management Module (MM)

• Non-Maskable Interrupt (NMI)

• Operating System (OS)

• Peripheral Component Interconnect (PCI)

• Power Distribution Unit (PDU)

• Power On Self Test (POST)

• Remote Supervisor Adapter II (RSA II)

• Secure Sockets Layer (SSL)

• Serial over LAN (SoL)

• Servcie Pack (SP)

• Service Support Representative ( SSR )

• Simple Mail Transfer Protocol (SMTP)

• Simple Network Management Protocol (SNMP)

• Storage Area Network (SAN)

• System Event Log (SEL)

• Transmission Control Protocol (TCP)

• Trivial File Transfer Protocol (TFTP)

• Unified Extensible Firmware Interface (UEFI)

• UpdateXpress Firmware Update Scripts for BladeCenter (UXBC)

• Virtual Local Area Network (VLAN)

• Vital Product Data (VPD)

• Volt (V)

• Watt (W)

242

Checkpoint (1 of 2)

1. The _______________________ stores all major event and error

information for the BladeCenter and is the starting point for PD.

a. Ethernet Switch Module (ESM)

b. AMM

c. BIOS

d. Blade Server operating system log

2. True/False: In planning the BladeCenter management network,

bandwidth is the primary consideration.

3. The __________ enables the user to modify firmware settings from

the command line.

4. True/False: While AMM management can be done through a Web

interface, all switch modules must be configured using command

line.

243

Checkpoint solutions (1 of 2)

1. The _______________________ stores all major event and error information for the BladeCenter and is the starting point for PD.

a. Ethernet Switch Module (ESM)

b. AMM

c. BIOS

d. Blade Server operating system log

Answer: b

2. True/False: In planning the BladeCenter management network, bandwidth is the primary consideration.

Answer: False

3. The __________ enables the user to modify firmware settings from the command line.

Answer: Advanced Settings Utility (ASU)

4. True/False: While AMM management can be done through a Web interface, all switch modules must be configured using command line.

Answer: False

244

Checkpoint (2 of 2)

5. Select the correct statement regarding Blade Server status indicators.

a. Memory and processor LEDs are on the Blade Server front panel

b. All Blade Server status LEDs are on the Light Path diagnostics panel

c. Blade Server status and error LEDs are on the Front Panel, Control Panel and adjacent to components on the system board

d. Light Path status and error indicators require the Blade to be powered on

6. True/False: The UEFI is a functional replacement for legacy BIOS

7. True/False: To diagnose a Blade Server hardware problem, the first step to take would be to remove the Blade from the chassis and check the system board LEDs.

8. True/False: As a rule, power consumption is directly related to resultant heat output.

9. Which function should be used to view Service Processor configuration and hard disk drive health?

a. AMM Event Log

b. PreBoot DSA

c. AMM Monitor status page

245

Checkpoint solutions (2 of 2)

5. Select the correct statement regarding Blade Server status indicators.a. Memory and processor LEDs are on the Blade Server front panel

b. All Blade Server status LEDs are on the Light Path diagnostics panel

c. Blade Server status and error LEDs are on the Front Panel, Control Panel and adjacent to components on the system board

d. Light Path status and error indicators require the Blade to be powered on

Answer: c

6. True/False: The UEFI is a functional replacement for legacy BIOS

Answer: True

7. True/False: To diagnose a Blade Server hardware problem, the first step to take would be to remove the Blade from the chassis and check the system board LEDs.

Answer: False

8. True/False: As a rule, power consumption is directly related to resultant heat output.

Answer: True

8. Which function should be used to view Service Processor configuration and hard disk drive health?

a. AMM Event Log

b. PreBoot DSA

c. AMM Monitor status page

Answer: b

246

Page 42: IBM Bladecenter

42

Unit summary

Having completed this unit, you should be able to:

• Identify the BladeCenter components used to provide PD information

• List the planning elements required for the BladeCenter management network

• Select the functions available to modify firmware settings

• List the blade server indicators and Light Path Components

• Select the steps appropriate in diagnosing blade server hardware failures

• Identify the utility to use in displaying BladeCenter component health

247 5.3

Systems management with

Advanced Management Module

Unit objectives

After completing this unit, you should be able to:

• List the features of the BladeCenter Advanced Management Module

• Describe the role of the Advanced Management Module in BladeCenter chassis

and component management

• List the tasks available through the Advanced Management Module GUI Interface

• List the procedures for BladeCenter resource control and configuration provided

by the Advanced Management Module

• Identify and select the procedure to update the Advanced Management Module

firmware

249

Advanced Management Module

• Hot-swappable module

• Powerful and robust systems

management

• Proxy for expansion modules

• Controls all aspects of power,

connectivity, and communication

• Reliability, availability, and

serviceability (RAS)

Center point for IBM BladeCenter’ infrastructure intelligence:

250

BladeCenter management core

Power

Blower

Control Panel

CDROM

Floppy

Advanced

Management Module

Enet

Enet

Service Processor

CPU

Enet

VPD

LEDs

Voltage

Temperature

CPU

Interface

Flash Update

Processor

Blade

Configuration Watchdog Control

Chassis VPD Status Events/Alerts Configuration Control Indicators

Presence, Tachometer

Speed Control, Fault LED

Presence

VPD

Events/Alerts

Control

Configuration

Chassis

Ethernet Switch

Module

Blade

Control

Configuration

Events/Alerts

Inventory

VPD

Management

ServerIBM Director

Group/Association

Topology/Discover

Events/Alerts/Actions

Logs/Inventory

Deployment/Updates

251

Advanced Management Module: Block diagram

USB - K/M

Connector

PPC

440

GP

Memory

Nor

Flash RTC

NIC-1

NIC-2

100 MB

Switch

(mezzanine)

USBVIDEO/

FPGA

I2c

MM –LEDs

Flt,Actv,Pwr

ENET Jack

Video

Connector

RS485

M

I

D

P

L

A

N

E

USB 1.1/2.0 - Blade K/M

& Media

SM ENET Mgmt Ports

Video from Blade

RS485 to

Blade(s)

Serial Port

USB

uDOC

Nand Flash

Media Tray

Local MM i2c

CPLDChassis identification

and other functions…

OS Filesystem,

persistent storage

Debug Port

(hidden)

252

Page 43: IBM Bladecenter

43

USB virtualization for keyboard, mouse and media

USB virtualization in Management Module II(Switched Solution)

Management Module IIUSB

DVD Drive

USBKeyboard

USBMouse

USB Host Controller

USBDeviceChip

Bla

de

USB SwitchingMechanism

Remote PresenceSoftware

Bla

de

Bla

de

Bla

de

Bla

de

Bla

de

Bla

de

253

Key way

Advanced Management Module: Installation

Blade Chassis Rear Bay IDs

Hot-swappable management module

254

Advanced Management Modules: Connections and indicators

Video

Connector

Power-on

LEDS

Serial

Console

Connector

RJ45

10/100 Ethernet

Connector

RJ45

USB Dual

Stack

Pin-hole

Reset

Release

handle

MAC Address

Activity LEDSError

LEDS

Port Link LED

Port Activity LED

Advanced Management Module LEDS

255

Establishing a physical connection

To establish the physical connection to the Advanced Management Module

use one of the following methods:

1. Use a Category 5 UTP straight

through cable to connect the

Ethernet port on the management

module to a switch in a network that

has an networked management

station (PC or notebook).

2. Use a Category 5 cross-over

Ethernet cable to connect the

Advanced Management Module to a

standalone management station (PC

or notebook).

256

IBM BladeCenter management

257

Secure management network

Management Console

IBM Director / DHCP ServerConnection to Primary Advanced

Management Module in Bay 1.

Internal connection

Pri

vate

LA

N

An optional second Advanced Management Module in

Bay 2 connected to the “Private LAN” will offer

redundancy

258

Page 44: IBM Bladecenter

44

Advanced Management Module: Login

The initial user name and password for the management module are:

• User Name: USERID (all capital letters)

• Password: PASSW0RD (all capital letters and note the zero, not O, in

PASSW0RD)

259

Advanced Management Module: Menu

The menu appears in the left-hand

frame of your browser. The menu is

divided into five main sections for

easy navigation. These sections

are:

• Monitors

• Blade Tasks

• I/O Module Tasks

• MM Control

• Service Tools

260

Advanced Management Module: Monitors

• Monitors

• Blade Tasks

• I/O Module Tasks

• MM Control

• Service Tools

261

Monitors: System status - Blade servers

262

Monitors: System status - Blade server detail

263

Monitors: System status - Modules

3

4

1

2

264

Page 45: IBM Bladecenter

45

Monitors: System status - Other monitors

265

Monitors: Event Log

266

Monitors: LEDs

267

Monitors: Power use and planning

268

Monitors: Fuel gauge - Power usage graph

269

Monitors: Hardware VPD

270

Page 46: IBM Bladecenter

46

Monitors: Firmware VPD (1 of 2)

271

Monitors: Firmware VPD (2 of 2)

272

Monitors: Remote Chassis

DiscoverySelectio

n

Display

273

Advanced Management Module: Blade Tasks

• Monitors

• Blade Tasks

• I/O Module Tasks

• MM Control

• Service Tools

274

Blade Tasks: Power and restart

Power

Button

275

Blade Tasks: Remote Control

276

Page 47: IBM Bladecenter

47

Remote Control: Blade server console

Remote Control Task Bar

277

Blade Tasks: Update Blade Firmware

278

Blade Tasks: Configuration

279

Blade Tasks: Serial Over LAN (SOL)

280

Advanced Management Module: I/O Module Tasks

• Monitors

• Blade Tasks

• I/O Module Tasks

• MM Control

• Service Tools

281

I/O Module Tasks: Admin/Power/Restart

282

Page 48: IBM Bladecenter

48

I/O Module Tasks: Configuration

283

I/O Module Tasks: Advanced Configuration (1 of 3)

284

I/O Module: Advanced configuration (2 of 3)

285

I/O Module: Advanced configuration (3 of 3)

286

I/O Module: Firmware Update

287

Advanced Management Module: MM Control

• Monitors

• Blade Tasks

• I/O Module Tasks

• MM Control

• Service Tools

288

Page 49: IBM Bladecenter

49

MM Control: General settings

289

MM Control: Login Profiles (1 of 2)

290

MM Control: Login Profiles (2 of 2)

291

MM Control: Alerts notification

292

MM Control: Port Assignments

293

MM Control: Network interfaces

294

Page 50: IBM Bladecenter

50

MM Control: Network Protocols

295

MM Control: Security

296

MM Control: Firmware update

297

MM Control: Configuration Management

298

MM Control: Configuration file

299

MM Control: Configuration Wizard

300

Page 51: IBM Bladecenter

51

MM Control: Restart MM

301

Advanced Management Module: Service Tools

• Monitors

• Blade Tasks

• I/O Module Tasks

• MM Control

• Service Tools

302

Service Tools: Settings and Service Data

303

Service Tools: AMM Status

304

AMM Status: MM Firmware Update Status

305

AMM Status: MM Connectivity Status

306

Page 52: IBM Bladecenter

52

AMM Status: MM BIST Results

307

Service Tools: Service Advisor

308

AMM default IP addresses

• Management Module External Port

– Static: 192.168.70.125 (DHCP attempted first)

• Switch Module One

– Static: 192.168.70.127

• Switch Module Two

– Static: 192.168.70.128

• Switch Module Three

– Static: 192.168.70.129

• Switch Module Four

– Static: 192.168.70.130

309

Key words

• Advanced Management Module (AMM)

• Concurrent Keyboard, Video, and Mouse (cKVM)

• Keyboard, Video, Mouse (KVM)

• Vital Product Data (VPD)

• I2C (I squared C) – Inter-Integrated Circuit

• Light Weight Directory Access protocol (LDAP)

• RS485

• Reliability, Availability, and Serviceability (RAS)

• Secure Shell (SSH)

• Telnet

310

Checkpoint (1 of 2)

1. Select the correct statement describing the features provided by Advanced Management Module (AMM)

a. The AMM provides management only common chassis components, not Blade Servers

b. The AMM provides management for Blade Servers if additional software drivers are installed on the Blade Servers

c. The AMM provides management to common chassis components and Blade Servers and does not require additional software or drivers

d. Management through the AMM requires IBM Director

2. True/False: The IBM BladeCenter requires a standalone management server, shipped with each BladeCenter chassis.

3. True/False: The Advanced Management Module (AMM) requires one Blade-server dedicated to chassis management.

4. True/False: In order for the AMM to discover and identify the VPD for a Blade-server, the Blade-server does not have to be powered on.

5. Which statement is correct regarding the AMM?

a. Replacement of an AMM requires a power-down of the BladeCenter chassis

b. Each BladeCenter model implements a specific AMM for that model

c. Access to the AMM is local only, through the KVM connection on the module

d. The AMM provides both local and remote access to the BladeCenter, and is standardized across chassis, with functions specific where needed.

311

Checkpoint solutions (1 of 2)

1. Select the correct statement describing the features provided by Advanced Management Module (AMM)

a. The AMM provides management only common chassis components, not Blade Servers

b. The AMM provides management for Blade Servers if additional software drivers are installed on the Blade Servers

c. The AMM provides management to common chassis components and Blade Servers and does not require additional software or drivers

d. Management through the AMM requires IBM Director

Answer: c

2. True/False: The IBM BladeCenter requires a standalone management server, shipped with each BladeCenter chassis.

Answer: False

3. True/False: The Advanced Management Module (AMM) requires one Blade-server dedicated to chassis management.

Answer: False

4. True/False: In order for the AMM to discover and identify the VPD for a Blade-server, the Blade-server does not have to be powered on.

Answer: True

5. Which statement is correct regarding the AMM?

a. Replacement of an AMM requires a power-down of the BladeCenter chassis

b. Each BladeCenter model implements a specific AMM for that model

c. Access to the AMM is local only, through the KVM connection on the module

d. The AMM provides both local and remote access to the BladeCenter, and is standardized across chassis, with functions specific where needed.

Answer: d

312

Page 53: IBM Bladecenter

53

Checkpoint (2 of 2)

6. True/False: A configuration application is required at the workstation to configure the AMM.

7. Select the correct statement regarding GUI tasks available through the AMM

a. Configuration can be performed for external storage controllers

b. Blade Server boot sequence can be displayed but not changed

c. Tasks include: Monitors, Blade Tasks, I/O Module Tasks, MM Control and Service Tools

d. An external KVM must be attached to the AMM to view a Blade Server OS console

8. Which statement is correct regarding AMM firmware update?

a. All activity must be stopped prior to restarting (rebooting) the AMM

b. The AMM firmware can be loaded through the Web interface and a non-disruptive restart can be done

c. On BladeCenter models H and E, both primary and secondary AMMs must be updated separately

d. Installation of a new AMM in a chassis requires re-configuration of the AMM

313

Checkpoint solutions (2 of 2)

6. True/False: A configuration application is required at the workstation to configure the AMM.

Answer: False

7. Select the correct statement regarding GUI tasks available through the AMM

a. Configuration can be performed for external storage controllers

b. Blade Server boot sequence can be displayed but not changed

c. Tasks include: Monitors, Blade Tasks, I/O Module Tasks, MM Control and Service Tools

d. An external KVM must be attached to the AMM to view a Blade Server OS console

Answer: c

8. Which statement is correct regarding AMM firmware update?

a. All activity must be stopped prior to restarting (rebooting) the AMM

b. The AMM firmware can be loaded through the Web interface and a non-disruptive restart can be done

c. On BladeCenter models H and E, both primary and secondary AMMs must be updated separately

d. Installation of a new AMM in a chassis requires re-configuration of the AMM

Answer: b

314

Unit summary

Having completed this unit, you should be able to:

• List the features of the BladeCenter Advanced Management Module

• Describe the role of the Advanced Management Module in BladeCenter

chassis and component management

• List the tasks available through the Advanced Management Module

GUI Interface

• List the procedures for BladeCenter resource control and configuration

provided by the Advanced Management Module

• Identify and select the procedure to update the Advanced Management

Module firmware

315