Sun Cluster Overview for Solaris (821-0518)

8/6/2019 Sun Cluster Overview for Solaris (821-0518)

http://slidepdf.com/reader/full/sun-cluster-overview-for-solaris-821-0518 1/46

Sun Cluster Overview or SolarisOS

Sun Microsystems, Inc.4150 Network CircleSanta Clara, CA 95054U.S.A.

PartNo: 821–0518–10November 2009, Revision A



Copyright2009 SunMicrosystems, Inc. 4150 Network Circle, Santa Clara,CA 95054 U.S.A. Allrights reserved.

SunMicrosystems, Inc. hasintellectual property rightsrelatingto technology embodied in theproduct that is describedin this document.In particular, andwithoutlimitation, these intellectualproperty rightsmay include oneor more U.S. patents or pending patentapplications in theU.S. andin other countries.

U.S. Government Rights– Commercialsotware. Governmentusers are subject to the Sun Microsystems, Inc. standard license agreement and applicable provisionso theFARand itssupplements.

This distribution may include materials developed by thirdparties.Parts o theproduct maybe derived rom Berkeley BSDsystems, licensed rom theUniversity o Caliornia. UNIX is a registered trademarkin theU.S. andothercountries, exclusively licensed through X/OpenCompany, Ltd.

Sun, SunMicrosystems,the Sunlogo,the Solaris logo, theJavaCofeeCup logo, docs.sun.com,RSM, SunStorEdge, Java,and Solaris aretrademarks or registeredtrademarks o SunMicrosystems,Inc. or itssubsidiaries in theU.S. andothercountries. AllSPARCtrademarks areused under license andare trademarks orregistered trademarks o SPARC International, Inc. in the U.S. and othercountries. Products bearing SPARC trademarksare basedupon an architecture developedby Sun Microsystems, Inc.

TheOPEN LOOKandSunTM GraphicalUser Interacewas developedby SunMicrosystems, Inc. orits users andlicensees.Sun acknowledgesthe pioneering efortso Xeroxin researching anddeveloping theconcept o visualor graphicaluser interaces orthe computer industry.Sun holds a non-exclusive licenseromXeroxtotheXeroxGraphical User Interace, which license also coversSun'slicensees whoimplement OPEN LOOK GUIs andotherwise complywith Sun's written licenseagreements.

Products covered by andinormationcontained in this publication arecontrolled by U.S. ExportControl laws andmay be subject to theexport orimport laws inother countries. Nuclear,missile,chemicalor biological weapons or nuclear maritime enduses or endusers,whether director indirect,are strictly prohibited. Exportor reexport to countriessubject to U.S. embargo or to entities identiedon U.S. exportexclusion lists, including, butnot limited to,the deniedpersons andspecially designated nationals listsis strictly prohibited.

DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDINGANYIMPLIEDWARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPTTOTHEEXTENTTHAT SUCH DISCLAIMERS AREHELD TO BE LEGALLY INVALID.

Copyright2009 SunMicrosystems, Inc. 4150 Network Circle, Santa Clara,CA 95054 U.S.A. Tous droitsréservés.

SunMicrosystems, Inc. détient lesdroits de propriétéintellectuellerelatisà la technologie incorporée dans le produit quiest décritdansce document.En particulier,et ce sans limitation, cesdroits de propriétéintellectuellepeuvent inclure un ou plusieursbrevets américains ou desapplications de breveten attente auxEtats-Uniset dans d'autres pays.

Cette distribution peut comprendre des composantsdéveloppés par des tierces personnes.

Certaines composants de ce produit peuvent être dérivées du logiciel Berkeley BSD, licenciéspar l'Universitéde Caliornie. UNIXest unemarque déposée auxEtats-Uniset dans d'autres pays;elle est licenciéeexclusivement par X/Open Company, Ltd.

Sun, SunMicrosystems,le logo Sun, le logo Solaris, le logo Java Cofee Cup, docs.sun.com,RSM, SunStorEdge, Java et Solaris sont desmarques de abrique ou desmarques déposées de SunMicrosystems, Inc., ou sesliales,aux Etats-Unis et dans d'autres pays. Toutesles marques SPARCsont utiliséessous licence et sont desmarques de abrique ou desmarques déposées de SPARCInternational,Inc. auxEtats-Uniset dans d'autres pays. Les produits portant lesmarques SPARCsont baséssur une architecture développéepar Sun Microsystems, Inc.

L'interace d'utilisation graphiqueOPEN LOOK et Suna étédéveloppée parSun Microsystems, Inc. pour ses utilisateurset licenciés. Sunreconnaît leseforts depionniers de Xeroxpour la recherche et le développement du concept des interacesd'utilisation visuelle ou graphique pour l'industrie de l'inormatique.Sun détientunelicence nonexclusive de Xeroxsur l'interace d'utilisation graphiqueXerox, cette licence couvrant égalementles licenciésde Sunqui mettent en place l'interaced'utilisation graphiqueOPEN LOOK et qui, en outre,se conorment auxlicencesécrites de Sun.

Les produits quiont l'objet de cette publication et lesinormations qu'il contient sont régis parla legislation américaine en matière de contrôle desexportations etpeuvent être soumis au droit d'autres pays dans le domaine desexportations et importations. Lesutilisations nales, ou utilisateursnaux, pour desarmesnucléaires,des missiles, des armeschimiques ou biologiquesou pour le nucléairemaritime,directement ou indirectement, sont strictementinterdites. Les exportations ouréexportations vers despays sous embargo desEtats-Unis,ou vers desentités gurantsur leslistes d'exclusion d'exportation américaines,y compris, mais de manièrenonexclusive, la liste de personnesqui ont objet d'un ordre de ne pasparticiper,d'uneaçondirecte ou indirecte, auxexportations desproduitsou desservicesquisont régispar la legislation américaine en matière de contrôle des exportations et la listede ressortissants spéciquement designés, sont rigoureusement interdites.

LA DOCUMENTATION EST FOURNIE "EN L'ETAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITESSONT FORMELLEMENT EXCLUES,DANSLA MESUREAUTORISEE PAR LA LOIAPPLICABLE, Y COMPRIS NOTAMMENTTOUTEGARANTIE

IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L'APTITUDE A UNE UTILISATION PARTICULIERE OU A L'ABSENCE DE CONTREFACON.

091123@23031



Contents

Preace .....................................................................................................................................................5

1 Introduction to SunCluster ................................................................................................................. 9

Making Applications Highly Available With Sun Cluster .................................................................9

Availability Management ............................................................................................................ 10

Failover and Scalable Services and Parallel Applications ........................................................ 10

IP Network Multipathing ............................................................................................................ 11

Storage Management ................................................................................................................... 11

Campus Clusters .......................................................................................................................... 13

Monitoring Failure .............................................................................................................................. 13

Administration and Conguration Tools ........................................................................................ 14

Sun Cluster Manager ................................................................................................................... 14

Command-Line Interace ........................................................................................................... 15

Sun Management Center ............................................................................................................ 15

Role-Based Access Control ......................................................................................................... 15

2 Key ConceptsorSunCluster ............................................................................................................. 17

Clusters, Nodes, and Hosts ................................................................................................................. 17

Zone Cluster ......................................................................................................................................... 19

Features and Benets o a Zone Cluster .................................................................................... 19

Cluster Interconnect ........................................................................................................................... 20

Cluster Membership ........................................................................................................................... 20

Cluster Conguration Repository ..................................................................................................... 21

Quorum Devices .................................................................................................................................. 21

Fault Monitors ..................................................................................................................................... 22

Data Services Monitoring ........................................................................................................... 22Disk-Path Monitoring ................................................................................................................. 22

3



IP Multipath Monitoring ............................................................................................................ 22

Quorum Device Monitoring ....................................................................................................... 23

Data Integrity ....................................................................................................................................... 23Split Brain and Amnesia .............................................................................................................. 23

Fencing ................................................................................................................................................. 24

Failast ................................................................................................................................................... 24

Shared Devices, Local Devices, and Device Groups ........................................................................ 25

Shared Devices ............................................................................................................................. 25

Local Devices ................................................................................................................................ 26

Device Groups .............................................................................................................................. 26

Data Services ........................................................................................................................................ 27

Description o a Resource Type .................................................................................................. 27

Description o a Resource ........................................................................................................... 27

Description o a Resource Group ............................................................................................... 28

Data Service Types ....................................................................................................................... 28

System Resource Usage ....................................................................................................................... 29

System Resource Monitoring ..................................................................................................... 30

CPU Control ................................................................................................................................. 30

Visualization o System Resource Usage ................................................................................... 31

3 SunCluster Architecture ....................................................................................................................33

Sun Cluster Hardware Environment ................................................................................................ 33

Sun Cluster Sotware Environment ................................................................................................... 34

Cluster Membership Monitor .................................................................................................... 35

Cluster Conguration Repository (CCR) ................................................................................. 36

Cluster File Systems ..................................................................................................................... 36

Scalable Data Services ......................................................................................................................... 37

Load-Balancing Policies .............................................................................................................. 38Multihost Disk Storage ....................................................................................................................... 39

Cluster-Interconnect Components ................................................................................................... 39

IP Network Multipathing Groups ..................................................................................................... 40

Public Network Interaces ........................................................................................................... 41

Index ......................................................................................................................................................43

Contents

SunClusterOverviewor Solaris OS • November 2009, RevisionA4



Preace

SunTM Cluster Overview orSolaris OS introduces the Sun Cluster product by explaining the

purpose o the product and how Sun Cluster achieves this purpose. This book also explains key concepts or Sun Cluster. The inormation in this document enables you to become amiliarwith Sun Cluster eatures and unctionality.

Related Documentation

Inormation about related Sun Cluster topics is available in the documentation that is listed inthe ollowing table. All Sun Cluster documentation is available at http://docs.sun.com.

Topic Documentation

Overview SunClusterOverview orSolaris OS

SunCluster 3.2 11/09DocumentationCenter

Concepts SunClusterConceptsGuide orSolaris OS

Hardware installation and

administration

SunCluster 3.1- 3.2HardwareAdministrationManual orSolaris OS

Individual hardware administration guides

Sotware installation SunCluster Sotware Installation Guideor Solaris OS

SunClusterQuick StartGuide orSolaris OS

Data service installation and

administration

SunClusterData Services Planning andAdministrationGuideor Solaris

OS

Individual data service guides

Data service development SunClusterData ServicesDeveloper’s Guideor Solaris OS

System administration SunCluster SystemAdministrationGuideor Solaris OS

SunClusterQuick Reerence

Sotware upgrade SunClusterUpgradeGuide orSolaris OS

Error messages SunClusterErrorMessagesGuide orSolaris OS

5

http://docs.sun.com/

http://docs.sun.com/doc/821-0518


























http://docs.sun.com/





Command Function

prtconf -v Displays the size o the system memory and reports inormation about peripheral

devices

psrinfo -v Displays inormation about processors

showrev -p Reports which patches are installed

prtdiag -v Displays system diagnostic inormation

scinstall -pv Displays Sun Cluster sotware release and package version inormation

scstat Provides a snapshot o the cluster status

scconf -p Lists cluster conguration inormation

scrgadm -p Displays inormation about installed resources, resource groups, and resource types

Also have available the contents o the /var/adm/messages le.

Typographic Conventions

The ollowing table describes the typographic conventions that are used in this book.

TABLE P–1 TypographicConventions

Typeface Meaning Example

AaBbCc123 The names o commands, les, and directories,and onscreen computer output

Edityour .login le.

Use ls -a to list all les.

machine_name% you have mail.

AaBbCc123 What you type, contrasted with onscreen

computer output

machine_name% su

Password:

aabbcc123 Placeholder:replacewith a realname orvalue The command toremove a le is rm

flename.

AaBbCc123 Book titles, new terms, and terms to be

emphasized

Read Chapter 6 in the User's Guide.

A cache isa copy thatis stored

locally.

Do not save the le.

Note: Some emphasized items

appear bold online.

Preace

7



Shell Prompts in Command ExamplesThe ollowing table shows the deault UNIX® system prompt and superuser prompt or the Cshell, Bourne shell, and Korn shell.

TABLE P–2 Shell Prompts

Shell Prompt

C shell machine_name%

C shell or superuser machine_name#

Bourne shell and Korn shell $

Bourne shell and Korn shell or superuser #

Preace




Introduction to Sun Cluster

A Sun Cluster conguration is an integrated hardware and Sun Cluster sotware solution that is

used to create highly available and scalable services. This chapter provides a high-level overview

o Sun Cluster eatures.

This chapter contains the ollowing sections:

■ “Making Applications Highly Available With Sun Cluster” on page 9■ “Monitoring Failure” on page 13■ “Administration and Conguration Tools” on page 14

Making Applications Highly Available With Sun Cluster

A cluster is a collection o loosely coupled computing nodes that provides a single client view o

network services or applications, including databases, web services, and le services.

In a clustered environment, the nodes are connected by an interconnect and work together as a

single entity to provide increased availability and perormance.

Highly available clusters provide nearly continuous access to data and applications by keeping

the cluster running through ailures that would normally bring down a single server system. Nosingle ailure, hardware, sotware, or network, can cause a cluster to ail. By contrast,

ault-tolerant hardware systems provide constant access to data and applications, but at a

higher cost because o specialized hardware. Fault-tolerant systems usually have no provision

or sotware ailures.

1C H A P T E R 1

9



Each Sun Cluster system is a collection o tightly coupled nodes that provide a singleadministration view o network services and applications. The Sun Cluster system achieves highavailability through a combination o the ollowing hardware and sotware:

■ Redundant disk systems provide storage. These disk systems are generally mirrored topermit uninterrupted operation i a disk or subsystem ails. Redundant connections to thedisk systems ensures that data is not isolated i a server, controller, or cable ails. Ahigh-speed interconnect among Solaris hosts provides access to resources. All hosts in thecluster are also connected to a public network, enabling clients on multiple networks toaccess the cluster.

■ Redundant hot-swappable components, such as power supplies and cooling systems,

improve availability by enabling systems to continue operation ater a hardware ailure.Hot-swappable components provide the ability to add or remove hardware components in aunctioning system without bringing it down.

■ Sun Cluster sotware's high-availability ramework detects a node ailure quickly andmigrates the application or service to another node that runs in an identical environment.At no time are all applications unavailable. Applications unafected by a down node are ully available during recovery. Furthermore, applications o the ailed node become available assoon as they are recovered. A recovered application does not have to wait or all otherapplications to complete their recovery.

Availability Management

An application is highly available i it survives any single sotware or hardware ailure in thesystem. Failures that are caused by bugs or data corruption within the application itsel areexcluded. The ollowing apply to highly available applications:

■ Recovery is transparent rom the applications that use a resource.

■ Resource access is ully preserved across node ailure.

■ Applications cannot detect that the hosting node has been moved to another node.

■ Failure o a single node is completely transparent to programs on remaining nodes that usethe les, devices, and disk volumes that are attached to this node.

Failover and Scalable Services and ParallelApplications

Failover and scalable services and parallel applications enable you to make your applicationshighly available and to improve an application's perormance on a cluster.

A ailover service provides high availability through redundancy. When a ailure occurs, you

can congure an application that is running to either restart on the same node, or be moved toanother node in the cluster, without user intervention.

MakingApplicationsHighly Available With Sun Cluster




To increase perormance, a scalable service leverages the multiple nodes in a cluster to

concurrently run an application. In a scalable conguration, each node in the cluster can

provide data and process client requests.

Parallel databases enable multiple instances o the database server to do the ollowing:

■ Participate in the cluster■ Handle diferent queries on the same database simultaneously ■ Provide parallel query capability on large queries

For more inormation about ailover and scalable services and parallel applications, see “Data

Service Types” on page 28.

IP Network Multipathing

Clients make data requests to the cluster through the public network. Each Solaris host is

connected to at least one public network through one or multiple public network adapters.

IP network multipathing enables a server to have multiple network ports connected to the same

subnet. First, IP network multipathing sotware provides resilience rom network adapter

ailure by detecting the ailure or repair o a network adapter. The sotware then simultaneously

switches the network address to and rom the alternative adapter. When more than one

network adapter is unctional, IP network multipathing increases data throughput by spreading

outbound packets across adapters.

Storage Management

Multihost storage makes disks highly available by connecting the disks to multiple Solaris hosts.

Multiple hosts enable multiple paths to access the data. I one path ails, another one is available

to take its place.

Multihost disks enable the ollowing cluster processes:■ Tolerating single-host ailures.

■ Centralizing application data, application binaries, and conguration les.

■ Protecting against host ailures. I client requests are accessing the data through a host that

ails, the requests are switched over to use another host that has a direct connection to the

same disks.

■ Providing access either globally through a primary host that “masters” the disks, or by direct

concurrent access through local paths.


Chapter 1 • Introduction to Sun Cluster 11



Volume Management Support

A volume manager enables you to manage large numbers o disks and the data on those disks.

Volume managers can increase storage capacity and data availability by ofering the ollowingeatures:

■ Disk-drive striping and concatenation■ Disk-mirroring■ Disk-drive hot spares■ Disk-ailure handling and disk replacements

Sun Cluster systems support the ollowing volume managers:

■ Solaris Volume Manager■ Multi-owner Solaris Volume Manager or Sun Cluster■ Veritas Volume Manager

Solaris I/O Multipathing (MPxIO)

Solaris I/O multipathing (MPxIO), which was ormerly named Sun StorEdge Trac Manager,is ully integrated in the Solaris Operating System I/O ramework. Solaris I/O multipathing

enables you to represent and manage devices that are accessible through multiple I/O controllerinteraces within a single instance o the Solaris operating system.

The Solaris I/O multipathing architecture provides the ollowing eatures:

■ Protection against I/O outages due to I/O controller ailures■ Automatic switches to an alternate controller upon an I/O controller ailure■ Increased I/O perormance by load balancing across multiple I/O channels

Hardware Redundant Array o Independent Disks Support

Sun Cluster systems support the use o hardware Redundant Array o Independent Disks(RAID) and host-based sotware RAID. Hardware RAID uses the storage array's or storagesystem's hardware redundancy to ensure that independent hardware ailures do not impactdata availability. I you mirror across separate storage arrays, host-based sotware RAID ensuresthat independent hardware ailures do not impact data availability when an entire storage array is oine. Although you can use hardware RAID and host-based sotware RAID concurrently,

you need only one RAID solution to maintain a high degree o data availability.

Cluster File System Support

Because one o the inherent properties o clustered systems is shared resources, a clusterrequires a le system that addresses the need or les to be shared coherently. In a Sun Clusterle system, a cluster fle systemenables users or applications to access any le on any node o thecluster by using remote or local standard UNIX APIs.

Sun Cluster systems support the ollowing cluster le systems:





■ UNIX® File System (UFS) – Uses Sun Cluster Proxy System (PxFS)■ Veritas File System (VxFS) – Uses PxFS

Sun Cluster sotware supports the ollowing as highly available ailover local le systems:■ UFS■ Solaris ZFSTM

■ Sun QFS■ VxFS

I an application is moved rom one node to another node, no change is required or theapplication to access the same les. No changes need to be made to existing applications to ully

utilize the cluster le system.

Campus Clusters

Standard Sun Cluster systems provide high availability and reliability rom a single location. I your application must remain available ater unpredictable disasters such as an earthquake,

ood, or power outage, you can congure your cluster as a campus cluster.

Campus clusters enable you to locate cluster components, such as Solaris hosts and sharedstorage, in separate rooms that are several kilometers apart. You can separate your hosts andshared storage and locate them in diferent acilities around your corporate campus orelsewhere within several kilometers. When a disaster strikes one location, the surviving hostscan take over service or the ailed host. This enables applications and data to remain availableor your users. For additional inormation about campus cluster congurations, see theSun

Cluster 3.1 - 3.2HardwareAdministrationManual or Solaris OS.

Monitoring FailureThe Sun Cluster system makes the path between users and data highly available by usingmultihost disks, multipathing, and a cluster le system. The Sun Cluster system monitors

ailures or the ollowing:■ Applications – Most o the Sun Cluster data services supply a ault monitor that periodically

probes the data service to determine its health. A ault monitor veries that the applicationdaemon or daemons are running and that clients are being served. Based on the inormationthat is returned by probes, a predened action such as restarting daemons or causing aailover can be initiated.

■ Disk Paths – Sun Cluster sotware supports disk-path monitoring (DPM). DPM improvesthe overall reliability o ailover and switchover by reporting the ailure o a secondary disk path.

Monitoring Failure








■ Internet Protocol (IP) Multipath – Solaris IP network multipathing sotware on Sun Cluster

systems provide the basic mechanism or monitoring public network adapters. IP

multipathing also enables ailover o IP addresses rom one adapter to another adapter when

a ault is detected.

■ Quorum Devices - Sun Cluster sotware supports quorum device monitoring by

periodically testing that quorum works on quorum devices. When Sun Cluster sotware

detects a ailure, the Sun Cluster system reports the ailure and marks the quorum device

that is not working correctly. When the Sun Cluster system detects that a previously ailed

quorum device now operates correctly, the system automatically brings the quorum device

back into service. Bringing the quorum device back into service includes placing the correct

quorum reservation inormation on the device. The Sun Cluster system automatically monitors any congured quorum device that is not in maintenance mode, regardless o

type.

Administration and Confguration Tools

You can install, congure, and administer the Sun Cluster system either though the Sun ClusterManager GUI or through the command-line interace (CLI).

The Sun Cluster system also has a module that runs as part o Sun Management Center sotware

that provides a GUI to certain cluster tasks.

Sun Cluster ManagerSun Cluster Manager is a browser-based tool or administering Sun Cluster systems. The Sun

Cluster Manager sotware enables administrators to perorm system management and

monitoring, sotware installation, and system conguration.

The Sun Cluster Manager sotware includes the ollowing eatures:

■ Built-in security and authorization mechanisms

■ Secure Sockets Layer (SSL) support

■ Role-based access control (RBAC)

■ Pluggable Authentication Module (PAM)

■ NAFO and IP network multipathing group administration acilities

■ Quorum devices, transports, shared storage device, and resource group administration

■

Sophisticated error checking and autodetection o private interconnects

Administration and ConfgurationTools




Command-Line InteraceThe Sun Cluster command-line interace (CLI) is a set o utilities you can use to install and

administer Sun Cluster systems, and administer the volume manager portion o Sun Clustersotware.

You can perorm the ollowing Sun Cluster administration tasks through the Sun Cluster CLI:

■ Validating a Sun Cluster conguration

■ Installing and conguring Sun Cluster sotware

■ Updating a Sun Cluster conguration

■ Managing the registration o resource types, the creation o resource groups, and theactivation o resources within a resource group

■ Changing node mastery and states or resource groups and device groups

■ Controlling access with role-based access control (RBAC)

■ Shutting down the entire cluster

Sun Management CenterThe Sun Cluster system also has a module that runs as part o Sun Management Centersotware. Sun Management Center sotware serves as the cluster's base or administrative andmonitoring operations and enables system administrators to perorm the ollowing tasksthrough a GUI or CLI:

■ Conguring a remote system

■ Monitoring perormance■ Detecting and isolating hardware and sotware aults

Sun Management Center sotware can also be used as the interace to manage dynamicreconguration within Sun Cluster servers. Dynamic reconguration includes domaincreation, dynamic board attach, and dynamic detach.

Role-Based Access ControlIn conventional UNIX systems, the root user, also reerred to as superuser, is omnipotent, withthe ability to read and write to any le, run all programs, and send kill signals to any process.Solaris role-based access control (RBAC) is an alternative to the all-or-nothing superusermodel. RBAC uses the security principle o least privilege, which is that no user should be givenmore privilege than necessary or perorming his or her job.

RBAC enables an organization to separate superuser capabilities and package them into specialuser accounts or roles or assignment to specic individuals. This separation and packaging





enables a variety o security policies. Accounts can be set up or special-purpose administratorsin such areas as security, networking, rewall, backups, and system operation.





Key Concepts or Sun Cluster

This chapter explains the key concepts related to the hardware and sotware components o theSun Cluster system that you need to understand beore working with Sun Cluster systems.


■

“Clusters, Nodes, and Hosts” on page 17■ “Zone Cluster” on page 19■ “Cluster Interconnect” on page 20■ “Cluster Membership” on page 20■ “Cluster Conguration Repository” on page 21■ “Quorum Devices” on page 21■ “Fault Monitors” on page 22■ “Data Integrity” on page 23■

“Fencing” on page 24■ “Failast” on page 24■ “Shared Devices, Local Devices, and Device Groups” on page 25■ “Data Services” on page 27■ “System Resource Usage” on page 29

Clusters, Nodes, and HostsA cluster is a collection o one or more nodes that belong exclusively to that collection. In acluster that runs on the Solaris 10 OS, a global cluster anda zone cluster are types o clusters. In acluster that runs on any version o the Solaris OS that was released beore the Solaris 10 OS, anodeis a physicalmachine that contributes to cluster membership and is not a quorum device.Ina cluster that runs on the Solaris 10 OS, the concept o a node changes. A node is a Solariszone that is associated with a cluster. In this environment, a Solaris host , or simply host ,isoneo

the ollowing hardware or sotware congurations that runs the Solaris OS and its ownprocesses:

2C H A P T E R 2

17



■ A “bare metal” physical machine that is not congured with a virtual machine or as ahardware domain

■ A Sun Logical Domains (LDoms) guest domain

■ A Sun Logical Domains (LDoms) I/O domain

■ A hardware domain

In a Solaris 10 environment, a voting node is a zone that contributes votes to the total number o quorum votes, that is, membership votes in a cluster. This total determines whether the clusterhas sucient votes to continue operating. A non-votingnode is a zone that doesnot contributeto the total number o quorum votes, that is, membership votes in a cluster.

In a clustered environment, the nodes are connected by an interconnect and work together as asingle entity to provide increased availability and perormance.

In a Solaris 10 environment, a global cluster is a type o cluster that is composed only o one ormore global-cluster voting nodes and optionally, zero or more global-cluster non-voting nodes.

Note – A global cluster can optionally also include solaris8, solaris9, lx (Linux), or native

brand, non-global zones that are not nodes, but high availability containers (as resources).

A global-cluster voting node is a native brand, global zone in a global cluster that contributes votes to the total number o quorum votes, that is, membership votes in the cluster. This totaldetermines whether the cluster has sucient votes to continue operating. A global-clusternon-voting node is a native brand, non-global zone in a global cluster that does not contribute votes to the total number o quorum votes, that is, membership votes in the cluster.

In a Solaris 10 environment, a zone cluster is a type o cluster that is composed only o one ormore cluster brand, voting nodes. A zone cluster depends on, and thereore requires, a globalcluster. A global cluster does not contain a zone cluster. You cannot congure a zone clusterwithout a global cluster. A zone cluster has, at most, one zone cluster node on a machine.

Note – A zone-cluster node continues to operate only as long as the global-cluster voting nodeon the same machine continues to operate. I a global-cluster voting node on a machine ails, all

zone-cluster nodes on that machine ail as well.

The Sun Cluster sotware enables you to have one to sixteen Solaris hosts in a cluster, dependingon the hardware conguration. Contact your Sun representative or inormation about thenumber o Solaris hosts that are supported on your particular hardware conguration.

Solaris hosts in a cluster are generally attached to one or more disks. Solaris hosts that are notattached to disks use the cluster le system to access the multihost disks. Solaris hosts in paralleldatabase congurations share concurrent access to some or all disks.

Clusters,Nodes,and Hosts




Every node in the cluster is aware when another node joins or leaves the cluster. Also, every

node in the cluster is aware o the resources that are running locally as well as the resources that

are running on the other cluster nodes.

Solaris hosts in the same cluster should have similar processing, memory, and I/O capability to

enable ailover to occur without signicant degradation in perormance. Because o the

possibility o ailover, each host should have sucient capacity to meet service level agreements

i a node ails.

Zone ClusterThis section describes the primary eatures and benets o a zone cluster.

Features and Benefts o a Zone Cluster

A zone cluster provides the ollowing eatures and benets.

■ Application ault isolation – A ailure o applications on one zone cluster does not afect

applications on other zone clusters. For example, i a zone cluster node starts, halts, or

reboots, nodes in other zone clusters are not afected.

■ Security – Applications that are, or a person who is, logged into a zone cluster node cannot

see or modiy elements in the global cluster or in other zone clusters. A zone cluster only

contains those elements, such as le systems, ZFS datasets, or network resources that are

explicitly congured as part o that zone cluster. A ailover application in a zone cluster canail over or switch over only rom one node in a zone cluster to another node in the same

zone cluster. All instances o a scalable application run only in the same zone cluster. The

zone cluster is a security container that applications cannot escape.

■ Resource management – You can apply the ull range o Solaris resource management

controls to a zone cluster. Consequently, you can control all applications on a node in a zone

cluster at the zone level. This control enables you to better manage the resources that are

available to a zone cluster node. For example, this control enables you to place anapplication in a zone cluster and reduce the number o CPUs. You can thus reduce your

per-CPU license ee.

■ Delegated administration – You can delegate the ability to manage applications in a zone

cluster to an administrator who is operating in that zone cluster. A zone cluster unctions

independently o the global cluster and other zone clusters. As the global zone

administrator, you can set up cross-cluster dependencies and anities and administer

applications in the zone cluster.

ZoneCluster

Chapter 2 • Key Concepts or Sun Cluster 19



■ Simplied cluster – All you need to do in a zone cluster is administer the applications andresources that are used by those applications. As the global-zone administrator, you cancreate, manage, and remove a zone cluster at any time by issuing a command both inside

and outside that zone cluster. You can do so without afecting the global cluster or otherzone clusters.

Cluster InterconnectThe cluster interconnect is the physical conguration o devices that are used to transercluster-private communications and data service communications between Solaris hosts in the

cluster.

Redundant interconnects enable operations to continue over the surviving interconnects whilesystem administrators isolate ailures and repair communication. The Sun Cluster sotwaredetects, repairs, and automatically re-initiates communication over a repaired interconnect.

For more inormation, see “Cluster-Interconnect Components” on page 39.

Cluster MembershipThe Cluster Membership Monitor (CMM) is a distributed set o agents that exchange messagesover the cluster interconnect to complete the ollowing tasks:

■ Enorcing a consistent membership view on all nodes (quorum)

■ Driving synchronized reconguration in response to membership changes

■

Handling cluster partitioning■ Ensuring ull connectivity among all cluster members by leaving unhealthy nodes out o the

cluster until they are repaired

The main unction o the CMM is to establish cluster membership, which requires acluster-wide agreement on the set o nodes that participate in the cluster at any time. The CMMdetects major cluster status changes on each node, such as loss o communication between oneor more nodes. The CMM relies on the transport kernel module to generate heartbeats across

the transport medium to other nodes in the cluster. When the CMM does not detect a heartbeatrom a node within a dened timeout period, the CMM considers the node to have ailed andthe CMM initiates a cluster reconguration to renegotiate cluster membership.

To determine cluster membership and to ensure data integrity, the CMM perorms theollowing tasks:

■ Accounting or a change in cluster membership, such as a node's joining or leaving thecluster

■ Ensuring that an unhealthy node leaves the cluster

Cluster Interconnect


Q D i



■ Ensuring that an unhealthy node remains inactive until it is repaired

■ Preventing the cluster rom partitioning itsel into subsets o nodes

See “Split Brain and Amnesia” on page 23 or more inormation about how the cluster protectsitsel rom partitioning into multiple separate clusters.

Cluster Confguration RepositoryThe Cluster Conguration Repository (CCR) is a private, cluster-wide, distributed database orstoring inormation that pertains to the conguration and state o the cluster. To avoid

corrupting conguration data, each node must be aware o the current state o the clusterresources. The CCR ensures that all nodes have a consistent view o the cluster. The CCR isupdated when error or recovery situations occur or when the general status o the clusterchanges.

The CCR structures contain the ollowing types o inormation:

■ Cluster and node names■ Cluster transport conguration■ The names o Solaris Volume Manager disk sets or Veritas disk groups■ A list o nodes that can master each disk group■ Operational parameter values or data services■ Paths to data service callback methods■ DID device conguration■ Current cluster status

Quorum DevicesA quorum device is a shared storage device or quorum server that is shared by two or morenodes and that contributes votes that are used to establish a quorum. The cluster can operateonly when a quorum o votes is available. The quorum device is used when a cluster becomespartitioned into separate sets o nodes to establish which set o nodes constitutes the newcluster.

Both cluster nodes and quorum devices vote to orm quorum. By deault, cluster nodes acquirea quorum vote count o one when they boot and become cluster members. Nodes can have a vote count o zero when the node is being installed, or when an administrator has placed a nodeinto the maintenance state.

Quorum devices acquire quorum vote counts that are based on the number o nodeconnections to the device. When you set up a quorum device, it acquires a maximum vote counto N -1 where N is the number o connected votes to the quorum device. For example, a quorumdevice that is connected to two nodes with nonzero vote counts has a quorum count o one (twominus one).

Quorum Devices


FaultMonitors



Fault MonitorsSun Cluster system makes all components on the ”path” between users and data highly available

by monitoring the applications themselves, the le system, and network interaces.

The Sun Cluster sotware detects a node ailure quickly and creates an equivalent server or theresources on the ailed node. The Sun Cluster sotware ensures that resources unafected by theailed node are constantly available during the recovery and that resources o the ailed nodebecome available as soon as they are recovered.

Data Services MonitoringEach Sun Cluster data service supplies a ault monitor that periodically probes the data serviceto determine its health. A ault monitor veries that the application daemon or daemons arerunning and that clients are being served. Based on the inormation returned by probes,predened actions such as restarting daemons or causing a ailover, can be initiated.

Disk-Path MonitoringSun Cluster sotware supports disk-path monitoring (DPM). DPM improves the overallreliability o ailover and switchover by reporting the ailure o a secondary disk-path. You canuse one o two methods or monitoring disk paths. The rst method is provided by thecldevice command. This command enables you to monitor, unmonitor, or display the statuso disk paths in your cluster. See the cldevice(1CL) man page or more inormation aboutcommand-line options.

The second method or monitoring disk paths in your cluster is provided by the Sun ClusterManager graphical user interace (GUI). Sun Cluster Manager provides a topological view o the monitored disk paths. The view is updated every 10 minutes to provide inormation aboutthe number o ailed pings.

IP Multipath Monitoring

Each Solaris host in a cluster has its own IP network multipathing conguration, which candifer rom the conguration on other hosts in the cluster. IP network multipathing monitorsthe ollowing network communication ailures:

■ The transmit and receive path o the network adapter has stopped transmitting packets.

■ The attachment o the network adapter to the link is down.

■ The port on the switch does not transmit or receive packets.

■ The physical interace in a group is not present at system boot.

FaultMonitors


DataIntegrity

http://docs.sun.com/doc/820-7361/cldevice-1cl?a=view





Quorum Device MonitoringSun Cluster sotware supports the monitoring o quorum devices. Periodically, each node in the

cluster tests the ability o the local node to work correctly with each congured quorum devicethat has a congured path to the local node and is not in maintenance mode. This test consistso an attempt to read the quorum keys on the quorum device.

When the Sun Cluster system discovers that a ormerly healthy quorum device has ailed, thesystem automatically marks the quorum device as unhealthy. When the Sun Cluster systemdiscovers that a ormerly unhealthy quorum device is now healthy, the system marks thequorum device as healthy and places the appropriate quorum inormation on the quorum

device.The Sun Cluster system generates reports when the health status o a quorum device changes.When nodes recongure, an unhealthy quorum device cannot contribute votes to membership.Consequently, the cluster might not continue to operate.

Data IntegrityThe Sun Cluster system attempts to prevent data corruption and ensure data integrity. Becausecluster nodes share data and resources, a cluster must never split into separate partitions thatare active at the same time. The CMM guarantees that only one cluster is operational at any time.

Split Brain and Amnesia

Two types o problems can arise rom cluster partitions: split brain and amnesia. Split brainoccurs when the cluster interconnect between Solaris hosts is lost and the cluster becomespartitioned into subclusters, and each subcluster believes that it is the only partition. Asubcluster that is not aware o the other subclusters could cause a conict in shared resources,such as duplicate network addresses and data corruption.

Amnesia occurs i all the nodes leave the cluster in staggered groups. An example is a two-nodecluster with nodes A and B. I node A goes down, the conguration data in the CCR is updated

on node B only, and not node A. I node B goes down at a later time, and i node A is rebooted,node A will be running with old contents o the CCR. This state is called amnesia and mightlead to running a cluster with stale conguration inormation.

You can avoid split brain and amnesia by giving each node one vote and mandating a majority o votes or an operational cluster. A partition with the majority o votes has a quorum and isenabled to operate. This majority vote mechanism works well i more than two nodes are in thecluster. In a two-node cluster, a majority is two. I such a cluster becomes partitioned, anexternal vote enables a partition to gain quorum. This external vote is provided by a quorumdevice. A quorum device can be any disk that is shared between the two nodes.

DataIntegrity


Fencing



Table 2–1 describes how Sun Cluster sotware uses quorum to avoid split brain and amnesia.

TABLE 2–1 Cluster Quorum, and Split Brain and Amnesia Problems

Par tition Type Quorum Solution

Split brain Enables onlythe partition (subcluster) with a majority ovotes torun asthe cluster

(only one partition can exist with such a majority).Ater a node loses the race or

quorum, that node panics.

Amnesia Guarantees that whena cluster isbooted, ithas atleast one nodethatwas a membero

the most recent cluster membership (and thus has the latest conguration data).

FencingWhen split brain occurs, not all nodes can communicate, so individual nodes or subsets o nodes might try to orm individual or subset clusters. Each subset or partition might “believe” ithas sole access and ownership to the multihost disks. Attempts by multiple nodes to write to thedisks can result in data corruption.

I a node loses connectivity with other nodes, the node attempts to orm a cluster with the nodeswith which communication is possible. I that set o nodes does not orm a quorum, Sun Clustersotware halts the node and “ences” the node rom the disks. Thus, Sun Cluster sotwareprevents the node rom accessing the disks. Only current member nodes have access to thedisks, ensuring data integrity.

You can turn of encing or selected disks or or all disks.

Caution – I you turn of encing under the wrong circumstances, your data can be vulnerable tocorruption during application ailover. Examine this data corruption possibility careully whenyou are considering turning of encing. I your shared storage device does not support the SCSIprotocol, such as a Serial Advanced Technology Attachment (SATA) disk, or i you want toallow access to the cluster's storage rom hosts outside the cluster, turn of encing.

Failast The purpose o ailast is to halt a component that is not healthy enough to continue correctoperations. Sun Cluster sotware includes many ailast mechanisms that detect diferentunhealthy conditions.

I the Sun Cluster system detects a critical ailure on the global-cluster voting node, the systemorcibly shuts down the Solaris host.

When the Sun Cluster system detects a critical ailure on any other type o node, or example, a

global-cluster non-voting node or zone-cluster node, the system reboots that node.

g


Shared Devices,Local Devices, and Device Groups



Sun Cluster sotware monitors the nodes that belong to the cluster. Communication or nodeailures can change the number o nodes in a cluster. I the cluster does not maintain sucient votes, Sun Cluster sotware halts that set o nodes.

Sun Cluster sotware maintains a number o critical cluster-specic daemons. Some daemonssupport the global-cluster voting node, while others support other types o nodes. A daemon iscritical to the node that the daemon supports, which might difer rom the node on which thedaemon runs. For example, some daemons in the global zone support a non-global zone. Forthis reason, these daemons are critical to the health o the non-global zone rather than theglobal zone.

Shared Devices, Local Devices, and Device GroupsThe cluster le system makes all les across a cluster equally accessible and visible to all nodes.Similarly, Sun Cluster sotware makes all devices on a cluster accessible and visible throughoutthe cluster. That is, the I/O subsystem enables access to any device in the cluster, rom any node,without regard to where the device is physically attached. This access is reerred to as shared

device access.

Shared Devices

Sun Cluster systems use shared devices to provide cluster-wide, highly available access to any device in a cluster, rom any node.

How Sun Cluster Uses Shared Devices

Generally, i a node ails while providing access to a shared device, the Sun Cluster sotwareswitches over to another path to the device and redirects the access to that path. This redirectionis easy with shared devices because the same name is used or the device regardless o the path.Access to a remote device is perormed in the same way as on a local device that uses the samename. Also, the API to access a shared deviceon a cluster is the same as the API that is used to

access a device locally.

Sun Cluster shared devices include disks, CD-ROMs, and tapes. However, disks are the only multiported shared devices that are supported. This limited support means that CD-ROM andtape devices are not currently highly available devices. The local disks on each server are alsonot multiported, and thus are not highly available devices.

The DID ramework assigns a common (normalized) logical name to each disk, CD-ROM, andtape device in the cluster. This assignment enables consistent access to each device rom any

node in the cluster.


Shared Devices,Local Devices, and Device Groups



Device ID

The Sun Cluster sotware manages shared devices through a construct that is known as thedevice ID (DID) driver. This driver is used to automatically assign unique IDs to every device inthe cluster, including multihost disks, tape drives, and CD-ROMs.

The DID driver is an integral part o the shared device access eature o the cluster. The DIDdriver probes all nodes o the cluster and builds a list o unique disk devices. The DID driveralso assigns each device a unique major and minor number that is consistent on all nodes o thecluster. Access to the shared devices is perormed by using the normalized ID logical name,instead o the traditional Solaris logical name.

This approach ensures that any application accessing disks, such as Solaris Volume Manager,uses a consistent path across the cluster. This consistency is especially important or multihostdisks, because the local major and minor numbers or each device can vary rom node to node.These numbers can change the Solaris device naming conventions as well.

Local Devices

The Sun Cluster sotware also manages local devices. These devices are accessible only on aSolaris host that is running a service and has a physical connection to the cluster. Local devicescan have a perormance benet over shared devices because local devices do not have toreplicate state inormation on multiple hosts simultaneously. The ailure o the domain o thedevice removes access to the device unless the device can be shared by multiple hosts.

Device GroupsDevice groups enable volume manager disk groups to become “shared” because they providemultipath and multihost support to the underlying disks. Each cluster Solaris host that isphysically attached to the multihost disks provides a path to the device group.

In the Sun Cluster system, you can control multihost disks that are using Sun Cluster sotwareby registering the multihost disks as device groups. This registration provides the Sun Clustersystem with inormation about which nodes have a path to which volume manager disk groups.

The Sun Cluster sotware creates a raw device group or each disk and tape device in the cluster.These cluster device groups remain in an oine state until you access them as shared deviceseither by mounting a cluster le system or by accessing a raw database le.




Data Services



Data services utilize several types o resources. Applications such as Apache Web Server or SunJava System Web Server utilize network addresses (logical hostnames and shared addresses) onwhich the applications depend. Application and network resources orm a basic unit that is

managed by the RGM.

Description o a Resource Group

Resources that are managed by the RGM are placed into resource groups so that they can bemanaged as a unit. A resource group is a set o related or interdependent resources. Forexample, a resource derived rom a SUNW.LogicalHostname resource type might be placed in

the same resource group as a resource derived rom an Oracle database resource type. Aresource group migrates as a unit i a ailover or switchover is initiated on the resource group.

Data Service Types

Data services enable applications to become highly available and scalable services help preventsignicant application interruption ater any single ailure within the cluster.

When you congure a data service, you must congure the data service as one o the ollowingdata service types:

■ Failover data service■ Scalable data service■ Parallel data service

Description o a Failover Data Service

Failover is the process by which the cluster automatically relocates an application rom a ailedprimary node to a designated redundant secondary node. Failover applications have theollowing characteristics:

■ Capable o running on only one node o the cluster■ Not cluster-aware■ Dependent on the cluster ramework or high availability

I the ault monitor detects an error, it either attempts to restart the instance on the same node,or to start the instance on another node (ailover), depending on how the data service has beencongured. Failover services use a ailover resource group, which is a container or applicationinstance resources and network resources (logical hostnames). Logical hostnames are IPaddresses that can be congured up on one node, and later, automatically congured down onthe original node and congured up on another node.

Clients might have a brie interruption in service and might need to reconnect ater the ailoverhas nished. However, clients are not aware o the change in the physical server that is

providing the service.


System ResourceUsage



Description o a Scalable Data Service

The scalable data service enables application instances to run on multiple nodes

simultaneously. Scalable services use two resource groups. The scalable resource group contains

the application resources and the ailover resource group contains the network resources

(shared addresses) on which the scalable service depends. The scalable resource group can be

online on multiple nodes, so multiple instances o the service can be running simultaneously.

The ailover resource group that hosts the shared address is online on only one node at a time.

All nodes that host a scalable service use the same shared address to host the service.

The cluster receives service requests through a single network interace (the global interace).

These requests are distributed to the nodes, based on one o several predened algorithms thatare set by the load-balancing policy. The cluster can use the load-balancing policy to balance the

service load between several nodes.

Description o a Parallel Application

Sun Cluster systems provide an environment that shares parallel execution o applications

across all the nodes o the cluster by using parallel databases. Sun Cluster Support or Oracle

Real Application Clusters is a set o packages that, when installed, enables Oracle Real

Application Clusters to run on Sun Cluster nodes. This data service also enables Sun Cluster

Support or Oracle Real Application Clusters to be managed by using Sun Cluster commands.

A parallel application has been instrumented to run in a cluster environment so that the

application can be mastered by two or more nodes simultaneously. In an Oracle Real

Application Clusters environment, multiple Oracle instances cooperate to provide access to the

same shared database. The Oracle clients can use any o the instances to access the database.Thus, i one or more instances have ailed, clients can connect to a surviving instance and

continue to access the database.

System Resource Usage

System resources concern aspects o CPU usage, memory usage, swap usage, and disk andnetwork throughput.

Sun Cluster sotware enables you to monitor how much o a specic system resource is being

used by anobject type such as a node, disk, network interace, Sun Cluster resource group, or

Solaris zone. Monitor system resource usage can be part o your resource management policy.

Sun Cluster also enables you to control the CPU assigned to a resource group and to control the

size o the processor set a resource group runs in.





System Resource MonitoringBy monitoring system resource usage through Sun Cluster sotware, you can collect data that

reects how a service using specic system resources is perorming and you can discoverresource bottlenecks or overload and so preempt problems and more eciently manageworkloads. Data about system resource usage can help you determine what hardware resourcesare under utilized and what applications are using a lot o resources. Based on this data you canassign applications to nodes that have the necessary resources and choose the node to which toail over. This consolidation can help you optimize the way that you use your hardware andsotware resources.

I you consider a particular data value to be critical or a system resource, you can set a threshold or this value. When setting a threshold, you also choose how critical this threshold is by assigning it a severity level. I the threshold is crossed, Sun Cluster changes the severity level o the threshold to the severity level you choose. For more inormation about conguring datacollection and threshold, see Chapter 10, “Conguring Control o CPU Usage,” in SunCluster SystemAdministrationGuide orSolaris OS.

CPU ControlEach application and service running on a cluster has specic CPU needs. Table 2–2 lists theCPU control activities available on diferent versions o the Solaris Operating System.

TABLE 2–2 CPUControl

Solaris Version Zone Control

Solaris 9 Operating System N/A Assign CPU shares

Solaris 10 Operating System Global Assign CPU shares

Solaris 10 Operating System Non-global Assign CPU shares

Assign number o CPU

Create dedicated processor sets

Note – The Fair Share Scheduler must be the deault scheduler on the cluster i you want to apply CPU shares.

Controlling the CPU assigned to a resource group in a dedicated processor set in a non-globalzone ofers you the strictest level o control o CPU because i you reserve CPU or a resourcegroup, this CPU is not available to other resource groups. For inormation about conguringCPU control, see Chapter 10, “Conguring Control o CPU Usage,” in SunCluster System

Administration Guide orSolaris OS.



http://docs.sun.com/doc/820-7358/gbylg?a=view














Visualization o System Resource Usage

You can visualize system resource data and the CPU attribution in two ways, by using the

command line or through the Sun Cluster Manager graphical user interace. The output romthe command is a tabular representation o the monitoring data you request. Through the SunCluster Manager, you can visualize data in graphical orm. The system resources that youchoose to monitor determine the data you can visualize.




32



Sun Cluster Architecture

Sun Cluster architecture permits a group o systems to be deployed, managed, and viewed as a

single, large system.


■ “Sun Cluster Hardware Environment” on page 33■ “Sun Cluster Sotware Environment” on page 34■ “Scalable Data Services” on page 37■ “Multihost Disk Storage” on page 39■ “Cluster-Interconnect Components” on page 39■ “IP Network Multipathing Groups” on page 40

Sun Cluster Hardware Environment

The ollowing hardware components make up a cluster:

■ Solaris hosts that are connected to local disks (unshared) that provide the main computing

platorm o the cluster.

■ Multihost storage provides disks that are shared between Solaris hosts.

■ Removable media are congured as shared devices, such as tapes and CD-ROM.■ Cluster interconnect provides a channel or internode communication.

■ Public network interaces enable the network interaces that are used by client systems to

access data services on the cluster.

Figure 3–1 illustrates how the hardware components work with each other.

3C H A P T E R 3

33

Sun Cluster Sotware Environment




To unction as a cluster member, a Solaris host must have the ollowing sotware installed:

■ Solaris sotware

■ Sun Cluster sotware

■ Data service application

■ Volume management (SolarisTM Volume Manager or Veritas Volume Manager)

An exception is a conguration that uses volume management on the box. This

conguration might not require a sotware volume manager.

Figure 3–2 shows a high-level view o the sotware components that work together to create the

Sun Cluster sotware environment.

Administrativeconsole

Consoleaccess device

Cluster transport adapters

ttya ttya

Cluster transport cables

Clientsystems

Public network

Host 1 Host 2

Clusterinter-connect

Localdisks

Multihostdisks

Localdisks

Multi-pathinggroup

Storage interfaces

Multi-pathing

group

IP networkmultipathing


FIGURE 3–1 Sun Cluster HardwareComponents





Cluster Membership MonitorTo ensure that data is sae rom corruption, all nodes must reach a consistent agreement on thecluster membership. When necessary, the CMM coordinates a cluster reconguration o clusterservices in response to a ailure.

The CMM receives inormation about connectivity to other nodes rom the cluster transportlayer. The CMM uses the cluster interconnect to exchange state inormation during a

reconguration.

Local disks(global devices)

Multihost disks(global devices)

Data service

API

Clusternetworking

Global device

access

Clientsystems

Volumemanagement

TCP/IP

RGM

CCR

Kernel

CMMCluster

transport


Globalnamespace

Disk device groups containingdata services resources

Cluster

file system

Otherhosts

HAframework

User

FIGURE 3–2 Sun Cluster Sotware Architecture

Chapter 3 • Sun Cluster Architecture 35




Ater detecting a change in cluster membership, the CMM perorms a synchronizedconguration o the cluster. In this conguration, cluster resources might be redistributed,based on the new membership o the cluster.

The CMM runs entirely in the kernel.

Cluster Confguration Repository (CCR)

The CCR relies on the CMM to guarantee that a cluster is running only when quorum isestablished. The CCR is responsible or veriying data consistency across the cluster,perorming recovery as necessary, and acilitating updates to the data.

Cluster File Systems

A cluster le system is a proxy between the ollowing:

■ The kernel on one Solaris host and the underlying le system

■ The volume manager that is running on a Solaris host that has a physical connection to the

disk or disks

Cluster le systems are dependent on shared devices (disks, tapes, CD-ROMs). The shareddevices can be accessed rom any Solaris host in the cluster through the same le name (orexample, /dev/global/). That host does not need a physical connection to the storage device.You can use a shared device the same as a regular device, that is, you can create a le systemon ashared device by using newfs or mkfs.

The cluster le system has the ollowing eatures:

■ File access locations are transparent. A process can open a le that is located anywhere in thesystem. Also, processes on all hosts can use the same path name to locate a le.

Note – When the cluster le system reads les, it does not update the access time on thoseles.

■ Coherency protocols are used to preserve the UNIX le access semantics even i the le isaccessed concurrently rom multiple hosts.

■ Extensive caching is used with zero-copy bulk I/O movement to move le data eciently.

■ The cluster le system provides highly available advisory le-locking unctionality by usingthe fcntl(2) interaces. Applications that run on multiple cluster hosts can synchronizeaccess to data by using advisory le locking on a cluster le system le. File locks arerecovered immediately rom nodes that leave the cluster, and rom applications that ail

while holding locks.


Scalable DataServices



■ Continuous access to data is ensured, even when ailures occur. Applications are not

afected by ailures i a path to disks is still operational. This guarantee is maintained or raw

disk access and all le system operations.

■ Cluster le systems are independent rom the underlying le system and volumemanagement sotware. Cluster le systems make any supported on-disk le system global.

Scalable Data Services

The primary goal o cluster networking is to provide scalability or data services. Scalability

means that as the load ofered to a service increases, a data service can maintain a constantresponse time to this increased workload as new nodes are added to the cluster and new server

instances are run. A good example o a scalable data service is a web service. Typically, a scalable

data service is composed o several instances, each o which runs on diferent nodes o the

cluster. Together, these instances behave as a single service or a remote client o that service

and implement the unctionality o the service. A scalable web service with several httpd

daemons that run on diferent nodes can have any daemon serve a client request. The daemon

that serves the request depends on a load-balancingpolicy. The reply to the client appears tocome rom the service, not the particular daemon that serviced the request, thus preserving the

single-service appearance.

The ollowing gure depicts the scalable service architecture.




A i k i h di i k d ild d i k S i k i ll

Cluster-Interconnect Components



A sticky service has two types:ordinary sticky and wildcard sticky. Sticky services allowconcurrent application-level sessions over multiple TCP connections to share in-state memory (application session state).

Ordinary sticky services permit a client to share state between multiple concurrent TCPconnections. The client is said to be “sticky” toward the server instance listening on a singleport. The client is guaranteed that all requests go to the same server instance, i that instanceremains up and accessible and the load balancing policy is not changed while the service isonline.

Wildcard sticky services use dynamically assigned port numbers, but still expect client requeststo go to the same node. The client is “sticky wildcard” over ports toward the same IP address.

Multihost Disk StorageSun Cluster sotware makes disks highly available by utilizing multihost disk storage, which canbe connected to more than one node at a time. Volume management sotware can be used toarrange these disks into shared storage that is mastered by a cluster node. The disks are thencongured to move to another node i a ailure occurs. The use o multi-hosted disks in Sun

Cluster systems provides a variety o benets, including the ollowing:

■ Global access to le systems■ Multiple access paths to le systems and data■ Tolerance or single-node ailures

Cluster-Interconnect ComponentsYou can set up rom one to six cluster interconnects in a cluster. While a single clusterinterconnect reduces the number o adapter ports that is used or the private interconnect, itprovides no redundancy and less availability. I a single interconnect ails, moreover, the clusterspends more time perorming automatic recovery. Two or more cluster interconnects provideredundancy and scalability, and thereore higher availability, by avoiding a single point o ailure.

The Sun Cluster interconnect uses Fast Ethernet, Gigabit-Ethernet, InniBand, or the ScalableCoherent Interace (SCI, IEEE 1596-1992), enabling high-perormance cluster-privatecommunications.

In clustered environments, high-speed, low-latency interconnects and protocols or internodecommunications are essential. The SCI interconnect in Sun Cluster systems ofers improvedperormance over typical network interace cards (NICs).

The RSM Reliable Datagram Transport (RSMRDT) driver consists o a driver that is built on

top o the RSM API and a library that exports the RSMRDT-API interace. The driver provides


h d O l R l Appli ti Cl t p Th d i l h

IP Network Multipathing Groups



enhanced Oracle Real Application Clusters perormance. The driver also enhances

load-balancing and high-availability (HA) unctions by providing them directly inside the

driver, making them available to the clients transparently.

The cluster interconnect consists o the ollowing hardware components:

■ Adapters – The network interace cards that reside in each cluster host. A network adapter

with multiple interaces could become a single point o ailure i the entire adapter ails.

■ Switches – The switches, also called junctions, that reside outside o the cluster hosts.

Switches perorm pass-through and switching unctions to enable you to connect more than

two hosts. In a two-host cluster, unless the adapter hardware requires switches, you do not

need switches because the hosts can be directly connected to each other through redundantphysical cables. Those redundant cables are connected to redundant adapters on each host.

Congurations with three or more hosts require switches.

■ Cables – The physical connections that are placed between either two network adapters or

an adapter and a switch.

Figure 3–4 shows how the three components are connected.


Public network adapters are organized into IP multipathing groups (multipathing groups).

Each multipathing group has one or more public network adapters. Each adapter in a

multipathing group can be active, or you can congure standby interaces that are inactive

unless a ailover occurs.

Multipathing groups provide the oundation or logical hostname and shared address

resources. The same multipathing group on a node can host any number o logical hostname or

shared address resources. To monitor public network connectivity o cluster nodes, you can

create multipathing.

For more inormation about logical hostname and shared address resources, see theSunCluster

Data Services Planning andAdministrationGuide orSolaris OS.

Host

Cable

OtherHosts

TransportSwitch

TransportAdapter

FIGURE 3–4 Cluster Interconnect


Public Network Interaces









Public Network Interaces

Clients connect to the cluster through the public network interaces. Each network adapter card

can connect to one or more public networks, depending on whether the card has multiplehardware interaces. You can set up hosts to include multiple public network interace cardsthat are congured so that multiple cards are active, and serve as ailover backups or oneanother. I one o the adapters ails, the Solaris Internet Protocol (IP) network multipathingsotware on Sun Cluster is called to ail over the deective interace to another adapter in thegroup.




42



Index

A

access control, 15-16

adapters, See network, adapters

administration, tools, 14-16

agents, See data services

amnesia, 23

applications

Seealso data services

ault-tolerant, 9-13

high availability, 9-13monitoring, 13-14

parallel, 10-11, 29

availability management, 10

C

cldevice command, 22Cluster Conguration Repository (CCR), 21, 36

Cluster Membership Monitor (CMM), 20-21, 35-36

clusters, 17-19

campus, 13

communication, 20

conguration, 21, 36

le systems, 12-13, 36-37

interconnects, 20, 39-40members, 17-19, 35-36

membership, 20-21

partitioning, 23-24

public network, 41

command-line interace (CLI), 15

components

hardware, 33

sotware, 34-37conguration

parallel database, 18

repository, 21, 36

tools, 14-16

Controlling CPU, 30

CPU, control, 30

D

data integrity, 23-24

Data Service Development Library API (DSDL

API), 27-29

data services

denition, 27-29ailover, 28

ault monitoring, 13-14

parallel, 29

resource groups, 28

resource types, 27

resources, 27-28

scalable

architecture, 37-39pure, 38-39

resources, 29

sticky, 38-39

types, 28-29

databases, 10-11

43

devices H

Index



groups, 26

IDs, 26

local, 26quorum, 21

shared, 25-26

disk-path monitoring (DPM), 22

disks

device groups, 26

local, 25-26

management, 12

mirroring, 12multihost, 11-13, 25-26, 26, 39

quorum, 21

shared devices, 25-26

drivers, See devices, IDs

Eenvironment

hardware, 33

sotware, 34-37

F

ailast, 24-25ailover

data services, 28

provision by Oracle Real Application Clusters

sotware, 29

services, 10-11

transparent, 10

ailuredetection, 13-14

hardware and sotware, 13-14

ault tolerance, 9-13

encing, 24

le locking, 36

le systems

clusters, 12-13, 36-37

mounting, 36-37

hardwarecluster interconnects, 39

environment, 33ailure, 13-14high availability, 9-13multipathing, 12redundant array o independent disks (RAID), 12

high availability, 9-13hosts, 17-19

IIDs, device, 26interconnects,See cluster, interconnectsinteraces, 22, 40-41, 41Internet Protocol (IP), 29IP network multipathing, 11, 22, 40-41IPMP

See IP network multipathing

Lload balancing

description, 37policies, 38-39

local devices, 26logical hostname, ailover data services, 28

Mmembership, 17-19, 20-21, 35-36monitoring

disk paths, 22ailure, 13-14network interaces, 22object type, 29system resources, 30

mounting, 36-37MPxIO, See Solaris I/O multipathingmultihost storage, 11-13

multipathing, 11, 22




split brain, 23, 24

Index



storagearray, 12

management, 11-13multihost, 11-13, 39Sun Cluster Manager, 14, 22

system resource usage, 31Sun Cluster Support or Oracle Real Application

Clusters, 29Sun Management Center, 15Sun StorEdge Trac Manager, 12

system resource, threshold, 30system resource monitoring, 30system resource usage, 29system resources

monitoring, 30object type, 29usage, 29

Tthreshold, system resource, 30tools, 14-16Trac Manager, 12

VVeritas Volume Manager (VxVM), 12 volume management, 12, 39 vote count, quorum, 21


Documents

Sun Cluster Overview for Solaris (821-0518)