Embed Size (px)
Citation preview
Clustering SQL 2005 with ESX 2.5.x and SAN
EMC Proven™ Professional Knowledge Sharing June, 2007
Bartley Corbin Sr. Implementation Specialist
EMC Corporation® [email protected]
Table of Contents 1 VM Clustering Overview............................................................................................................... 3
1.1 The New Standard for VM Clustering ............................................................................ 3 1.2 Clustering Software.............................................................................................................. 4 1.3 Clustering Hardware ............................................................................................................ 4
2 Clustering with ESX Server (Breaking all the rules)......................................................... 5 2.1 ESX Server with MSCS Requirements .......................................................................... 5 2.2 Setting up the Hardware for SAN with ESX Server ................................................. 6
2.2.1 Configuring Your EMC CLARiiON® Storage System ............................................. 6 2.2.2 Configuring VMFS Volumes on SANs ........................................................................ 6 2.2.3 Using Raw SCSI Disk Mapping to Cluster ............................................................... 7 2.2.4 Clustering Across Physical Machines......................................................................... 8
2.3 VMFS Locking and SCSI Reservation ............................................................................ 8 2.3.1 VMFS File System Locking ............................................................................................ 9 2.3.2 Locking at SCSI Disk Level ........................................................................................... 9 2.3.3 Using LUN Masking to Avoid Locking Issues.......................................................... 9
2.4 Configuring Virtual Machine Clusters with Shared Disks....................................... 9 2.5 Two Node Cluster with MSCS on a Single ESX Host ............................................. 10
2.5.1 Creating the First Node's Base Virtual Machine.................................................. 10 2.5.2 Creating the Second Node Virtual Machine .......................................................... 14 2.5.3 Installing Microsoft Cluster Service ......................................................................... 14 2.5.4 Running Microsoft Cluster Service ........................................................................... 15
3 Installing and Configuring SQL Server 2005 ..................................................................... 16 3.1 SQL Server Installation .................................................................................................... 16
3.1.1 Creating the First Cluster Server.............................................................................. 16 3.1.2 Creating the Second Cluster Server........................................................................ 17
3.2 Verifying SQL Services...................................................................................................... 17 3.3 Failover Test ......................................................................................................................... 20
Disclaimer: The views, processes or methodologies published in this article are those of the author. They do not necessarily reflect EMC Corporation’s views, processes, or methodologies.
1 VM CLUSTERING OVERVIEW
1.1 THE NEW STANDARD FOR VM CLUSTERING VM Clustering uses two or more virtual servers hosted on one or more physical servers running ESX to provide another layer of redundancy or continuous service. The following graphic illustrates VM Clustering in a box with two VMs on one physical ESX host:
ESXHost
MSC
S
VM
VM
Figure 1-1
In this illustration, all the nodes in a cluster serve up storage to various clients. The primary storage node distributes data store files requested from various analysts throughout the environment for data reduction from the ingest zone. Clustering redirects these requests to the other node if one crashes, providing redundancy, increasing availability, or providing longer up-times to the end user. Power Path® is not supported in earlier versions of ESX. Network Load Balancing, included in Windows, can balance request loads. Please remember that ESX Server 2.x does not support active load balancing; only path failover is available. Instead, set the preferred path to enable some level of load balancing across HBAs (known as static load balancing). In this scenario, you could have a combination of the illustrated solution, displayed above, if you are running an ESX environment; and use MSCS in combination with replication software such as SAN Copy™ to replicate your storage data. We chose AAM and RepliStor® for this engagement. However, you could easily virtualize this environment. I have included an orange MSCS arrow ( ) to demonstrate where you could insert the ESX host(s) as pictured above into this environment.
Figure 1-2
This article describes a more complex configuration: a single node serving a database. If that node crashes, the clustering software must restart the database on another node. The database application knows how to recover from a crash. In normal operation, additional nodes are used for running other applications so we only need to be concerned with the database application in this scenario.
This example gets increasingly complex when we add a quorum disk that resolves tie-break situations when the voting set of nodes disagrees on the current state of the cluster.
All applications running need to be cluster aware to take advantage of clustering services.
Such applications can:
• Be stateless, as Web servers and VPN servers • Have built-in recovery features, like those in database servers, mail servers, file servers or print
servers
1.2 CLUSTERING SOFTWARE The scope of the available clustering software for this article includes:
• Microsoft Clustering Service (MSCS) – Provides fail-over support for 2 to 8-node clusters for applications such as databases, file servers and mail servers
• AutoStart™ formerly known as Automated Availability Manager (AAM) used in conjunction with RepliStor
1.3 CLUSTERING HARDWARE A typical clustering setup includes:
• Disks that are shared between nodes are required if the application uses dynamic data; examples include mail servers and databases. The shared disks may be shared raw SCSI disks or a VMFS volume in a storage area network via Fibre Channel.
• Extra network connectivity between nodes for monitoring heartbeat status
• A method for redirecting incoming requests
2 CLUSTERING WITH ESX SERVER (BREAKING ALL THE RULES)
2.1 ESX SERVER WITH MSCS REQUIREMENTS Clustering, with respect to ESX Server and SANs, provides failover support for your applications in virtual machines (VMs) between ESX Server machines. The primary requirement is that all the clustered virtual machines reside on VMFS volumes within a SAN when you are using the SAN to support multiple ESX Server machines. There is an additional layer of application redundancy when clustering in a VM environment. The two primary advantages of virtualization include the ability to run several virtual machines on one physical computer, and to migrate the virtual machines to other physical computers seamlessly using live migration or Vmotion Technology. (Figure 2-1)
Figure 2-1
FIGURE 2-1
NOTE You cannot boot your ESX Server machines from the SAN if you are using Microsoft Clustering Services. The ESX Server boot from the SAN option is still viable if you are using other clustering services that rely
on access to the VMFS volume.
More information is available in your Microsoft Cluster Service documentation and your VMware ESX Server Administrator’s Guide. Before beginning, please read vendor documentation if you need additional background (i.e. basics for working with ESX or MSCS).
The following lists the ESX with MSCS requirements:
• ESX Server running version 2.5.x • Microsoft Cluster Service, on Windows Server 2003 install disk • SAN connectivity and allocated storage or shared SCSI disk • Network connectivity • Additional NICs for heartbeat monitoring • Additional HBAs
Requirements specific to this install:
• OS – Windows Server 2003 Enterprise Edition R2 • Windows SQL Server 2005 with Service Pack 1
NOTE The R2 edition provides Microsoft licensing advantages with VMware. Recent license restructuring with Microsoft allows one instance of the R2 edition to run on four separate VMs simultaneously. If running
the Longhorn version, instances on a single server are limitless from a licensing perspective. Be sure to have a clean build of R2.
DO NOT try to upgrade an existing Server 2003 box. You will experience multiple WMI errors and your updates will fail. There were also issues installing Service Pack 1 for SQL 2005 in a clustered
environment when running VMware. Greater detail is provided in a later section.
2.2 SETTING UP THE HARDWARE FOR SAN WITH ESX SERVER
2.2.1 Configuring Your EMC CLARiiON® Storage System EMC CLARiiON storage systems work with ESX Server hosts in SAN configurations.
To configure:
1. Install and configure your storage device. 2. Configure zoning at the switch level. 3. Create RAID groups. 4. Create and bind LUNs. 5. Register the servers connected to the SAN. 6. Create storage groups to contain the servers and LUNs.
Refer to EMC documentation for information on configuring the storage systems. For ESX environments, you will have to break from the original “Best Practice.” In ESX environments, you must make your LUNs available to all of your ESX hosts to make your VMs Vmotion capable or to enable clustering. Vmotion as well as clustering will not work in a virtualized environment if the storage is not visible to all ESX hosts in your environment.
NOTE ESX Server exhibits the same alignment problems seen on ALL Intel based platforms. Please refer to
EMC document Storage Best Practice for VMware 2x to address this issue. It can become performance limited as environments grow.
2.2.2 Configuring VMFS Volumes on SANs Ensure that only one ESX Server system has access to the SAN while you are using the VMware Management Interface to configure the SAN and format the VMFS-2 volumes.
NOTE After you have finished the configuration, ensure that all partitions on the physically shared SAN disk are
set for public or shared access for access by multiple ESX Server systems as when using Vmotion.
In addition, change the VMFS partition access rights where you store the shared virtual disks. VMFS partitions are configured for public access by default. The VMFS partition needs to be configured for
shared access to support clustering.
In other words, Vmotion works in public (default) or shared access mode, but clustering only works in shared access mode.
Maximum Number of VMFS Volumes ESX Server supports up to 128 LUNs. These include both local volumes and LUNs visible on a SAN. A large number of visible LUNs could exceed the limit.
If this occurs, use LUN masking or zoning to prevent the server from seeing LUNs it doesn't need to access. How you do this, will depend on the type of storage array and the management software used.
2.2.3 Using Raw SCSI Disk Mapping to Cluster The shared disk used for clustering can also be a raw shared SCSI disk, rather than a shared VMDK file on a VMFS volume. Using a raw SCSI disk as a shared disk simplifies initial setup. Also, it may be useful for importing an existing physical cluster that already has cluster data on a SCSI disk.
In addition, using a raw SCSI disk as a shared disk allows a virtual machine to participate in a cluster with a physical machine. For example, the virtual machine can be used as the passive node for a physical machine that is the active node.
Figure 2-2
Replace the instructions in section 2.5.1.1 Virtual Disk Configuration with the following steps for the virtual machine to access a physical disk.
To add a physical SCSI controller and shared raw SCSI disks, access the Hardware tab.
1. Click Add Device. 2. Click Hard disk. 3. Click System LUN/Disk to give your virtual machine direct access to a SAN or shared storage
volume. 4. Choose the LUN/Partition you want to attach to this VM as a raw disk.
NOTE Physical disks are identified by a vmhba number in an ESX Server. For example, vmhba0:1:2:1 means physical
adapter vmhba0, target 1, LUN 2, partition 1. When the final number is (:0), it indicates that you are specifying the entire disk, rather than a particular partition.
5. Choose the virtual SCSI node to which you want to attach the raw disk.
NOTE Shared disks must be attached to a separate SCSI controller from the system disk. Select SCSI 1:1.
If it is not available, choose the next available SCSI controller.
6. Click OK. A new virtual disk and SCSI Controller appear on the Hardware tab.
7. Click Edit next to SCSI Controller to change the bus sharing from none to physical.
8. From the Bus Sharing drop-down list choose physical, then click OK. Setting the bus sharing to physical ensures that all the SCSI reserve and reset commands go through to the physical disk.
Repeat steps 1-8 to create an additional shared raw disk using SCSI 1:2. You have completed the virtual machine configuration. 2.2.4 Clustering Across Physical Machines For maximum flexibility, put each shared virtual disk (VMDK) on a separate VMFS on a separate SCSI disk (or LUN) since SCSI reservation can be done only at the granularity of a complete SCSI disk or LUN. If two shared virtual disks are on the same VMFS, reserving one disk will also reserve the other disk. If you put more than one shared disk on the same VMFS (as in the example installation above), always put the virtual disks in the same MSCS resource group. That ensures the disks are always reserved by the same computer and always faile over at the same time.
Ensure that the swap file for each ESX Server machine is not on any of the physical shared disks. Since the shared disks are reserved by one host or another, the swap file may become inaccessible to one of the hosts during the cluster’s operation.
If you are accessing the shared disks on a SAN using an Emulex host bus adapter, take the following additional steps:
• On the Options > Advanced Settings page, set DiskUseDeviceReset to 0 and DiskUseLUNReset to 1.
• Supply an extra parameter to the Emulex driver when it is loaded by editing the file /etc/vmware/hwconfig. First, identify the bus, slot and function holding the first (or only) Emulex card. You can find this information by looking at the Startup Profile page. Then, to the end of /etc/vmware/hwconfig, add a line with the following format:
device.vmnix.5.13.0.options = "lpfc_delay_rsp_err=0"
Here, the numbers 5.13.0 specify the bus, slot and function where the Emulex card is located. If you have more than one Emulex card, you should have only a line referencing the first card.
If you are doing multipathing and can access your shared cluster disk via multiple HBAs from a host, then you need to modify another configuration file.
1. Go to the Options > Advanced Settings page
2. Set DiskResetOnFailover to 1
If an HBA failover occurs on the host, it issues a SCSI bus reset after the failover happens. This bus reset allows the second HBA to access a shared disk, even if the disk has been reserved by the first HBA. The clustering software then automatically reserves the disk again via the second HBA.
2.3 VMFS LOCKING AND SCSI RESERVATION For a shared SCSI disk that can be accessed by multiple ESX Server machines, two kinds of locking may be in use. They are somewhat independent and can cause confusion. The shared SCSI disk may be on a shared SCSI bus or, more likely, on a storage area network (SAN).
2.3.1 VMFS File System Locking VMFS file system locking is the first kind of locking. It ensures there is no corruption caused by multiple accesses to the file system by different hosts. ESX Server locks VMFS file systems on a server level when a VMFS file system is configured as either public or shared.
Configure the VMFS as a shared file system for failover clustering. Then, the locking protocol is slightly relaxed to allow multiple virtual machines on different servers to access the same VMFS file simultaneously.
However, file system commands do the same locking as the public file systems (that is, per-VMFS in VMFS-1 volumes and per-file in VMFS-2 volumes).
NOTE Only one server at a time can access a VMFS if the VMFS volume is configured in public mode (change
to Shared) .
If a VMFS is used to store a virtual disk that is accessed by multiple virtual machines on multiple physical servers for the purposes of failover clustering, configure the VMFS as a shared file system.
If one server is accessing the VMFS-1 volume through a virtual machine, a file system operation by another host will fail. For example, if you have a cluster between two VMs and you attempt to start or issue a command to start the second node, the command fails with a message that says:
vmkfstools: file system is locked by another server
2.3.2 Locking at SCSI Disk Level The second type of locking is at the SCSI disk level, also called SCSI disk reservation.
Any server connected to a SCSI disk can issue a SCSI command to reserve the disk. As long as that reservation exists, no other server can access the disk. If no other server is already reserving the disk, the current server obtains a reservation on the disk. All SCSI commands to that disk by other servers fail with an appropriate error code.
For example, if you have a cluster between two VMs and you attempt to start; or a command is attempted on a VMFS on a disk that is reserved by another server; the command fails with a message that says:
vmkfstools: shared SCSI disk is reserved by another server
Use 'vmkfstools -L release/reset' to end reservation if no other server is using the SCSI reservation.
2.3.3 Using LUN Masking to Avoid Locking Issues Locking issues are likely to happen on a SAN where multiple users may be accessing some of the same disks or may mistakenly access a disk assigned to another user.
Use LUN masking or zoning to limit which disks are visible to each server in the system to minimize the impact of one user on another. This is particularly useful when you have multiple environments, such as a test environment and a development environment that you want to isolate from the production environment. It could be catastrophic if one server unexpectedly locks or reserves the wrong SCSI disk.
2.4 CONFIGURING VIRTUAL MACHINE CLUSTERS WITH SHARED DISKS To create a set of clustered virtual machines, you need to configure each of them with:
• A primary virtual SCSI host adapter with one SCSI virtual disk • At least two virtual network adapters
• A public network adapter connected to vmnicx (that is, to vmnic0 or higher). A vmnic is a virtual machine device that uses a network adapter dedicated to the virtual machines.
• A private network adapter connected to vmnicx (that is, to vmnic0 or higher) or to vmnet_x (that is, to vmnet_0 or higher). This device selection must match in all virtual machines in a cluster set. This is the network adapter that the clustering service will use to monitor the heartbeat between nodes.
The remaining default virtual machine devices should be added (such as the CD-ROM drive and the floppy disk drive). In addition to the above devices, shared storage requires:
• A secondary virtual SCSI host adapter • One or more shared virtual disks attached to the secondary SCSI host adapter
Each virtual machine has five PCI slots available by default. Four of these slots are used in this configuration (two network adapters and two SCSI host bus adapters). This leaves one PCI slot for a third network adapter if needed.
2.5 TWO NODE CLUSTER WITH MSCS ON A SINGLE ESX HOST This procedure creates a two-node cluster using Microsoft Cluster Service on a single ESX Server machine.
(Names used here are for illustration purposes only; however, names should be logical to the purpose of the said node.)
SQL1 – host name of node 1 of the cluster SQL2 – host name of node 2 of the cluster SQLCLUSTER – public host name of the cluster Shared Disk – VMFS volume label of the shared storage Resource Disk – VMFS volume label of the local storage Quorum Disk – VMFS volume label of the quorum storage
NOTE Virtual disks stored on the Resource Disk and Shared Disk can also be stored on the same partition. Use
the partition label on which these virtual disks reside.
2.5.1 Creating the First Node's Base Virtual Machine 1. Access the VMware Management User Interface (MUI) by typing the following:
Log on as the user who will own the virtual machine.
2. Click Add Virtual Machine and keep the default Guest Operating System selection of Microsoft Windows 2003 Server.
NOTE This example uses Windows 2003 Server Enterprise Edition R2 as the guest operating system. You may
substitute another Windows operating system that supports Microsoft Cluster Service.
3. Change the Display Name field to describe the virtual machine — for example, MSCS Node 1 (SQL1).
4. Change the Location of the virtual machine configuration file to /home/<user>/vmware/SQL1/SQL1.vmx.
5. Click Next. 6. Select the number of processors you want the guest operating system to use, up to 2. 7. Change Memory to show the amount of RAM you want to allocate to this virtual machine. 8. Click Next. 9. Click Blank to create a new virtual disk. 10. Choose the VMFS volume on which you want to store the virtual disk. 11. Give the virtual disk image a unique name — for example, SQL1.dsk. 12. If you need a primary SCSI disk larger than 4GB, enter the appropriate value in the Capacity field. 13. Choose the virtual SCSI node to which you want to attach the virtual disk. 14. Click Persistent to verify the disk mode. By default, the disk mode is set to persistent. 15. Click Next.
You have successfully created the virtual machine.
The hardware tab for this virtual machine appears. From that tab, you now need to add additional hardware devices.
Virtual Disk Configuration You need a shared SCSI controller and shared SCSI disk for shared access to clustered services and data.
To add the shared SCSI controller and shared SCSI disk, click the Hardware tab, and:
1. Click Add Device. 2. Click Hard disk. 3. Click Blank to create a new virtual disk. 4. Choose the VMFS volume on which you want to store the virtual disk. 5. Give the virtual disk image a unique name — for example, quorum.dsk. 6. Enter the appropriate value in the Capacity field. 7. Choose the virtual SCSI node to which you want to attach the virtual disk.
NOTE Shared disks must be attached to a separate SCSI controller. Select SCSI 1:1
1. Click Persistent to verify the disk mode. By default, the disk mode is set to persistent. 2. Click OK.
NOTE A new virtual disk and SCSI Controller 1:1 are now visible on the hardware tab.
3. Click Edit next to SCSI Controller 1:1, change the bus sharing from none to virtual. 4. From the Bus Sharing drop-down list, select virtual, then Click OK.
Repeat the steps to create an additional shared virtual disk using SCSI 1:2 with the filename resource.dsk.
Repeat the steps to create an additional shared virtual disk using SCSI 1:3 with the filename shared.dsk.
Network Device Configuration Microsoft Cluster Service uses an additional virtual network adapter to maintain the cluster heartbeat. To add this adapter, click the Hardware tab for this virtual machine, then:
Click Add Device and then Click Network Adapter.
From the Device Binding drop-down list, choose a private network adapter connected to vmnicx (that is, to vmnic0 or higher) or to vmnet_x (that is, to vmnet_0 or higher). This device selection must match in all virtual machines in a cluster set.
This attaches the second Ethernet adapter to a private network between the cluster nodes. The clustering service uses this network adapter to monitor the heartbeat between nodes. It may be easier to create two separate Virtual Switches. Plan carefully before moving forward. One example of how you could utilize Virtual switching in your ESX Cluster environment follows. (Figure 2-3)
SCSI2SCSI2SCSI1SCSI1
VSCSI1VSCSI1VSCSI1 VSCSI1 VSCSI2 VSCSI2 VSCSI2 VSCSI2
VNIC2 VNIC2 VNIC2 VNIC2 VNIC1VNIC1Internal0 Switch Internal0 Switch
External Network External Network
VNIC1 VNIC1
External0 Switch External0 Switch
VLAN B VLAN B SQL2 Node
VM
SQL2 Node VM
SQL1 Node VM
SQL1 Node VM
Figure 2-3 Finally, Click OK. You have created the first cluster node virtual machine.
Installing the Guest Operating System Now you need to install Windows 2003 Server Enterprise Edition R2 in the virtual machine you just created. Be sure to insert the OS CD Windows 2003 Server Enterprise Edition R2 in the ESX Server machine’s CD-ROM drive.
NOTE Make sure that you have selected the CD-ROM device on the hardware configuration tab and not the ISO image with directory path. ISO images may be more convenient, but may cause issues later in the install.
In the management user interface (MUI), click the blue terminal icon next to the virtual machine's name to launch the remote console. Logon as the user who created the virtual machine or as root. Remember that user name and password are case sensitive.
Now, power on your VM and install Windows 2003 Server Enterprise Edition R2 and accept all the default options during the installation. Install the VMware Tools and change acceleration to FULL mode as shown in (Figure 2-4) below.
Figure 2-4
Cloning the First Node’s Base Virtual Machine Now that you have a virtual machine with Windows 2003 Server Enterprise Edition R2 installed, you can save time by cloning it as follows:
1. Run sysprep.exe, which is available on the Windows 2003 CD in the \support\tools\deploy.cab file. This strips the security ID (SID) assigned to the guest operating system and resets the machine information and the TCP/IP network configuration.
2. Shut down the guest operating system and power off the virtual machine.
3. Remove the Windows 2003 Server Enterprise Edition R2 CD from the server's CD-ROM drive.
4. Click Manage Files on the MUIs Overview page.
5. Drill down to the vmfs folder or the directory folder storing the (SQL1.dsk) file. This may take some time to refresh.
6. Select the check box next to the (SQL1.dsk) file.
7. Copy and Paste this file in the same folder so that you can change the filename to (SQL2.dsk).
This concludes the cloning process. Now continue with creating the second node virtual machine.
2.5.2 Creating the Second Node Virtual Machine Create a new virtual machine as follows:
Refer to Section 2.5.1 Creating the First Node’s Base Virtual Machine above. Be sure to change the Display Name field to describe the virtual machine — for example, MSCS Node 2 (SQL2). For the remaining steps, replace (SQL1) with (SQL2). Virtual Disk Configuration Refer to Section 2.5.1.1 Virtual Disk Configuration above. Network Device Configuration Refer to Section 2.5.1.2 Network Device Configuration above. You have created the second cluster node virtual machine.
Go to the MUIs Overview page. The management interface should list both virtual machines and indicate that they are powered off.
2.5.3 Installing Microsoft Cluster Service In the MUIs Overview page, click Node 1 (SQL1) virtual machine. Click Start.
Follow the Windows Server 2003 Enterprise Edition R2 mini-setup prompts to enter the OSs serial number, the host name (SQL1) and the IP addresses. Note that you need to enter the addresses for both public and private network adapters.
• For the public network adapter, enter an IP address that belongs to the physical network.
• For the private IP address, you may use an address like 192.168.x.x with a class C subnet mask (255.255.255.0).
Windows automatically reboots once you have completed this process.
After the reboot, open the Disk Administrator to ensure that all shared disks are visible and displayed in the left panel as basic disks. Format all shared virtual disks NTFS if they are not already formatted. Assign the first shared disk to drive Q: (quorum) and the second disk to R: (resource) and the third to S: (shared). If you have joined this virtual machine to an existing Active Directory domain, skip the next step.
Run dcpromo.exe from the command prompt to start the Active Directory Wizard. Set up the current machine as a domain controller. For the domain name, use something like VMcluster.domain.com where domain.com is your DNS domain and VMcluster is your Active Directory domain. This node may be setup as a new domain tree and also a new domain forest, or it may join existing ones. This step requires the installation of a DNS server. Set the domain permissions as mixed mode unless you plan otherwise.
You will need a service account with Domain Administrative privileges. So, add an account named Cluster, then check User cannot change password and Password never expires.
Now you are ready to add MSCS. Insert the Windows 2003 Server Enterprise Edition R2 CD in the server's CD-ROM drive>Go to Add/Remove Programs>Select Add/Remove Windows Components.
Select the Cluster Service component; Click Next. Follow the prompts to install the service. As you configure Cluster Service, choose Form a New Cluster and then specify the cluster name (SQLCLUSTER) and the cluster IP address. This is the address that will represent the cluster. It must be on the same network as vmnic0. You will be prompted for the cluster service account that you created above; so, enter Cluster. The cluster service should manage all shared disks.
Stop the cluster service on the local node SQL1 (from Cluster Manager, right-click the node name), so the second virtual machine can access the shared disks.
In the MUIs Overview page you must right click Node 2 (SQL2) virtual machine and then Start. Start Node 2 (SQL2) virtual machine. Repeat all of the steps above. In the Node 2 (SQL2) virtual machine, repeat all of the steps above with one exception: select Join a Cluster instead of Form a New Cluster and then specify the cluster name (SQLCLUSTER).
This concludes the Microsoft Cluster Service installation and configuration.
2.5.4 Running Microsoft Cluster Service Microsoft Cluster Service should operate normally in the virtual machine once installed.
NOTE Some disk errors are recorded in the Windows event log in normal operation. These error messages have
a format similar to
The driver detected a controller error on \Device\Scsi\BusLogic3
They should be reported periodically only on the passive node of the cluster and should also be reported when the passive node is taking over during a failover. The errors are reported because the active node of the cluster has reserved the shared virtual disk(s). The passive node periodically probes the shared
disk and receives a SCSI reservation conflict error. This is normal operation.
Related Knowledge Base Articles Required patches and fixes for MSCS clustering support with Windows 2003 SP1 and R2:
• ESX-3199476 Patch for ESX Server 3.0.1 (3199476) • ESX-2066306 Patch for ESX Server 3.0.1 (2066306) • The Hotfix from Microsoft for Blue Screen on Windows 2003 SP1 and R2 Appears When
Volumes are Mounted (2267) Recommended KBs for clustering in general:
• Setup for Microsoft Clustering Service (http://www.vmware.com/pdf/vi3_vm_and_mscs.pdf) • Must Use Physical Compatibility Mode for Shared Disks When Using Microsoft Clustering Service
with Raw Device Mappings (1822) • Microsoft Cluster Service Doesn't Start (1455) • Manually Setting Cluster IP Address for Clustered VirtualCenter Server (2195771) • Creation of New RDM for MSCS Cluster Node when Cluster Service is Active on Another Node
(1862) ESX Server 2.5.x Windows 2003 SP1 introduced a change that resulted in MSCS clusters not functioning correctly in any ESX Server version. Fixes are available for:
• ESX Server 2.5.3 Upgrade Patch 4 (http://www.vmware.com/download/esx/). • ESX Server 2.5.4 Upgrade Patch 2 ( http://www.vmware.com/support/esx25/doc/esx-254-
200611-patch.html). • You must also apply the hot fix available from Microsoft, as described in
http://support.microsoft.com/kb/911030.
3 INSTALLING AND CONFIGURING SQL SERVER 2005
3.1 SQL SERVER INSTALLATION After you create and configure the virtual machines and have successfully installed MSCS, install two instances of SQL Server 2005 Database Service, one for each of the virtual machines in the cluster. The cluster servers are called SQL1 and SQL2.
3.1.1 Creating the First Cluster Server The following steps explain how to use the command prompt to perform an attended installation. This makes SQL Server installation on multiple machines faster and simpler.
1. Place the SQL Server 2005 installation DVD in the DVD drive.
2. Click on the Start button on the task bar.
3. Click Run.
4. Type cmd to open the MS-DOS shell command prompt.
5. Type the following command at the prompt:
Start /wait E:\servers\setup.exe /qb INSTANCENAME=SQL1 ADDLOCAL=SQL_Engine SAPWD=<password> SQLACCOUNT=<domain>\<username> SQLPASSWORD=<password> AGTACCOUNT=<domain>\<username> AGTPASSWORD=<password> SQLBROWSERACCOUNT=<domain>\<username> SQLBROWSERPASSWORD=<password> SECURITYMODE=SQL
Replace the drive letter (E:) with the correct DVD drive for your system. Replace <domain>, <username>, and <password> with the correct values for your system.
Setup cycles through several installation screens, installing SQL Server 2005 based on the parameters you specified using the command prompt. The following items appear in the Add/Remove programs under control panel.
Figure 3-1
A log file (Summary.txt) displaying the install messages and any errors can be found under C:\Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\. Additional log files are located in C:\Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Files.
NOTE Refer to Microsoft KB article (918357) containing important information about how to modify the registry.
Back up the registry before you modify it; and stop all MSCS services before proceeding.
ERROR Message: BUG: Error message when you try to install SQL Server 2005 Service Pack 1: "Unable to install Windows Installer MSP file".
NOTE Refer to Microsoft KB article (910851). Important: You receive error messages when you try to set up a
clustered instance of SQL Server 2005.
ERROR Message: Tried setup and it failed "Remote setup failed" "SQL Server Remote Setup .job" (setup.exe) 6/19/2006 6:23:11 PM ** ERROR ** Unable to start task. The specific error is: 0x80070005: Access is denied. Try using the Task page Browse button to locate the application. [***** Most recent entry is above this line *****]
3.1.2 Creating the Second Cluster Server Repeat steps 1 through 5 above to create a SQL Server on the second node in the cluster, with the following changes:
Replace the instance name in the command with INSTANCENAME=SQL2. Make any changes to the domain, usernames, and passwords as necessary.
3.2 VERIFYING SQL SERVICES Once you have successfully installed SQL Server, complete the following steps to verify the SQL services.
1. Click the Start button in the taskbar. 2. Click Run. 3. Type services.msc to view the available services.
Figure 3-2
If the services on your machine match those in (Figure 3-2), your SQL services are running correctly. Close the window.
4. Check that all necessary configuration tools installed correctly by clicking Start > Program Files > Microsoft SQL Server 2005 > Configuration Tools.
Figure 3-3
The list of tools on your machine should match those shown in Figure 3-3.
5. Select SQL Server Configuration Manager. The management window appears.
Figure 3-4
Enable all of the protocols exactly as shown in Figure 3-4.
NOTE Refer to Microsoft KB article (912397). DO NOT enable the VIA Protocol. This will render your SQL
cluster useless and is irreversible without more Registry edits. If you make an error, you have to begin again.
ERROR Message: SQL Server error log: Server <server name> initialization failed with error 0x32, status code 0x1c.
6. Ensure that the TCP/IP protocol for server settings is configured properly if you are hard coding the port number (Figure 3-5).
Figure 3-5
7. Click on the SQL Server Surface Area Configuration. Make sure the remote connection is enabled as shown in Figure 3-6.
Figure 3-6
8. IMPORTANT: Before connecting from the client, add the server name Alias to the SQL Server configuration at the client machine, as shown in Figure 3-7 on the following page.
Figure 3-7
9. Connect the SQL server SQL1 using SQL Server Management Studio. Make sure that you use the same port number used in the SQL Server configuration.
3.3 FAILOVER TEST After your successful installation, ensure that failover works properly on SQL1 and SQL2 servers for both Cluster Group and SQL Group. For testing, set Failback on both cluster servers to immediate. It is disabled by default.
1. Start Cluster Administrator.
2. Click on Resources to see which server owns the resources. Particularly, observe who owns the Cluster Group.
3. Right click on Cluster Group, and select Move Group. Only the cluster group should be moved from SQL1 (referred to as SERVERC2MOD) to SQL2 (referred to as SERVERC3MOD) or vise versa (refer to Figure 3-8).
Figure 3-8
4. Remote login to SQL1 if you have not already done so. Make sure all resources are owned by
SQL1 server. If not, move them all to SQL1 (referred to as SERVERC2MOD), as shown in Figure 3-9 below.
Figure 3-9
5. From start, click on SQL Server Configuration Manager to make sure all services are running in
SQL1 server (see Figure 3-10).
Figure 3-10
NOTE Do not be concerned If SQL Server Reporting Services is not running; you will configure it later.
6. Remote login to SQL2. Check the above services in SQL2. With the exception of SQL Server
Browser, all services should be stopped, as shown in Figure 3-11.
Figure 3-11
7. Attempt to start a stopped service such as SQL Server. It should NOT start. You should get an error message, because all services are running on SQL1. In this case SQL1 owns the cluster, so SQL2 should not be able to start the service.
8. Now, unplug the power cable from SQL1 (This assumes VMs on separate ESX Hosts). Within seconds, you should see that all resources are automatically moved to SQL2 (referred to as SERVERC3MOD) due to Failover, as shown in Figure 3-12 below.
Figure 3-12
NOTE CAUTION: Though we do not recommend powering down the ESX server unexpectedly, it must be done
at least once to ensure disaster recovery. Gracefully shutting down the server or powering down the virtual machine will not emulate a true crash scenario.
9. From Start, select SQL Server Configuration Manager, and click on SQL Server 2005 Services. The services shown in Figure 3-13 should be running in SQL2 server after failover:
Figure 3-13
NOTE SQL Server Reporting Services appears to be missing in SQL2 because Reporting Services is not a
cluster aware application as mentioned in Section 1 of this article.
10. Remotely log in to your Virtual SQL IP address. You should see SQL2 server. Do not log off.
11. Plug the power cable into ESX and bring up the virtual machine and SQL1 server. If you have set failback to immediate, all services should automatically be moved to SQL1. If failback was not set to immediate, you must manually move all services to SQL1.
12. Check your Virtual SQL. Notice that you need to log in again, but upon login, instead of SQL2, you should see the SQL1 server. This is because resources are now owned by SQL1 after Failback.
Now you have completed your install and configuration of Microsoft SQL Server 2005 running on ESX 2.5.x Virtual Infrastructure in a clustered VM environment and a SAN or shared raw SCSI disk back end.
