Provisioning Services Stream Process Architecture and Advanced Troubleshooting ConceptsKeith McLaughlin Lead Escalation EngineerMay 11th 2010
• PVS Architecture Overview
• Stream Process Architecture
• Authentication Architecture
• Stream Process Troubleshooting
Citrix Confidential - Do Not Distribute
Agenda
Network Storage
Provisioning Services
Target Device 3
Target Device 1
Target Device 2
Workloads
A
B
What is Provisioning Services?
Network Storage
Provisioning Services
Target Device 3
Target Device 1
Target Device 2
Workloads
A
B
PXE Booting a Provisioning Services TargetDHCP Server
PXETFTP
BOOT
BOOT
BOOTDHCP Discover
DHCP Discover
DHCP Discover
IP InfoIP InfoIP Info
PXE InfoPXE InfoPXE InfoBootstrapBootstrapBootstrap
Login Process
1.1.1.1
1.1.1.2
1.1.1.3
Streaming from 1.1.1.4 Bootstrap1.1.1.2
1.1.1.1
Login Process
Request for vDisk3
PVS Target Device
StreamServer
1.1.1.3:vDisk3
Stream ServiceStream Service
Stream Service and the Stream Process
Citrix Confidential - Do Not Distribute
Stream ProcessStream Process
Stream Process Responsibilities
Citrix Confidential - Do Not Distribute
Stream ServiceStream Service
StreamDB
StreamDB
SQL Database
Target DevicevDisk Location
Login
MacAuthentication
I/O Request
Stream ProcessStream Process
• Interacts with SQL Database
• Manages the target device login process
• Streams I/O
• Manages Server Side Write Cache
• Client and Server Side Logging
• Reads the vDisk pvp – Sidecar file
Citrix Confidential - Do Not Distribute
Stream Process Responsibilites
• Class and type of vDisk
• Active Directory Support on/off switch
• Disk mode (Standard/Private/Differential Disk)
Citrix Confidential - Do Not Distribute
Vdiskname.pvp (Side Car File)
Stream Process
Citrix Confidential - Do Not Distribute
Stream ServiceStream Service
Stream ProcessStream
Process
Manager
Manager
StreamDB
StreamDB
SQL Database
Streamed ImagevDisk Location
PVS Console connected locally to the PVS Server
Citrix Confidential - Do Not Distribute
MAPIMAPI SoapServerSoap
Server
PVS Console
Manager
Manager
MCLI
ADAD
PVS Console changes to remote PVS Server
Citrix Confidential - Do Not Distribute
MAPIMAPI SoapServerSoap
Server
PVS Console
Manager
Manager
MCLI
ADAD
StreamDB
StreamDB
StreamProcessStreamProcess
StreamServiceStreamService M
anag
erM
anag
er
PVS Server A PVS Server B
More on Manager
Citrix Confidential - Do Not Distribute
Stream ServiceStream Service
Stream ProcessStream
Process
Manager
Manager
MAPIMAPI
StreamDB
StreamDB
SoapServerSoap
Server
PVS ConsoleSQL
Database
Manager
Manager
MCLI
Streamed Image
ADAD
vDisk Location
PVS Server A
StreamDB
StreamDB
Stream Process and Active Directory
Citrix Confidential - Do Not Distribute
Stream Service
Stream ProcessStream
Process
Manager
Manager
MAPIMAPISoap
Server
PVS ConsoleSQL
Database
Manager
Manager
MCLI
Streamed Image
ADADDomain Controller
PVS Server A
Authentication
Citrix Confidential - Do Not Distribute
Stream ServiceStream Service
Manager
Manager
StreamDB
StreamDB
SQL Database
Streamed ImagevDisk Location
StreamProcessStreamProcess
Load Balancing and High Availability
Citrix Confidential - Do Not Distribute
Stream ProcessStream
ProcessStream ServiceStream Service
StreamDB
StreamDB
Streamed Image
Stream ServiceStream Service
Stream ProcessStream
Process
StreamDB
StreamDB
SQL Database
Streamed Image
Server IP Server IP
Device Count Device Count
Update Time Update Time
• Narrowing down scope of issue by determining where in the boot cycle the issue is occurring.
• Packet Traces
• Capturing Kernel Dump Files
Citrix Confidential - Do Not Distribute
Troubleshooting
• DHCP – PXE boot downloading of bootstrap file
• Login to the PVS Server
• Streaming of the I/O Pre-BNIStack (Citrix Target Device driver)
• BNIStack up burst traffic.
Citrix Confidential - Do Not Distribute
Boot Stages
Citrix Confidential - Do Not Distribute
Analyzing Packet Captures
Packet Capture PXE TFTP download
Citrix Confidential - Do Not Distribute
TFTP
Provisioning Server 10.1.1.1
DHCP
Target 10.1.1.100PXE
• Target does not receive IP address
• Target gets IP but times out to TFTP
• PXE Errors No boostrap found
Citrix Confidential - Do Not Distribute
Common Issues During DHCP TFTP Download
TFTP
Provisioning Server 10.1.1.1
DHCP
Target 10.1.1.100PXE
Login Process Traffic
Citrix Confidential - Do Not Distribute
TFTP
Provisioning Server 10.1.1.1
Target 10.1.1.100
Stream Process
TFTP - Login
• Incorrect Bootstrap IP information
• Login port blocked
• Target not entered in Database
• Stream Service failed to start
Citrix Confidential - Do Not Distribute
Common issues during login process
TFTP
Provisioning Server 10.1.1.1
DHCP
Target 10.1.1.100PXE
Streaming Pre-BniStack
Citrix Confidential - Do Not Distribute
TFTP
Stream Process
OS Booting UNDI Driver
Target 10.1.1.100
• Large Send Offload
• Network instability
• Conflicts with UNDI Driver
Citrix Confidential - Do Not Distribute
Common issues pre BNIStack
Determining if BNIStack Driver is up
Citrix Confidential - Do Not Distribute
TFTP
Stream Process
OS Booting BNISTACK Driver
Target 10.1.1.100
Provisioning Server 10.1.1.1
• Filter Driver conflicts
• Firewall/Antivirus Services kickoff
• Antivirus Updates being loaded
• Domain Profile creation
Citrix Confidential - Do Not Distribute
Common issues post BNIStack
Citrix Confidential - Do Not Distribute
Windows Kernel Dump files
• Capturing Kernel Dump files using Win2k3 and XP
• Capturing Kernel Dump files using Win2k8 Vista and Windows 7
• Capturing Kernel Dump files in XenServer and ESX
Citrix Confidential - Do Not Distribute
Capturing Kernel Dump Files
• Crash dumps to be on same drive as pagefile
• Page file to be on the system drive
• Redirection of dump not supported
• Live debug can be accomplished via serial cable.
Citrix Confidential - Do Not Distribute
Capturing Kernel Dump Files Win2k3 and XP Limitations
• Enhancements allow kernel dumps to be directed to a location other then a system drive
• Requires a local hard drive with enough free space for the dump file. (Pagefile Size)
Citrix Confidential - Do Not Distribute
Capturing Kernel Dump Files Win2k8 Vista and Windows 7
• Create a new string value under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl named DedicatedDumpFile
• Create Value data box, type <drive>:\<dedicateddumpfile.sys>
• Create DWORD Value DumpFileSize use decimal under base and type the name of dump file in megabytes
• More detailed information in this process can be found:• http://support.microsoft.com/kb/969028 • http://support.citrix.com/article/ctx123642
Citrix Confidential - Do Not Distribute
Capturing Kernel Dump Files Win2k8 Vista and Windows 7
• Vista, Windows 7, Win2k8 redirection only possible if XenTools are not installed
• Live Debug via WinDBG possible by redirecting serial to DOM 0
Citrix Confidential - Do Not Distribute
Capturing a Kernel dump via XenServer
Citrix Confidential - Do Not Distribute
Capturing a Kernel Dump Via Xenserver
DO
M 0
XEN SERVER
WinDBGSockPipe.EXESerial Redirect
http://support.citrix.com/article/ctx124979
Citrix Confidential - Do Not Distribute
• Workstation 7 utility vmss2core
• Suspend hung VM
• Download suspended VM session (.vmss)
• Retrieve downloaded file and run command vmss2core –W filename.vmss
Citrix Confidential - Do Not Distribute
Capturing Kernel dump via ESX
http://support.citrix.com/article/ctx125086
Citrix Confidential - Do Not Distribute
• Provisioning Server reduces storage costs and management time
• Stream Process responsibilites
• How to identify where in the boot process a PVS Target is failing
• The different ways to capture a kernel level dump file from a provisioned target.
Citrix Confidential - Do Not Distribute
Recap
TechEdge Survey, Video Postings & PPTs
• The TechEdge survey will be emailed out to end-user customers
• If you complete the survey, you will be entered to win a $250 Amazon gift card. The winner will be announced June 1st.
• View TechEdge videos & PPTs on the Knowledge Center by Monday, May17th http://support.citrix.com/techedge2010
Visit www.citrix.com/SummitEDU for more information
Continue Your Learning