Diagnostics

XenClient Enterprise 4.5Diagnostics


Copyright © 2013 CitrixPage 2

Table of Contents

Engine and Synchronizer Problem Reporting Page 3

Problem Reports and Technical Support Service Requests Page 4

Problem Reporting FAQ Page 5

Engine Problem Reporting Applet Page 6

Engine Problem Report Form Page 7

Engine Problem Report Status Page 8

Engine Diagnostics and Synchronizer Page 9

Requesting Computer Diagnostics from Synchronizer Page 10

Collecting Computer Diagnostics from Synchronizer Page 11

Engine Debug Levels Page 12

Engine TCP Dumps Page 13

Synchronizer Problem Reports Page 14

Synchronizer Problem Report Form Page 15

Synchronizer Problem Report Tasks Page 16

Generating Synchronizer Diagnostics Page 17

Synchronizer Publish Diagnostics Page 18

Synchronizer Log Files Page 19

Synchronizer Installation and Upgrade Logs Page 20

Windows Crash Dumps Page 21

Manual Crash Dump Generation Page 22

Log Files in Windows Virtual Machines Page 23

Windows Ring Buffer Tracing Page 24



Engine and Synchronizer Problem Reporting

The XenClient Enterprise Engine and Synchronizer both have built-in features for reporting problems directly to the Citrix support and product teams. Here is how it works:

XCESynchronizer

XCEEngine

User submits report from Engine or Synchronizer.

XCE Report Server

Report is uploaded to a cloud-based report server over HTTPS.

Reports are available to Citrix support and product engineers.

In order to submit a problem report, the Engine or Synchronizer must be connected to a network with Internet access. Any firewalls or HTTP proxies in the network must allow HTTPS uploads to the host “nxbr.virtualcomputer.com” (this hostname has historical significance).



Problem Reports and Technical Support Service Requests

• Submitting a problem report from Engine or Synchronizer does not automatically create or update a Citrix Technical Support service request (SR).

• If you require Technical Support assistance with a XenClient Enterprise issue, a service request should be opened before submitting the problem report.

• The service request ID (or SR number) should be included in the subject of the problem report.

• If it is not practical to open a SR before submitting the problem report, then you can submit the problem report first. When you open the SR, indicate that a problem report was submitted, and include the following information so it can be found:

• Whether it was an Engine or Synchronizer problem report.

• Approximate date and time the problem report was submitted.

• Subject of the problem report.

• Email address of the person submitting the problem report.



Problem Reporting FAQ

What is contained within a problem report?

Mostly log files and configuration data. If you are curious you can request diagnostics from the Synchronizer and look at the contents. Ensure to exclude personal data from the problem reports.

Does a problem report open a Service Request with Citrix technical support?

No. See here for details. If you require technical assistance with XenClient Enterprise, you must open a Service Request through the usual channels. Problem reports enhance, but do not replace, the Service Request process.

Does Citrix really look at the problem reports?

Yes. Every problem report is reviewed to see if it is something we must investigate further or follow up on.

What kind of response should I expect from Citrix after submitting a problem report?

It depends. We receive many problem reports and it is not possible to respond to every one. But we might ask for more information, or suggest a possible fix or workaround for the reported problem. We can only respond if a valid email address is provided in the problem report.

How do I track the status of my problem report?

If you are submitting a problem report for an issue that must be tracked to resolution, open a service request with Citrix Technical Support. The service request can be used to track the issue. There is currently no way for the report submitter to update or review the report after it is submitted.

Why does the Engine problem report get stuck in “Pending” status?

Usually this means the computer is not connected to the internet, or something in the network (like a HTTP proxy) is blocking the upload. You can collect Engine diagnostics from the Synchronizer console instead (details), or move the computer to a network with less restrictive access to the internet.

Why is the report server “nxbr.virtualcomputer.com”?

This name has historical significance, as the XenClient Enterprise product evolved from the NxTop product developed by Virtual Computer, Inc.



Engine Problem Reporting Applet

Click the “Problem Reporting” applet.

Start the XCE Engine control panel.

Select “Tools by Category”.



Engine Problem Report Form

SubjectProvide a brief but descriptive subject. If the problem report is related to a support case or service request, include the case or service request number.

DescriptionThe description should include:•Detailed description of the problem•Summary of events leading up to the problem•Error messages displayed in the Engine•What was done to recover from the problem?•Who requested the problem report?

EmailEnter your email address so we know who submitted the report.

PriorityThe desired priority and should be based on how the problem is impacting your business operations.

ReproducibilityIndicate how reproducible the problem is.

SubmitClick Submit for submitting the problem report.



Engine Problem Report Status

Pending StatusWhen a bug report is initially submitted, it will first go into “Pending” status. This means the Engine is processing the problem report and uploading it to the report server.

Success StatusThe problem report will transition to “Success” status after the report has been generated and uploaded to the report server.

Wait For Success Before RestartingIf you are required to restart or shutdown the computer, wait for the problem report to reach “Success” status first. Otherwise the problem report would not get submitted, and interesting information about the state of the computer might get lost.

Reports Stuck in Pending StatusProblem reports can get stuck in “Pending” status if the computer does not have network access, or if a network firewall blocks uploads to the report server. The Engine might also have problems with some HTTP proxy servers.



Engine Diagnostics and Synchronizer

Engine diagnostics can be requested from the Synchronizer console. The content of the diagnostics is the same as an Engine problem report. This might be useful in the following circumstances.

•The computer is connected to a network that does not have internet access, or that has a firewall or HTTP proxy that might block the report from being uploaded to the report server over the internet.

•The engine policy is configured to block access to the launcher screen and control panel, so the problem reporting applet is unavailable.

•The computer is in a remote location and cannot be easily accessed (the remote access feature might be helpful here).



Requesting Computer Diagnostics from Synchronizer

Select the computer in the navigation panel.

Click “Request Diagnostics”.

Select the “Diagnostics” tab to see the status of the diagnostics request.



Collecting Computer Diagnostics from Synchronizer

After requesting the computer diagnostics, the request goes into a pending state, during which the following steps occur.•Synchronizer waits for Engine to check for updates.•Engine gets the message that it should generate diagnostics.•Engine generates a diagnostics package.•Engine uploads the diagnostics to Synchronizer.

After the Engine uploads diagnostics to Synchronizer, a link appears which can be used to download them. Ensure that the request state is “No Request Pending” otherwise you might get an old set of diagnostics.



Engine Debug Levels

• The debug level for Engine logging is controlled in the Problem Reporting section of the control panel.

• The available settings are:

• OFF: This is the default setting and is usually sufficient for most troubleshooting purposes.

• VERBOSE: Occasionally requested, usually for wireless networking issues.

• VERY VERBOSE: This debug level can result in unusably verbose logs and should only be set when specifically requested.

• If verbose logs are requested, the debug level should be set before reproducing the problem.

• The debug level is reset automatically after a period of time, or when the Engine restarts.



Engine TCP Dumps

• A TCP dump in the Engine can be started from the problem reporting control panel.

• The TCP data cannot be extracted directly from the Engine, but it is included in Engine problem reports or diagnostics.

• To collect TCP dump data for a networking problem:

• Turn on the TCP dump as shown.

• Reproduce the problem. The TCP dump buffer file has a limited size so the problem should be reproduced quickly.

• Submit a problem report or collect Engine diagnostics from Synchronizer.

• The TCP dump is automatically reset after a period of time, or when the Engine restarts.

• TCP dumps from the Engine are often requested in conjunction with TCP dumps taken in a Virtual Machine using a tool like WireShark (http://www.wireshark.org).



Synchronizer Problem Reports

Synchronizer diagnostics can be generated for the primary Synchronizer server, or a remote Synchronizer server. First, select the server.

Click “Submit Bug Report”.

Select the “Submit a Bug Report” option if you want to generate Synchronizer diagnostics and upload them to the Citrix report server.

Select “Generate Diagnostics Only” if you want to generate Synchronizer diagnostics without uploading them to the Citrix report server.



Synchronizer Problem Report Form

SubjectProvide a brief but descriptive subject. If the problem report is related to a support case or service request, include the case or service request number.

DescriptionThe description should include:•Detailed description of the problem•Summary of events leading up to the problem•Error messages displayed in the console•What was done to recover from the problem•Who requested the problem report

EmailEnter your email address so we know who submitted the report.

PriorityThe desired priority and should be based on how the problem is impacting your business operations.

ReproducibilityIndicate how reproducible the problem is.

Date and TimeIndicate the date and approximate time when the error occurred.



Synchronizer Problem Report Tasks

• When a request for Synchronizer diagnostics is submitted, a task is created to handle the request.

• The initial task state is “Queued” and it might take up to a minute to start running.

• When the task is complete, the diagnostics are ready.

• When the task is complete, the diagnostics have been uploaded to the Citrix report server. They can also be downloaded from the Synchronizer.



Generating Synchronizer Diagnostics

To generate Synchronizer diagnostics without uploading them to the Citrix report server:

•Select the primary or remote server in the console navigation panel.

•Click “Submit Bug Report”.

•Choose the “Generate Diagnostics Only” option.

A background task is created to handle the request. When the task is complete, the diagnostics are available to download from the Synchronizer console.



Synchronizer Publish Diagnostics

There is a special set of diagnostics available from the Synchronizer for publish failures. When a publish operation fails, these diagnostics can be downloaded from the failed publish task.

•Locate the failed publish task in the Synchronizer console.

•Click the indicated icon to display the task details. The icon looks like this:

•Click the link to download publish diagnostics.

•The publish log might also be useful to review.

Publish diagnostics cannot be uploaded to the Citrix report server from the Synchronizer. If Citrix support asks for publish diagnostics, you must retrieve them manually then send them through email or upload to ShareFile or a FTP server.



Synchronizer Log Files

• Whenever possible, Synchronizer diagnostics should be generated through the Synchronizer console using the methods outlined previously.

• But sometimes it might not be possible to collect Synchronizer diagnostics, for example, if a problem with the Synchronizer prevents the console from starting properly.

• In these cases the Synchronizer log files can be collected directly. The default folder for Synchronizer log files is:C:\Program Files (x86)\Apache Software Foundation\Tomcat 6.0\Logs

• If you are asked to collect Synchronizer diagnostics, but cannot collect them through the Synchronizer console, zip up this folder instead and send it in.

• This folder might be several megabytes in size so you might be required to use ShareFile or a FTP server to submit them.



Synchronizer Installation and Upgrade Logs

When running the Synchronizer installation program to install or upgrade Synchronizer, the following log files are created which might be useful in troubleshooting install or upgrade problems.

The Synchronizer install program will create a log file in the Temp folder for the user running the installer. Files beginning with “bitrock_installer” are Synchronizer install log files.

The installer can also generate log files in the Synchronizer installation folder. But if the initial installation of Synchronizer fails, this folder is usually deleted as part of the post-install cleanup process.



Windows Crash Dumps

If a Windows Virtual Machine crashes or blue-screens, it should generate a crash dump file into one of the following locations:

• L:\NxTop\KERNEL.DMP• C:\KERNEL.DMP• C:\Windows\MEMORY.DMP

Note that “L:\NxTop” is a hidden folder. The dump file might have a different name but it should end in a “.DMP” suffix. These files can be large so they should be uploaded to FTP or ShareFile, not sent as email attachments.



Manual Crash Dump Generation

Sometimes it might be useful to force Windows to crash and generate a dump file. This can be done by enabling the hotkey labeled “Platform: Crash VM for Debug” in the Engine control panel.

•Open the Engine control panel and choose “Tools by Category view”.•Open the “Hot Keys” applet.•Enable the hotkey labeled “Platform: Crash VM for Debug”.•Bring the Virtual Machine you want to crash to the foreground.•Enter the hotkey, this should cause the Virtual Machine to blue-screen.



Log Files in Windows Virtual Machines

The following log files are available in Windows Virtual Machines running on XCE Engine. •If you are asked for “Windows VM log files”, collect all of these.•If you are only asked for the “NxTopMonitor.log” file, the others are optional.L:\NxTop\Logs\NxTopMonitor.logThis file contains lots of useful information about how the Virtual Machine is interacting with the Engine and can be very helpful in troubleshooting issues with monitors, networking, optical drives, and many other problems within a Windows Virtual Machine.Note:•L:\NxTop\ is a hidden folder.•Some Virtual Machines might have the local drive mapped to a different drive letter.C:\Program Files\NxTop PV Drivers\This folder is in “Program Files (x86)” on 64-bit Windows. Get these files:•setupapi.dev.log•install.log•DPINST.LOGC:\Windows\inf\setupapi.dev.logThis file contains the latest record of device installs in Windows.



Windows Ring Buffer Tracing

The NxTop PV Drivers package includes a utility that can be used to dump the Windows kernel ring buffer. This is often useful when troubleshooting issues in a deployed Virtual Machine. In particular it can be useful when USB devices are not working as expected.

1.Start a command prompt with Administrator privileges.2.Navigate to one of these folders:

• 32-bit Windows: C:\Program Files\NxTop PV Drivers• 64-bit Windows: C:\Program Files (x86)\NxTop PV Drivers

3.Change directory into the “winxp”, “winlh”, or “win7x64” folder.4.Change directory into the “vcilog” folder.5.Run this command to start the trace: vcirb.exe > L:\ringbuffer.log6.Reproduce the problem while the trace is active.7.To stop tracing, press ENTER in the command prompt window.8.Send L:\ringbuffer.log to whoever requested the tracing.

The trace output file “L:\ringbuffer.log” is an example. You can save it wherever you like.

Documents

Diagnostics