Advanced Troubleshooting of Citrix NetScaler

Jacob SalassiSr. Escalation Engineer / TRM

(TECH307: Deploying and Troubleshooting the Citrix NetScaler with XenApp and Web Interface)

Troubleshooting Techniques for NetScaler

Agenda

Deploying XenApp and Web Interface with NetScaler

Troubleshooting and Best Practices for XenApp / Web Interface with NetScaler

Troubleshooting Techniques: Key NetScaler Processes

Process Description

ns_master Runs Citrix NetScaler OS

nsvpnd SSL VPN File Transfers { Samba }

nsaaad RBA and SSL VPN External Auth

nsconf Writes the ns.conf file

nsauthd CLI Authentication

nslog.sh Controls Logging for newnslog

nssync HA sync

nsreadfile Used to read SSL Cert Files

nscrlrefresh SSL CRL list update

Troubleshooting Techniques: Key NetScaler Processes (cont)

Process Description

nslcd Runs the front panel LCD

nsfsyncd Sync bookmarks and ssl certs

nsnetsvc Used by the GUI for config changes

nsumond Runs the scriptable monitors

nsconmsg Controls writing of newnslog

nscollect Statistics gathering for historical reporting

nsdynrouting, nsdrevent Handle events, HA syncs

imi, ripd, ospfd, bgpd Routing processes

• /var - Hard Drive•Logs - /var/log and /var/nslog•Install - /var/nsinstall•Trace - /var/nstrace•Core Dumps - /var/crash and /var/core

•/flash - Flash Drive•Config - /flash/nsconfig•SSL Certificates - /flash/nsconfig/ssl•User Monitors - /flash/nsconfig/monitors•Custom options - /flash/nsconfig•NS Kernel - /flash

•/ - RAM Drive•OS

Troubleshooting Techniques: NetScaler File System

• Important files in /var/log•Ns.log•Messages

•Other files in /var/log•Httperror.log•cron•Dr_error.log•Httpaccess.log•License.log•Nscollect.log•Nsvpn.log•Nsvpnd.log•Snmpd.log

Troubleshooting Techniques: /var/log

• Important files in /var/nslog• newnslog•newnslog.*.gz

•Other files in /var/nslog•ns.log•nsumond.log•nslog.nextfile

Troubleshooting Techniques: /var/nslog

•Common show commands for system information:•show node•show info•show license

•Common show commands for vserver and service:•show lb vserver•show cs vserver•show service•show persistencesession•show connectiontable

•Other common show commands:•show route•show ip•show dns addrec -type proxy

Troubleshooting Techniques: Show Commands

Show node

Show feature

Show ns mode

Show lb vserver

Show lb vserver [name]

Show service [name]

Troubleshooting Techniques: “stat” command

•Common stat commands for system information:•Stat ns•Stat cpu•Stat interface

•Common stat commands for vserver and service:•Stat lb vserver•Stat cs vserver•Stat service

•Other common stat commands:•Stat dns•Stat ssl•Stat http

Stat interface [name]

Stat lb vserver [name]

Stat service [name]

Troubleshooting Techniques: nsconmsg•Nsconmsg common use cases

•View events•View console messages•View statistics•Debug system counters•Debug load balancing issues•Debug CPU/Memory utilization

•Nsconmsg common syntax•Nsconmsg –K /var/nslog/newnslog -d event•Nsconmsg –K /var/nslog/newnslog -d consmsg•Nsconmsg –K /var/nslog/newnslog -d oldconmsg•Nsconmsg –K /var/nslog/newnslog -s ConLb=2 –d oldconmsg•Nsconmsg –K /var/nslog/newnslog -s ConDebug=1 –d oldconmsg

MAKE SURE TO USE –K, NOT -k

Nsconmsg –K /var/nslog/newnslog -d event

Nsconmsg –K /var/nslog/newnslog –d consmsg

Nsconmsg –K /var/nslog/newnslog –d oldconmsg

Nsconmsg –K newnslog –s ConLB=2 –d oldconmsg

Troubleshooting Techniques: nstrace & nstcpdump

•Nstrace•Common syntax:

•Nstrace.sh –sz 0•Nstrace.sh –sz 0 –filter “SOURCEIP = 10.198.4.10” –link enabled

•Filter qualifiers and operators:• SOURCEIP, SOURCEPORT, DESTIP, DESTPORT, SVCNAME, VSVRNAME, STATE• ==, eq, !=, neq, >, gt, <, lt, >=, ge, <=, le, BETWEEN•Compound filters using || and &&

•Nstcpdmp•Common syntax:

•Nstcpdump.sh –X tcp port 80•Nstcpdump.sh –w testcapture.cap –X src host 10.198.4.10 tcp port 80

•Filter qualifiers and operators:•tcpdump standard

Nstrace.sh

Troubleshooting Techniques: Common NetScaler issues/resolutions

• HDD Issue symptoms:•Logging fails•/var missing

•Flash issue symptoms:•Config fails to save•Config saves partially•Sync fails•Device fails to boot

•Memory starvation symptoms:•Dropped sessions

•CPU starvation symptoms:•All services failing•All VIPS down•Degraded performance


Agenda (showing current item)



•PICTURE OF MON TOPO•PICTURE OF HA TOPO•PICTURE OF GSLB TOPO

Deploying Web Interface and XenApp with NetScaler Topologies: Smart Monitoring

Deploying Web Interface and XenApp with NetScaler Topologies: Component HA

Deploying Web Interface and XenApp with NetScaler Topologies: GSLB (Business Continuity)

Deploying Web Interface and XenApp with NetScaler: Configuration overview (HA/LB)

Deploying Web Interface and XenApp with NetScaler: Configuration (CLI)




Agenda (showing current item)



Problems and Resolutions

• Users getting logged off unexpectadly

• Load balancing is uneven

Issue #1: Users getting logged off (Symptom)

Issue #1: Users getting logged off (Problem)

Resolution: Enable proper persistence

Issue #2: Load balancing is uneven (Symptom)

Issue #2: Load balancing is uneven (Problem)





Resolution: Erroneous clients, change load balancing method

Resolution: Erroneous clients, change load balancing method

Continue Your Learning

The following course expands on today's topics and are recommended to support your Citrix solution:

• CNS-200 Basic Administration for Citrix NetScaler 9.0• CNS-300 Advanced Administration for Citrix NetScaler 9.0 Platinum Edition coming soon - September

Visit www.citrixeducation.com for more information

Before you leave…

• Download presentations starting Tuesday, May 12, from your My Schedule Tool located in your My Synergy Microsite event account

Documents

Advanced Troubleshooting of Citrix NetScaler