Troubleshooting Jabber - …d2zmdbbm9feqrf.cloudfront.net/2015/usa/pdf/BRKUCC-2347.pdfFailed to authenticate with CUCM (9.0+) ... • Troubleshooting Jabber Features • Directory

Troubleshooting Jabber Like a TAC Engineer

Robbie Horgan Test Engineer

BRKUCC-2347

• Troubleshooting Logins

• Troubleshooting Jabber Features

• Tools Used By Jabber

Agenda

Which Jabber client generates the highest number of cases for TAC?

• Jabber for Mac

• Jabber for iOS

• Jabber for Android

• Jabber for Windows


• Service Discovery

• Logins

• Certificates



Agenda

Service Discovery Overview

• Enables clients to automatically detect and locate services

• Queries network for DNS SRV records

• Allows administrators to centrally configure Jabber with server side changes.

• Removes the need to use bootstrap file or having to instruct users to manually configure client

• Goal of discovery is to find a primary source of authentication

Typical Service Discovery Questions Seen By TAC

• The Jabber client is unable to discover our SRV record

Service Discovery FlowUser Launches Client

Client looks for _cisco-uds

Client looks for _cuplogin

Is Expressway Available?

Connect to available services on corporate network

Connect to Cisco WebEx Messenger service


Client Determines Services Domain

Client checks if network is inside or outside the firewall, and if Expressway is deployed

Client starts monitoring for network changesClient sends cloud HTTP request to check for WebEx service

Yes

Yes

Yes

No

No

No

Prompt user to manually enter connection detailsNo

OmittedYes


Found?

Found?

Jabber Client Showing Service Discovery Failed

Service Discovery Failure Code and LogsAll detailed discovery-related logs have [service-discovery] logger name in them

Jabber client – csf-unified.log

Example: 2014-03-15 17:07:16,829 WARN [0x00000854] [vices\impl\DiscoveryHandlerImpl.cpp(414)]

[service-discovery] [DiscoveryHandlerImpl::callOnFailedDiscoveryResultOnDispatcherThread] - Discovery

Failure -> (id) name :: (1005) ServiceDiscoveryNoSRVRecordsFound

Service Discovery Failure Codes

ID Name UI message Description

1001 ServiceDiscoveryFailure Failed to discover network services. Unknown discovery failure

1002 ServiceDiscoveryAuthenticationFailure Your username or password is not correct. Failed to authenticate with CUCM (9.0+)

1003 ServiceDiscoveryCannotConnectToCucmServer Cannot communicate with the server. Cannot connect to CUCM (9.0+)

1004 ServiceDiscoveryNoCucmConfiguration Failed to discover network services. CUCM server is misconfigured

1005 ServiceDiscoveryNoSRVRecordsFound Failed to discover network services. No SRV records are found

1006 ServiceDiscoveryCannotConnectToEdge Cannot communicate with the server. Cannot connect to EDGE server

1007 ServiceDiscoveryNoNetworkConnectivity Cannot communicate with the server. No network connectivity

Failed nslookup of SRV record

Packet Capture of Service Discovery Failed

Configure SRV in DNS




Successful nslookup of SRV record

10.10.10.51

Successful Service Discovery

Packet Capture of Successful Service Discovery

Debugging Service Discovery Cache File

• Located in “%appdata%\Cisco\Unified Communications\Jabber\CSF\Config\service-location.xml”

<?xml version="1.0"?>

<UCServices>

<DomainName>ciscolive.com</DomainName>

<UCService>

<type>CUCM</type>

<connectionInformation>

<name>_cisco-uds</name>

<scope>UNKNOWN</scope>

<address>cucm9.ciscolive.com</address>

<protocol>_tcp</protocol>

<port>8443</port>

</connectionInformation>

</UCService>

</UCServices>

Reset Jabber to Install state

ProgramData

Users

[USERNAME]

AppData

Local

Roaming

Unified Communications

Unified Communications

Jabber

Jabber

CSF

CSF

ContactsHistoryLogsPhoto Cache

ConfigCredentialsCustomEmoticonsSecurity

Local

Profile

Roaming

Profile

Jabber can be reset to Install state by removing both

roaming and local profiles.

Jabber will reinitialize on next startup

Erase Jabber folders



• Logins

• Certificates



Agenda

Common Login Questions Seen By TAC

• Users are unable to login for a new deployment.

• Only this one user is unable to login.

• All users suddenly can not login.

Incorrect Password Entered by End User

Check End User Page Login


What Do the Logs Show?User is not licensed

IM&P Server - EPASSoapXXXXX.log (Client Profile Agent Logs)

2014-03-23 14:21:54,169 INFO [http-bio-443-exec-13] handlers.LoginHandler -

prelogin:queryString=EXECUTE PROCEDURE

ucSOAPPreLogin('estaal','CUP9','10.10.10.52','CUP9.ciscolive.com');

2014-03-23 14:21:54,170 DEBUG [http-bio-443-exec-13] imdb.ImdbGeneralAccessorUtil -

getValidAppusersResultset -- query: SELECT * FROM validappusers WHERE userid='estaal';

2014-03-23 14:21:54,188 WARN [http-bio-443-exec-13] handlers.LoginHandler - preLogin:PRELOGIN

reasoncode=FAILURE. User either not CUP licensed or not found in database

Ensure User is Licensed For CUP 8

• Users do not have proper licensing configured

• In Communications Manager 8.x this is under System > Licensing > Capabilities Assignment

• Verify in CUCM for the specific user:

Ensure User is Licensed For IM&P 9.x/10.x

• In Communications Manager 9.x this is under User Management > End User

• Verify in CUCM for the specific user:

Verify Connection To CUCM & Services Are Started

• Ensure System > CUCM Publisher shows green checkmarks. If not correct issue accordingly.

• Verify the Cisco UP Sync Agent is started on the IM&P Publisher Server.

• Verify this on the serviceability page under Tools > Control Center Feature Serivces

Verify User Licensing Has Synced From CUCM• Verify licensing has replicated to IM&P server (under User Management > End User):

• If the information in IM&P does not match what is in CUCM, try restarting the Cisco UP Sync Agent to

force a sync.

• Check Sync Status System (System > CUCM Publisher)


What Do the Logs Show?IM&P Server - EPASSoapXXXXX.log (Client Profile Agent Logs)

Wrong Password

<ns1:login client-version="9.6.1.18100" client-type="CUPC"

force="true"><ns1:username>estaal</ns1:username><ns1:password>...</ns1:password></ns1:login>




2014-03-23 14:26:10,417 DEBUG [http-bio-443-exec-17] imdb.ImdbGeneralAccessorUtil -

getValidAppusersResultset -- query: SELECT * FROM validappusers WHERE userid='estaal';

2014-03-23 14:26:10,549 INFO [http-bio-443-exec-17] handlers.LoginHandler - Wrong credential for :

estaal| IMS result code:1

IMS Result Codes1) Wrong Credentials

2014-03-23 14:26:10,549 INFO [http-bio-443-exec-17] handlers.LoginHandler - Wrong credential for : estaal| IMS result code:1

2) Account locked by Admin

2014-03-23 14:34:37,576 INFO [http-bio-443-exec-7] handlers.LoginHandler - Administratively locked for : estaal| IMS result code:2

3) Account Hack lock

2014-03-23 14:38:16,259 INFO [http-bio-443-exec-20] handlers.LoginHandler - Hack locked for : estaal| IMS result code:3

4) Account locked due to inactivity

7) User inactive in LDAP

2014-03-23 14:42:53,284 INFO [http-bio-443-exec-12] handlers.LoginHandler - End user status is INACTIVE in LDAP for : estaal| IMS

result code:7

Make Sure LDAP Authentication User ConnectsCUCM > System > LDAP > LDAP Authentication

Jabber Unable to Communicate With Server

Make Sure DNS Works

• Client PC must resolve what is configured under System > Cluster Topology

Verify Correct Services Started For Login

• Verify the XCP Connection Manager and XCP Authentication Server are started on the IM&P Publisher Server.

• Verify this on the serviceability page under Tools > Control Center Feature Services

Client Profile Agent Logs For Successful LoginIM&P Server - EPASSoapXXXXX.log (Client Profile Agent Logs)

Successful Login

<ns1:login client-version="9.6.1.18100" client-type="CUPC"

force="true"><ns1:username>estaal</ns1:username><ns1:password>...</ns1:password></ns1:login>




2014-03-23 14:10:01,709 INFO [http-bio-443-exec-12] handlers.LoginHandler - doLogin:IMS login result is

success for estaal| IMS result code:0

2014-03-23 14:10:01,713 INFO [http-bio-443-exec-12] handlers.LoginHandler - SOAP Login was

successful



• Logins

• Certificates



Agenda

Common Certificate Questions Seen By TAC

• I got my certificate signed by a CA; why do I still get a prompt?

Certificate Error Message As Seen By End User

How to see it in the logsJabber client – csf-unified.log

2014-03-27 11:35:50,729 DEBUG [0x00000ae4] [src\cert\common\CertificateData.cpp(130)] [csf.cert]

[cert::CertificateData::parseSubjectCNField] - Subject CN field : CUP9.ciscolive.com

2014-03-27 11:35:50,745 DEBUG [0x00000ae4] [ls\src\cert\utils\AltNameParser.cpp(331)] [csf.cert.utils]

[cert::AltNameParser::verify] - Looking for match with CUP9

2014-03-27 11:35:50,745 ERROR [0x00000ae4] [ls\src\cert\utils\AltNameParser.cpp(375)] [csf.cert.utils]

[cert::AltNameParser::verify] - No Match Found

2014-03-27 11:35:50,745 ERROR [0x00000ae4] [ls\src\cert\common\CertVerifier.cpp(267)] [csf.cert]

[cert::CertVerifier::buildCertResult] - Verification of identity: 'CUP9' failed.

2014-03-27 11:35:50,745 INFO [0x00000ae4] [mmon\PlatformVerificationHandler.cpp(42)] [csf.cert]

[cert::PlatformVerificationHandler::handlePlatformVerificationResultSynchronously] - Verification result :

FAILURE reason : [CN_NO_MATCH]

Login Certificate Flow

Port 8443

Tomcat Certificate

(Client Validates Certificate)

Continue Login Process

CUCM Tomcat Certificate

CUCM > System > Server


Port 8443

Tomcat Certificate



Instant Messaging & Presence Tomcat Certificate

CUCM User Management > User Settings > UC Service


Port 5222

cup-xmppCertificate



Instant Messaging & Presence XMPP certificate

IM&P > System > Cluster Topology


Port 8443

Tomcat Certificate


Complete Connection

Unity Connection Tomcat Certificate

CUCM User Management > User Settings > UC Service



• Directory Integration

• Desk Phone Control and Presence

• Jabber for iOS, Android and Mac


Agenda

Common Directory Questions Seen By TAC

• Why am I unable to search for users?

• Why is the IM address incorrect?

• All my users show offline, even though they are online.

• When I add a contact, they disappear immediately.

• My contacts phone number does not show.

Directory Not Connecting

Explanation of jabber-config.xml file

• Centrally configure parameters for Jabber

• Allows administrators to make changes to client

• Prevent the need of a COP file

Sample jabber-config.xml file

<DirectoryServerType>EDI</DirectoryServerType>

<PrimaryServerName>10.10.10.10</PrimaryServerName>

<ServerPort1>3268</ServerPort1>

<UseWindowsCredentials>0</UseWindowsCredentials>

<ConnectionUsername>[email protected]</ConnectionUsername>

<ConnectionPassword>P@ssw0rd</ConnectionPassword>

<SearchBase1>dc=example, dc=com</SearchBase1>

<?xml version="1.0" encoding="utf-8"?>

<config version="1.0">

<Directory>

</Directory>

</config>

Directory Search Working

Unable to add contact

• If you try to add a contact, but they immediately disappear from the contact list, this points to an issue with the subscriber message on the IM&P server.

• There are 2 common issues that TAC sees

• User trying to be added to the contact list is not licensed for IM&P

• User A tried to add User B, User A is associated to server A and User B is associated to server B. Server A and B and in different subnets. If the communication type under System > Cluster Topology > Settings in IM&P is set to MDNS, users on the different servers will not be able to add each other. The router communication type should be set to “Router-to-Router” when the servers are in different subnets.

Incorrect IM Address









<UserAccountName>telephoneNumber</UserAccountName>



<Directory>

</Directory>

</config>

Correct IM Address

Incorrect Phone Number




<Directory>








</Directory>

</config>

<BusinessPhone>ipPhone</BusinessPhone>

Correct Phone Number







Agenda

Typical Desk Phone Questions Seen By TAC

• All users are unable to get phone control.

• This one new user has no phone control.

• We do not see phone presence for our users.

Jabber Client Showing No Phone Control

Jabber Desk Phone ControlCUCM > User Management > End User

Jabber Desk Phone ControlCUCM > Device > Phone

Jabber Controlling 99xx/89xx phone

CUCM Server - SDL001_200_XXXXX.txt (CTI Manager Logs)

• AppInfo |[CTI-APP] [CTIHandler::OutputCtiMessage ] CTI FailureResponse ( seq#=3 result=2362179824 description=User not configured to access device that supports Connected Transfer and Conference feature)

CUCM > User Management > End User

CTI Logs Showing Login Timeout

CUCM Server - SDL001_200_XXXXX.txt (CTI Manager Logs)

• 14:32:49.188 |200 |AppInfo |||CTIHandler(2,200,22,2606812)|||[CTI-APP] [CTIHandler::OutputCtiMessage ] CTI ProviderOpenCompletedEvent(seq#=370) provider id=36161244 CM Version=9.1.2.10000-28 error code=2362179701 description=Directory login failed - timeout enableIpv6=0 NoOfDaysPwdToExp=4294967295

Check LDAP Authentication PortSystem > LDAP > LDAP Authentication

Check LDAP Authentication PortSystem > LDAP > LDAP Authentication

Jabber Client Showing Phone Control

Phone Presence Not Working

?

Check the SIP Trunk to IM&P

• Ensure that the SIP Trunk to IM&P on CUCM is resolvable

CUCM > Device > Trunk

Is PublishEPA being created?

CUCM Server - SDL001_100_XXXXX.txt (CCM Manager Logs)

• |SdlSig |PublishInd |restart0 |PublishEPA(1,100,106,2) |PublishManager(1,100,105,1) |1,100,13,18.14^14.48.69.104^SEP006440B580CE |[R:N-H:0,N:1,L:0,V:0,Z:0,D:0] users.size()=1 users=estaal; pattern=9121053 numPlanPkid=5fe08c87-d701-ddb9-38f7-eaab51652ae0 devicePkid=681733d4-c469-4720-a870-781956e4552d mobileNumber= model=Cisco-CP8961/9.3.2 state=2 isDnd=F firstRegisterDevice=F deviceMac=006440B580CE

• If not, check the line association in CUCM

CUCM > User Management > End User

Phone Presence Working







Agenda

Common Questions Seen By TAC

• Why is my mobile client unable to log in?

• Why can I search with Jabber for Windows, but none of the other clients?

Jabber for iOS/Android Unable to Login

What The Logs Show

Jabber client – Jabber-2014-05-07-16042250.log

INFO [3b09218c] - [JabberWerx][log] [LoginMgr]: #1, connectOnRetrievedServerInfo login,

cup:CUP9.ciscolive.com–

INFO [3b09218c] - [JabberWerx][log] [CupSoapCli]: login cup async, server:CUP9.ciscolive.com, user:******,

ver:9.6.1.0–

INFO [7dc0000] - [JabberWerx][log] [CupSoapCli]: CupSoapClientImpl::Login,

endpoint:https://CUP9.ciscolive.com:8443/EPASSoap/service/v80–

INFO [7dc0000] - [JabberWerx][log] [CupSoapCli]: Login() result, prim:, backup:, reason:SOAP 1.2 fault:

SOAP-ENV:Sender[no subcode]

"Host not found“

Detail: get host by name failed in tcp_connect()

Where Mobile Client Gets Server Information

• Mobile device must resolve what is configured under System > Cluster Topology

User Logged In

Unable to Search Directory

Be Sure jabber-config.xml BDI Section Is Configured

<DirectoryServerType>BDI</DirectoryServerType>

<BDIPrimaryServerName>XX.XX.XX.XX</BDIPrimaryServerName>

<BDIServerPort1>3268</BDIServerPort1>

<BDIConnectionUsername>[email protected]</BDIConnectionUsername>

<BDIConnectionPassword>P@ssw0rd</BDIConnectionPassword>

<BDIPresenceDomain>example.com</BDIPresenceDomain>

<BDISearchBase1>dc=example, dc=com</BDISearchBase1>





<Directory>

</Directory>

</config>

Working Directory Search

Which information will TAC ask for when you open a case?

a) Problem Report

b) Version numbers (server and client)

c) Time the problem occurred

d) UserID of the user who is having an issue.

All of the Above!




• Jabber-config.xml

• Understanding the Problem Report

Agenda

Check if jabber-config.xml is Formatted Properly

jabber-config.xml Correct Formatting

jabber-config.xml File Generator

jabber-config.xml File Generator

jabber-config-user.xml

• This file takes precedence over the jabber-config.xml file

• Is meant for testing purposes only




• Jabber-config.xml

• Understanding the Problem Report

Agenda

How To Create A PRT

Windows Android

How To Create A PRT

iOSMac

Jabber for Windows PRTInstall Parameter

Jabber for Windows PRTLocal and

Downloaded

Config

Jabber for Windows PRT

Local and

Internal memory

Cache


Low level Library

logs


General System

Info


Main Jabber

Logs


Jabber Version

and logged in

User Info


Crash Dump


How the PRT

was created


End-users

Comments

Jabber for Mac PRT

Jabber for iOS PRT

Conclusion

Participate in the “My Favorite Speaker” Contest

• Promote your favorite speaker through Twitter and you could win $200 of Cisco Press products (@CiscoPress)

• Send a tweet and include

• Your favorite speaker’s Twitter handle <rohorganCisco>

• Two hashtags: #CLUS #MyFavoriteSpeaker

• You can submit an entry for more than one of your “favorite” speakers

• Don’t forget to follow @CiscoLive and @CiscoPress

• View the official rules at http://bit.ly/CLUSwin

Promote Your Favorite Speaker and You Could be a Winner

http://bit.ly/CLUSwin

Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.

• Complete your session surveys though the Cisco Live mobile app or your computer on Cisco Live Connect.

Continue Your Education

• Demos in the Cisco Campus

• Walk-in Self-Paced Labs

• Table Topics

• Meet the Engineer 1:1 meetings

Thank you

Troubleshooting Video

Common Video Questions Seen By TAC

• I am unable to make/receive video calls.

• Video works for everyone, except one user.

Video Call Not Working

Medianet Not Installed

Medianet Installed, Still No Video• Ensure computer is tethered to phone

• If controlling 99xx/89xx, unplug camera

• Check for CDP/CAST traffic via packet capture

• Ensure video is enabled

Video Shows Connected, Still No Video• Only H.264 is supported

• Ensure call is internal

• Check location/region settings on CUCM

Video Call Working

Contact Photos

Common Contact Photo Questions Seen By TAC

• Where does Jabber get the contact photo from?

• Why can’t I see contact photos?

Packet Capture of Contact Photos Acquired Via URL

Contact Photos via Web Server<?xml version="1.0" encoding="utf-8"?>


<Directory>

<PhotoUriSubstitutionEnabled>true</PhotoUriSubstitutionEnabled>

<PhotoUriSubstitutionToken>sAMAccountName</PhotoUriSubstitutionToken>

<PhotoUriWithToken>http://10.10.10.50/images/sAMAccountName.jpg</PhotoUriWithToken>

</Directory>

</config>

Working

2014-03-23 11:28:20,255 DEBUG [0x00000848] [2g-person\src\main\person\Person.cpp(48)] [csf.person] [person::Person::Person] - Person (00F559A8) is created from record 03501E18(Roster:[email protected]). Total(created/live): 2/2

2014-03-23 11:28:20,520 DEBUG [0x000004e0] [ices\impl\contacts\PersonWatcher.cpp(30)] [personWatcherLogger] [PersonWatcher::onPersonDataChanged] - Person 00F559A8. hasPhoto?: 1

2014-03-23 11:28:20,520 DEBUG [0x00000848] [e\src\services\impl\ContactImpl.cpp(456)] [csf-contact-impl] [ContactImpl::setHasPhoto] - Setting hasPhoto to true for [email protected]

Non-working

2014-03-23 11:41:18,226 DEBUG [0x00000a64] [2g-person\src\main\person\Person.cpp(48)] [csf.person] [person::Person::Person] - Person (00F56128) is created from record 035BA1A8(ActiveDirectory:[email protected]). Total(created/live): 7/7

2014-03-23 11:41:18,226 DEBUG [0x00000848] [ce\src\services\impl\ContactImpl.cpp(89)] [csf-contact-impl] [ContactImpl::setPerson] - Set to person 00F56128(Justin Peters). Has photo? 0

Contact Photos via jpegPhotoJabber client – csf-unified.log

mailto:[email protected]

Soft Phone Control

Soft Phone Not Registering

• TFTP server not set under Application > Legacy Clients > Settings in IM&P

• TFTP Traffic is Blocked (port TCP 6970 and UDP 69)

• Client is not able to resolve CUCM hostname in Config File

• SIP traffic being blocked (port 5060 and 5061)

• No CCMCIP profile

• No device association

Jabber Soft Phone Call Fails Out To PSTN

• Even though “Never start calls with video” is selected, video is only muted

• Video capabilities are sent to Telco

• Telco refuses the call

• Transfer Capability = Unrestricted Digital

• Need to add “bearer-cap speech” to the voice-port for outbound calls.

ISDN Se0/0/0:23 Q931: TX -> SETUP pd = 8 callref = 0x03BB

Bearer Capability i = 0x8890

Standard = CCITT

Transfer Capability = Unrestricted Digital

Transfer Mode = Circuit

Transfer Rate = 64 kbit/s

Channel ID i = 0xAAAAAA

Exclusive, Channel 22

Progress Ind i = 0x8183 - Origination address is non-ISDN

Calling Party Number i = , '123456789'

Plan:ISDN, Type:National

Called Party Number i = , '987654321'

Plan:ISDN, Type:National

ISDN Se0/0/0:23 Q931: RX <- CALL_PROC pd = 8 callref = 0x83EE

Channel ID i = 0xAAAAAA

Exclusive, Channel 22

ISDN Se0/0/0:23 Q931: RX <- DISCONNECT pd = 8 callref = 0x83EE

Cause i = 0x82AF - Resource unavailable, unspecified

ISDN Se0/0/0:23 Q931: TX -> RELEASE pd = 8 callref = 0x03BB

Documents

Troubleshooting Jabber - …d2zmdbbm9feqrf.cloudfront.net/2015/usa/pdf/BRKUCC-2347.pdfFailed to authenticate with CUCM (9.0+) ... • Troubleshooting Jabber Features • Directory