Embed Size (px)
Citation preview
Network TroubleshootingSave Time and Solve Problems Faster with AccessEnforcer
Jun. 27, 2019
Today’s Speakers
Duane PettusSupport DirectorCalyptix Security
Warren ParkerTraining DirectorCalyptix Security
Adam SuttonMarketing Director
Calyptix Security
• Tools for troubleshooting• Troubleshooting
Slow internet RDP not accessible Internet not accessible CalyptixVPN not connecting CalyptixVPN disconnects unexpectedly DVR, Camera, etc. not accessible Email not passing Website not accessible
• Q + A
Agenda
Tools for Troubleshooting
• Info needed- Username and password for AccessEnforcer- Network config. (internal and external)
• Tools in AccessEnforcer- Link Status- Outbound Filtering Page- ARP Table- Packet Analyzer- Ping- DNS Lookup- Telnet
Tools for Troubleshooting
• Third-Party Tools- Wireshark- IPChicken.com- Putty- www.subnet-calculator.com- MXtoolbox.com- Downdetector.com- Remote Monitoring Service
• Additional Tools When On-Site- Laptop - Serial cable (DB9 TO RJ45)- USB adapter (optional)
Tools for Troubleshooting
Network Troubleshooting with AccessEnforcer
Where is the break in communication?
Troubleshooting
Issue: Slow internet
Slow internet
• Scenario- Client complains the internet is slow.
Slow internet
• Support Team Thoughts- ISP issues
- Is the installation new? - Have any ISP settings changed?
- Firmware update- Did the AccessEnforcer update recently?
- Bandwidth hog- Is a device on the network hogging
bandwidth? - For example, did someone turn on the
backup service early? - AccessEnforcer settings
- Has the configuration changed recently?
Slow internet
• Diagnose- Ping the IP address of the modem
- use -t for continuous- Check the network outages in the area
- https://downdetector.com/- Check for these issues in AccessEnforcer
- Link Status - Duplexing issues
- IDS/IPS- Outbound scanning
- ARP table- Device in front of the AccessEnforcer is reachable
- Route table (PPPOE)- IP address of device in front of the AE is listed
- Outbound filtering- Status going in and out
Slow internet
• Diagnose (cont.)- Live connections
- State count - If the number is high,
- High Graph Utilization- Run a packet capture to determine which
device is maxing out bandwidth- Network alerts
- Port 53 blocked (means Snort is restarting)- Run packet captures
Issue: RDP not working
• Scenario- New AccessEnforcer installed- Port-forwarding to RDP fails
• Support Team Thoughts: - Is another firewall blocking the
connection?- Windows firewall- Modem
RDP not working
• Diagnose - Probably an issue with Windows Firewall- Check Port Forwarding to ensure it’s
working properly- Go to Home > Diagnostics > Ping
- Ping the machine locally- Go to Home > Diagnostics > Packet Capture
- Capture packets on Host IP and port 3389- Go to Home > Diagnostics > Telnet
- Telnet to the internal IP address and port 3389
- Ensure the machine’s gateway points to the AccessEnforcer
RDP not working
Issue: Internet not accessible
Internet not accessible
• Scenario- Big storm last night. Comcast came,
replaced the modem, and tested that it could reach the internet.
- Now they have left and I cannot reach the internet.
Internet not accessible
• Support Team Thoughts:- You said BIG STORM, right?- Are power issues causing the
problem? - Did a power surge blow a network
port? - Switch, AccessEnforcer, Modem
- Did the AccessEnforcer lose power when writing to a critical or database file?
Internet not accessible
• Diagnose- Boot the AE with the serial cable and
see if you can get to a login prompt- Could be the WAN port or one of the
other ports is blown or missing- Check Link Status for all ports
- If no IP address, the device will not pass traffic without an IP
- Check if you were in Bridge Mode- Check if Comcast put the modem back to
defaults and you are now handed an internal IP address
Internet not accessible
• Diagnose (cont.)- Check if the Modem’s firewall is
enabled- Test the ports by taking the cable out
and watching the link status - Link light will go out and when you plug it
back in should go green- Check the ARP table for WAN IP
address and Gateway WAN IP Address.
Issue: CalyptixVPN not connecting
• Scenario- CalyptixVPN client fails to make a
connection
CalyptixVPN not connecting
• Support Team Thoughts- Is this the first CalyptixVPN client for this
site?- If not, are others able to connect?- If not, did they work previously?
- AccessEnforcer configuration correct? - CalyptixVPN enabled? - External IP address correct under Setup > VPN
> CalpytixVPN Settings?- CalyptixVPN configuration correct?
- Correct IP address in configuration? - TAP Adapter working properly?
CalyptixVPN not connecting
• Diagnose- Check the IP address on the client
- Right click the red circle and go to “edit Config”.
- See if it matches the public IP of the AccessEnforcer
- Check the Setup > VPN > CalyptixVPN page and make sure it is enabled and has the correct external IP address.
- Check the Home > CalyptixVPN Login attempts and see if clients have recently logged into the AccessEnforcer
- Check the Packet capture and see if you see traffic from the source IP address.
- If not, it is probably the TAP adapter
- If so, then click the Apply button on the CalyptixVPN settings page
CalyptixVPN not connecting
Issue: CalyptixVPN disconnects
CalyptixVPN disconnects
• Scenario- Calyptix VPN connects, but the
connection stops after a short while
• Support Team Thoughts - Is the same client trying to connect
from two locations at once (duplicate connection)?
CalyptixVPN disconnects
• Diagnose:- Check Home > CalyptixVPN Login
attempts- The Client number is generally logging in
an out- This generally means there is a duplicate client
connected at the same time. - I have seen where a client at the same location
has connected with a computer upstairs and is trying to login with their laptop downstairs with the same client.
- Create a new client called (Name) Laptop or (Name) phone and use that specific for that device
Issue: DVR, Camera, Etc. Not Connecting
• Scenario- Switching from another UTM firewall
and now the DVR does not work.
• Support Team Thoughts:- DVR
- Is the gateway correct?
- Port Forwarding- Is it configured correctly?
DVR, Camera, Etc. Not Connecting
• Diagnose- Gather information about the DVR
and what changed the other firewall- Log into AccessEnforcer
- Port Forwarding- Check the configuration
- Perform two packet captures- Try to reach the device from the LAN- Try to reach the device from another network- Check the capture for one-way or two-way
traffic- Look at the ports in the capture for clues
DVR, Camera, Etc. Not Connecting
Issue: Email not passing
Email not passing
• Scenario- Emails are no longer being sent or
received.
Email not passing
• Support Team Thoughts: - Is email entering the network? - Is email exiting the network? - Is email hosted behind the AccessEnforcer? - Does a third-party service filter spam before
forwarding to the network? - If so, is it on the Dynamic Blacklist?
- Are you getting a bounce back messages?- Did the ISP block port 25 on the modem?
Email not passing
• Diagnose: Inbound- Check the SMTP Rate limiter- Check a bounce back message against
the portal article https://online.calyptix.com/smtperror
- Check MXtoolbox and make sure that the MX record did not change
- Check the Port forwarding page that it did not stop taking traffic
- Setup the Packet captures and watch the traffic all the way through
Email not passing
• Diagnose: Outbound- Go to the Advanced page and make
sure that Only allow these SMTP … is not limited
- Hit the apply button on the Advanced page
- Watch the traffic to see if it gets to us on both the LAN and WAN
Issue: Website not accessible
• Scenario- Client cannot open a website but is
able to open others.- The client can open the website on
another network (one not behind an AccessEnforcer).
Website not accessible
• Support Team Thoughts:- Is something blocking access to the
website? - AccessEnforcer – IDS/IPS, Web Filter- ISP
- Is something blocking your IP? - Site host - Content hosting service (Akamai, AWS,
Microsoft)
- What firmware version is on the AccessEnforcer?
Website not accessible
• Diagnose:- Log into the AccessEnforcer
- Dynamic Blacklist
- Try to access the site from behind a different AccessEnforcer
- If the site opens, then the cause is unlikely to be a bug in the firewall
- Perform two packet captures- Get the website’s IP address (via DNS
lookup)- Watch the traffic
Website not accessible
Online Portal
• First place you should check for information on AccessEnforcer
• Location: https://online.calyptix.com/
• Resources include: - AccessEnforcer Handbook
- Training videos
- Release notes
- Price list, deals, and more
• Questions and issues we resolve often become Portal articles.
• Trouble accessing the portal? Contact [email protected]
Online Portal
Partner Deals
Safer for the Storms Ahead!• Big discounts on AccessEnforcer with 1 year
of services, 3 years of warranty
• Free month of service on all monthly firewalls deployed this month when you deploy 3 or more
See the deals:• Go to https://online.calyptix.com• Click “Partner Deals”
QUESTIONS?
Bonus: IPsec VPN Problems
• Scenario- IPsec VPN traffic is slow
• Mitigation: - Whitelist the internal IP range and the Peer IP address of
the remote site. The IDS/IPS will not inspect these packets so the speed through tunnel should increase.
• Steps- Log into AccessEnforcer- Go to Security > Network > Static Whitelist- Add the remote peer’s local range and the remote peer’s
IP address
IPsec VPN Problems
• Scenario- IPsec VPN tunnel went down unexpectedly and will not come
back up.
• Possible cause: - Remote peer’s external IP address was added to the Dynamic
Blacklist.
• Steps- Log into AccessEnforcer
- Go to Security > Network > Static Blacklist
- Check if the remote peer’s IP is listed
- If so, remove the IP from the list
- Go to Security > Network > Static Whitelist
- Add the remote peer’s external IP address
IPsec VPN Problems
• Scenario- AccessEnforcer will not connect to my Cisco
• Possible cause: - Misconfiguration of one or more of the devices.
• Steps: - In AccessEnforcer, the Phase 2's Diffie Hellman
Group must be set to None- Follow the instructions shown on this Online
Portal article: https://online.calyptix.com/node/224
IPsec VPN Problems
• Scenario- AccessEnforcer will not connect to my RV042/RV082
• Possible cause- The Linksys RV042 / RV082 does not support automatic keying with
the AccessEnforcer.
- There also appears to be a bug in the RV042 / RV082 GUI where the Incoming SPI will only accept a 7-character SPI value at most.
• Note- AccessEnforcer allows 7-character SPI’s to inter-operate with
Linksys devices.
• Mitigation- Set a manual keying policy instead.
IPsec VPN Problems
Bonus: Lost Email
• Scenario- Client did not receive a specific email.
• Steps- Confirm the sender is using correct email
address- Check bounce errors
- Check if sender received a bounce message
- Troubleshoot error messages - Identify the exact phrase/number of the error message- Identify the source of the error message (use Google)- Look to the error source for fastest resolution
Lost Email
• Common email bounce errors from AccessEnforcer
- 554 Your IP address ... is not allowed. (Error LR-55443.)
- Cause: IP address is on the SMTP Rate Limiter
- 554 Your IP address ... is not allowed. (Error OEG-55768.)
- Cause: IP address is on the Geographical Policy
- 554 Your IP address ... is not allowed. (Error 86117.)
- Possible cause: IP address is on the Static or Dynamic Blacklist
- Possible cause: IP address does not resolve correctly with the FQDN check
Lost Email
• Common email bounce errors from AccessEnforcer (cont.)
- 552 Message size exceeds fixed maximum message size- Possible cause: Message size limits on Exchange server need to be
increased.
- 550 Message rejected as spam by Content Filtering.- Possible cause: Content filtering on Exchange server is set too high.
Remove the filter on the Exchange server. This is common on Exchange 2007 and higher.
- 503 Need mail command - Possible cause: This is a common message that one of your drives on the
Exchange server is out of space and cannot accept any more connectors.
• Complete list of SMTP errors: https://online.calyptix.com/node/253
Lost Email
• Check if the email is in the user’s quarantine- Go to Security > Email > Quarantine by Address- Click the user’s email address to open the
quarantine- Click “See All” on the top-right- See if the email is listed
- Message highlighting
- Red – error from SMTP server
- Dull Red – message flagged as a virus
- Blue – forwarded to the SMTP server
- White – held in spam filter
- Dark grey – deleted
Lost Email
• Check if the email is in the user’s quarantine (cont.)
- If the message is found and is safe- Click the “release” icon on the left to forward
the email to the user
- If the message is found and no safe- Click the email to preview- Click “Show reason” for more information- Click “SMTP Error” to reveal the exact error
message
Lost Email
Bonus: Squid Error
• Steps- Find the IP of the website- Check the Dynamic Blacklist
- Go to Security > Network > Dynamic Blacklist
- Check if the IP address is listed- If the address is listed:
- Remove the address from the Blacklist
- Go to Security > Network > Static Whitelist- Add the IP address to the Whitelist
Squid Error
• Steps (cont.)- Change settings for IDS/IPS rule
- Go to Home > Network Alerts- Look for the IP address- Click the checkbox next to the IP address- Click the drop-menu and select “Ignore the
Rule that cause alert”- Restart IDS/IPS to apply the changes
- Go to Security > Network > IDS/IPS Settings- Click “Apply”
Squid Error
Squid Error: Example
Squid Error: Example
Bonus: Serial Cable
Serial cable
• AccessEnforcer ships with DB9 to RJ45 cable
Serial cable
• Recommend adding R-232 to USB cable
Serial cable
• Why It’s Used- A serial connection allows you to access
the booting scripts that Calyptix engineers can use to determine problems with the device.
- This will help determine if the device is booting correctly and if the unit can access the login screen.
- This is essential in determining problems with a device to see if it should be replaced or returned through the RMA process.
Serial Cable
• Connecting- Download Putty.exe and install
(http://www.putty.org/ )- Open the application- Under “Connection Type” click
“Serial” and then “Open”- Use the serial cable to connect the
AccessEnforcer to your computer
Serial Cable
