View
222
Download
0
Category
Preview:
Citation preview
IMPLEMENTATION GUIDE
Copyright © 2009, Juniper Networks, Inc. 1
IDP SERIES POLICY DESIGN AND OPTIMIZATION
Although Juniper Networks has attempted to provide accurate information in this guide, Juniper Networks does not warrant or guarantee the accuracy of the information provided herein. Third party product descriptions and related technical details provided in this document are for information purposes only and such products are not supported by Juniper Networks. All information provided in this guide is provided “as is”, with all faults, and without warranty of any kind, either expressed or implied or statutory. Juniper Networks and its suppliers hereby disclaim all warranties related to this guide and the information contained herein, whether expressed or implied of statutory including, without limitation, those of merchantability, fitness for a particular purpose and noninfringement, or arising from a course of dealing, usage, or trade practice.
2 Copyright © 2009, Juniper Networks, Inc.
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Table of Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Design Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Solution-Type Design Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
General Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Software and Hardware Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Solution Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Components of the Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Designing and Implementing a Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Defining Network and Host Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Defining a Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Steps to Implement a Recommended Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Defining Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Defining Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Customizing the Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Server Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Create Granular Notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Predefined and Custom Dynamic Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Terminal Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Exempt Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
False Positives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Optimize Traffic Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Client-to-Server/Server-to-Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Application Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Fine-tuning and Monitoring the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Security Profiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Security Explorer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
SCTOP Command-Line Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Appendix A: Resources for Ongoing Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Daily Signature Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Detector Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Known Issues Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
About Juniper Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Copyright © 2009, Juniper Networks, Inc. 3
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Table of Figures
Figure 1: IDP Series policy using the All_With_Logging template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Figure 2: Sample IDP Series deployment topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Figure 3: Defining a Web server as a host in NSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Figure 4: Predefined IDP Series policy templates available within NSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Figure 5: Creating a new security policy based on the recommended policy template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Figure 6: IDP Series rules created in the recommended policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Figure 7: Detailed description of a sample signature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Figure 8: Rules protecting internal servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Figure 9: Creating granular notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Figure 10: Listing of predefined attack group categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Figure 11: Defining a custom dynamic group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Figure 12: Exempt rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4 Copyright © 2009, Juniper Networks, Inc.
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Introduction
As the public Internet is increasingly infiltrated with rogue network traffic and threats, enterprises face the
challenges of providing secure network access while protecting their users, applications, and services from attack.
Juniper Networks® IDP Series Intrusion Detection and Prevention Appliances provide network administrators with a
powerful tool to monitor and prevent network threats—both at the protocol and application levels—by employing a
comprehensive, signature-based, attack database used in conjunction with a rule-based policy. As malware such as
worms and trojans often target vulnerabilities in protocols, the IDP Series also uses a feature called protocol anomaly
detection in conjunction with stateful signatures to inspect and detect threats before they reach the intended target.
In addition, IDP Series appliances can regulate and block unwanted peer-to-peer applications or instant messaging to
maintain compliance with corporate network use policies.
As network requirements and services are very diverse and the threats attacking them are continually evolving,
configuring intrusion prevention system (IPS) security policy to provide optimal coverage is a challenging task for
administrators. The IDP Series provides administrators with a highly flexible and configurable security policy and threat
coverage toolset. Often this level of flexibility comes with complexity and confusion around how IDP Series appliances
process rules and mitigate network threats. Once understood, the IDP Series provides administrators with a robust
number of policy management features to design the security rulebase, manage the attack objects applied, and
designate an appropriate action in response.
This document is designed to help security administrators, network operations engineers, and implementation partners
design and implement effective IDP Series security policies using the recommended policy, as well as customize and
optimize the configuration using additional features available within the IDP Series platform.
Scope
Network security administrators are continually challenged with a barrage of threats against their networks. Operating
system and application vulnerabilities are found on a daily basis. Malware such as botnets, worms, trojans, and spyware
infiltrate and steal valuable resources. Enterprises, large or small, must thwart these attacks to protect their intellectual
property. Multiple directions of threat, both internal and external, further complicate this complex task. Additionally,
unwanted peer-to-peer and chat applications constrain network bandwidth and reduce worker productivity.
In addition, increased compliance regulations and privacy laws require protections to ensure identity protection and
safeguarding of financial data. Health organizations and financial institutions are a few examples of organizations
required by federal regulations to protect consumer data from network breaches. Sensitive information such as
medical records, financial transactions, social security numbers, and credit card numbers must be protected against
unauthorized harvesting by malware. Regulatory compliance to standards such as the Health Insurance Portability and
Accountability Act (HIPAA), Peripheral Component Interconnect (PCI), and the Sarbanes-Oxley Act is essential.
Juniper Networks IDP Series Intrusion Detection and Protection Appliances provide a strong platform backed by the
Juniper Networks Security team to block network threats and ensure corporate network policy. The IDP Series offering
is a comprehensive tool, capable of providing key elements of compliance requirements. Both the standalone and
integrated Juniper Networks ISG Series Integrated Security Gateways and IDP Series are deployed in a range of small
to large enterprises, across multiple verticals. Additionally, service providers are deploying high-throughput versions of
IDP Series products. Data center applications require IPS protection to manage increasing threat sophistication as well
as rising regulatory pressures. The IDP Series provides a critical, focused defense behind a network firewall to block
network attacks and manage application-level traffic. Optimization of this layer is highly desirable, along with accurate
tuning and elimination of false positives.
This document is designed for security administrators, network operations engineers, and implementation partners. It
provides a design solution to build and optimize an effective security policy by implementing several newly developed
features in the IDP Series: recommended policy and recommended actions. The solution also discusses approaches
and features to customize IDP Series policy. Using these guidelines, an administrator can:
• Provide a thorough level of threat coverage
• Simplify ongoing policy management
• Minimize exposure to false positives
• Optimize IDP Series performance
Copyright © 2009, Juniper Networks, Inc. 5
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Design Considerations
While Juniper Networks IDP Series Intrusion Detection and Prevention Appliances provide an excellent defense against
today’s threats, administrators are often perplexed by the design of IPS policy and the selection of attack signatures to
defend against.
Often, administrators new to IPS will configure an IDP Series policy of type Any/Any/Any using the All_With_Logging
predefined template, hoping to get the full effect of the complete IDP Series signature database to protect their
networks. This will create a policy that matches Any source, Any destination, and inspects against Any attack signature.
This configuration, shown in the following screen capture, will result in an IDP Series policy that will inspect every
packet against the entire signature database—creating too many alerts and false positives, and resulting in
poor performance.
Figure 1: IDP Series policy using the All_With_Logging template
This document presents best practices to design an effective security policy using the following concepts and
features available within the IDP Series and its management software platform, Juniper Networks Network and
Security Manager.
• Begin with a network topology with defined protected segments
• List and define network segments, host, and server address objects within the NSM UI
• Create a new policy using the predefined “recommended policy” as a template
• Create specific rules for servers
• Review recommended actions
• Create and manage dynamic attack groups
• Create exempt rules
• Utilize Application Identification
• Leverage the Enterprise Security Profiler
• Correlate traffic with the Security Explorer
6 Copyright © 2009, Juniper Networks, Inc.
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Solution-Type Design Guidance
General Topology
The following topology is a typical IPS deployment at an enterprise where the IDP Series resides on the network edge
behind a firewall. The IDP Series in this position is inline with network traffic and can protect the servers (SMTP, WWW,
and DNS) as well as the additional clients and resources on the LAN. This sample topology will be used as a reference
for creating an IDP Series policy.
Figure 2: Sample IDP Series deployment topology
Software and Hardware Versions
This document is based on IDP Series release 4.1r2 along with NSM 2007.3r2.
The features and configuration samples discussed in this document were tested on a standalone Juniper Networks
IDP1100 Intrusion Detection and Prevention Appliance, though the principles apply to any standalone or integrated ISG
Series with IDP Series security module. Differences between standalone IDP Series products and integrated ISG Series/
IDP Series products are noted where applicable.
Border Router
NetScreen-5200Firewall
IDP Series
NetScreen-5200
Network andSecurity Manager
SMTP WWW DNS
DMZ
INTERNET
Copyright © 2009, Juniper Networks, Inc. 7
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Implementation
Solution Description
This document details a recommended approach to designing and customizing an effective IPS security policy by
specifying the usage of a number of key IDP Series features.
The IPS security policy in its entirety consists of the main IDP Series rulebase and five other rulebases: Exempt,
Backdoor, SYN Protector, Traffic Anomalies, and Network Honeypot. This document will focus on designing
and optimizing the main IDP Series rulebase and will also discuss the exempt rulebase. Juniper Networks NSM
Administrator’s Guide discusses the other rulebases in great detail.
Components of the Policy
The IDP Series main rulebase policy consists of the following essential parameters:
• Rule Sequence—The IDP Series policy processes traffic through a numbered sequence of rules, allowing for
prioritization of earlier rules over later rules.
• Address Objects—Before traffic inspection, the IDP Series filters traffic based on source and destination IP
addresses.
• Service—IDP Series appliances can filter inspection based on a transport layer service ((TCP, UDP, remote procedure
call (RPC), Internet Control Message Protocol (ICMP)) or port.
• Attack Objects—These form the basis of the stateful traffic inspection. In addition to REGEX-based pattern-
matching signatures, attack objects also consist of protocol anomalies that detect traffic that deviates from a
standard protocol. Attack objects are grouped into five severity levels (Critical, Major, Minor, Warning, and Info) as
well as many categories of traffic.
• Action—This defines the way IDP Series appliances handle traffic when an attack object is matched. There are a
total of nine actions available, which are discussed later in this guide.
• Notification—Each rule can have multiple notification responses, including recording a log event, various forms of
alerts, and packet logging.
This document will cover each of the aforementioned parameters while providing recommendations and examples to
help beginning IDP Series administrators create an effective policy suitable for their environment.
Designing and Implementing a Policy
Defining Network and Host Objects
The first step in creating a security policy is to identify the network segments and resources being protected. A detailed
network topology should be consulted to note subnets and work groups so that rules can be customized to permit and
deny network services. Servers should also be identified at this point and grouped by function so that attack objects
can be appropriately applied.
Within NSM, networks and hosts are added into the Object Manager as Address Objects. Once individual entries are
created, objects can be added to address groups to simplify rule creation.
The following figure defines a Web Server with an IP address of 10.1.2.101. After multiple hosts are created, hosts
requiring similar protection can be added to Groups.
8 Copyright © 2009, Juniper Networks, Inc.
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Figure 3: Defining a Web server as a host in NSM
Defining a Policy
Policy Templates
After defining Address Objects, one can begin defining the rules governing the policy. To simplify this, NSM has several
predefined IDP Series policies from which to choose:
Figure 4: Predefined IDP Series policy templates available within NSM
Some of these templates implement all attack objects. These are useful for product demos but are not practical
in production deployments as they are resource intensive. Some of the templates are specific to certain network
topologies: demilitarized zone (DMZ) environment, Domain Name System (DNS) server, FTP server, Web server, and
file server.
Recommended Policy
Customers are advised to start with the recommended policy as a template and customize additional rules with
specific networks, hosts, and application/protocol attacks that are significant for their network.
The recommended policy is a specifically designed set of rules and attack objects grouped into a policy by the Juniper Networks
Security team, which provides protection against the most relevant threats known to be present and proliferating through the
Internet. This policy provides a good starting point for protection, while allowing administrators to subsequently analyze traffic
traversing IDP Series appliances to further optimize the policy and best address their specific network policy requirements. This
approach will yield optimal IDP Series performance, while providing proper and appropriate coverage, and minimizing false
positive alerts and disruptions to legitimate traffic.
The attack groups contained within the recommended policy are updated as new vulnerabilities and threats are
found, so that the recommended policy or policies using the recommended attack groups have the latest signatures
automatically. The recommended policy also includes the attack objects protecting against Microsoft’s latest
vulnerabilities, typically providing zero-day coverage of these threats.
Updating the NSM Attack Database and applying it to IDP Series appliances will update the recommended policy with
the latest attack objects. The update procedure is further discussed in Appendix 1: Ongoing Maintenance.
Copyright © 2009, Juniper Networks, Inc. 9
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Steps to Implement a Recommended Policy
To create a new Policy based upon the recommended policy, select Security Policies within Policy Manager and click
on the (+) to create a new Security Policy. After naming the policy and specifying IDP Series as the device type, select
Use Predefined IDP Policy as Template. Choose recommended (predefined). Finally, assign the new policy to an IDP
Series appliance.
Figure 5: Creating a new security policy based on the recommended policy template
The recommended policy consists of nine rules in the main rulebase, providing protection against important TCP, ICMP,
HTTP, Simple Mail Transfer Protocol (SMTP), DNS, FTP, POP3, Internet Message Access Protocol (IMAP) attacks as
well as common Internet malware. The following figure shows the default recommended policy with the nine rules
applied to all traffic traversing IDP Series appliances.
10 Copyright © 2009, Juniper Networks, Inc.
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Figure 6: IDP Series rules created in the recommended policy
Defining Actions
When traffic matches an attack object, the IDP Series provides a number of possible actions as a response: No action,
Ignore remainder of connection, Drop Packet, Drop Connection, Close Client, Close Server, and Differentiated Services
(DiffServ) Marking. The Close actions will send reset (RST) packets for TCP traffic, while Drop actions will drop a
packet or connection without an RST packet. Depending on the nature of the attack and network service, it can be
advantageous to choose one action over another.
Recommended Actions
To simplify action selection, the Juniper Networks Security team has implemented a predefined recommended action
for signatures in the attack object database. These recommended actions are used by default in the recommended
policy and are available in standalone IDP Series 4.1 and Juniper Networks ScreenOS 6.0 for integrated ISG Series/
IDP Series products. The recommended action for a specific signature can be viewed by selecting Attack Objects > IDP
Objects > choose the specific signature > right-click > View.
The Juniper Networks Security team’s predefined recommended action provides the most appropriate response for the
threat. Traffic matching the most significant attack objects is dropped, while minor attack objects are marked for no
action. Recommended actions are assigned to predefined attack objects based on attack severity:
Copyright © 2009, Juniper Networks, Inc. 11
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
The recommended checkmark and recommended action fields are visible as well, as the detailed description of the
signature as shown in the following figure:
Figure 7: Detailed description of a sample signature
Defining Notifications
Once an action is taken, the IDP Series can be configured to provide notification of the event in multiple forms: logging,
email alerts, SNMP alerts, syslog alerts, trigger a script, and packet logging are a few of the options available. The
default notification in the recommended policy is to log the attacks.
Customizing the Policy
While the recommended policy provides a good baseline policy, an effective policy will have to be tailored to the
specific network, taking into account specific applications and servers used and any corporate or regulatory policies
needing enforcement.
Server Protection
The first step of customization is to create additional rules to protect internal servers. The rules apply additional server-
specific attack objects to internal servers. Using the simplified network topology presented earlier as an example,
additional rules are created to protect identified Web, SMTP, and DNS servers. While the recommended policy provides
basic protection of these services applicable to all traffic, more complete coverage for servers is recommended. One
may add predefined attack groups based on category and severity within the attack column.
12 Copyright © 2009, Juniper Networks, Inc.
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
In the following example, three new rules have been added to the recommended policy. Critical, Major, and Minor
Attack Objects are applied in rules specific to traffic destined for the mail, Web, and DNS servers. Additional database,
backup, or media servers should have similar rules applied.
Figure 8: Rules protecting internal servers
Create Granular Notification
In the previous example, notification action on all servers protecting rules is set as “Logging” despite having attacks
of differing severities. In a typical deployment, critical attacks will require alert notification so that quick corrective
action can be taken. To make the notification more granular, split the critical attacks into their own rules while adding
additional notification. Notification options including email, SNMP, syslog, and packet capture are all available by right-
clicking the Notification field.
In the following example, critical attacks on the Web servers will generate alert notifications, while minor-level SMTP,
Web, and DNS attacks have been configured to no notification. No notification may be desired if the network traffic
generates too much log data.
Figure 9: Creating granular notification
Predefined and Custom Dynamic Groups
To further customize the IDP Series rulebase, administrators need to apply custom rules built with relevant attack
objects for applications specific to their network. As the IDP Series attack object database is very comprehensive, it
is impractical to comb through it looking for relevant attacks. NSM addresses this by subdividing the attack database
into multiple predefined attack groups. Administrators can create very specific rules by using attack groups with
application- and protocol-specific categories.
Copyright © 2009, Juniper Networks, Inc. 13
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Figure 10: Listing of predefined attack group categories
Administrators may find the need to customize attack groups based on application, traffic direction, or severity. NSM
allows this by using custom attack groups. Custom attack groups can be created by filtering attacks based on the
following criteria:
• Product type—application, protocol, operating system, and so on
• Severity—info, warning, minor, major, critical
• Direction—traffic direction (client-to-server, server-to-client, or any)
• Attack Type—signature (threat matched against signature), anomaly (threat due to protocol deviation)
• Service Type—predefined TCP and UDP services
• False Positives—criteria if frequently, occasionally, rarely, or unknown for false positives
• Recommended—whether the attack is recommended or not
• Last Modified—date of modification for attack signature (field is deprecated and no longer used)
As new attack objects are added to the attack database by the Juniper Networks Security team, they are automatically
added to existing dynamic groups matching the defining criteria for that group.
A use case for custom attack groups would be an Internet service provider (ISP) or university administrator who wants
to allow peer-to-peer traffic, but protect against peer-to-peer vulnerabilities. This can be achieved by creating a
custom attack group containing filters for Product type Peer-To-Peer and severities Minor, Major, and Critical. Blocking
only this grouping would allow peer-to-peer traffic, but block exploits and attacks due to peer-to-peer. By making this
a dynamic custom attack group, future attack objects matching these criteria would automatically be added to the
dynamic group.
14 Copyright © 2009, Juniper Networks, Inc.
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Figure 11: Defining a custom dynamic group
Terminal Rules
Normal IDP Series rule processing occurs in a linear sequence starting with the first rule and processing until the last
rule. For each rule, traffic is matched against source/destination addresses and against the attack signature before
taking the defined action. Sometimes an administrator may want to match traffic against a set of criteria, take an
action on the matched packets, and not subject the traffic to inspection against subsequent IDP Series rules. This
can be useful to avoid multiple logs/alerts if traffic is known to match multiple signatures. This can also be useful to
troubleshoot or prevent false positives.
The “Terminate Match” checkbox creates such a terminal rule. In a terminal rule, traffic matching the source,
destination, and service is not subjected to inspection in subsequent rules. Caution must be used when using terminal
rules, as traffic only has to match the source, destination, and service to be considered terminal. The rule is terminal
even if the traffic does not match the attack. To optimize performance, terminal rules should be placed near the top of
the rulebase.
Terminal Rule Example:
(S)= Source, (D) = Destination, (Se)=Service, (A)= Attack
• Normal rule: Match (S), (D), (Se), (A) -> Take defined action, move to next rule
• Normal rule: Match (S), (D), (Se), but does not match (A) -> No action, move to next rule
• Terminal rule: Match (S), (D), (Se), (A) -> Take defined action, terminate rule processing
• Terminal rule: Match (S), (D), (Se), but does not match (A) -> No action, terminate rule processing
Copyright © 2009, Juniper Networks, Inc. 15
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Exempt Rules
The IDP Series rulebase can have attack objects that will match against traffic and produce false positives or irrelevant log
records. Alternatively, there may be specific source or destination addresses that should be excluded from attack detection.
The Exempt rulebase can be used to exclude specific source/destination pairs against specific attack objects using the
following flow of rule processing:
1. Traffic first matched a rule in the IDP Series rulebase.
2. Traffic is then matched against the Exempt rulebase.
3. If not matched in the Exempt rulebase, the specified action/log is carried out.
4. If traffic matches in both the IDP Series rulebase and a rule in the Exempt rulebase, then traffic passes without action/log.
Example:
The following IDP Series rule blocks all chat traffic from the finance network:
The exempt rule would allow MSN traffic to be permitted.
Figure 12: Exempt rules
False Positives
Unintended traffic drops or alerts are considered to be false positives. Tuning the IDP Series security policy to minimize
false positives is best achieved by crafting highly specific rules customized for addresses and specific attack objects.
Depending on the IDP Series’ placement and custom network patterns, some false positives will still be encountered.
The following approaches for handling false positives are recommended:
• Understand the attack object and its application.
• Review the attack objects description within the Object Manager to better understand how to apply the attack
object.
• Exempt the attack object from matching with an exempt rule.
- This will bypass the attack object. Create the exempt rule to be as specific as possible by using a specific source/
destination address pair and the attack object in the exempt rule.
• Collect a packet capture and notify the Juniper Security team.
- Configure a temporary 10 pre-packet and 20 post-packet post-capture under the notification column. If the
capture is TCP- based traffic, it should contain the TCP three-way handshake. Send the packet capture and
suspect attack object to signatures@juniper.net for analysis.
Example: Several MS-RPC evasion attack objects will match on SMB print job traffic generating a false positive.
As MS-RPC evasion attacks will appear on traffic originating on the WAN, apply MS-RPC evasion attack objects only
on traffic from the Internet on WAN links. MS-RPC evasion attack objects should not be applied on rules inspecting
internal LAN traffic.
16 Copyright © 2009, Juniper Networks, Inc.
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Performance Considerations
With the impressive number of features and options available to inspect traffic with IDP Series appliances, it is
important to optimize the policy and IDP Series tools utilized to maximize IPS effectiveness. As with any network
solution, the IDP Series model deployed has to be appropriate to the traffic levels of the inspected network. The peak
traffic levels should not regularly exceed the IDP Series rated capacity. This section reviews certain aspects of the IDP
Series options that are potentially taxing to performance and discusses approaches to minimize impact.
Optimize Traffic Filtering
One simple approach to improving and optimizing IDP Series policy performance is to direct traffic to relevant attack
objects. By filtering traffic in the rulebase according to source and destination addresses, packets are directed
toward specific attack signatures thereby increasing performance and also reducing false positives. An example rule
implementing this would be to filter traffic inspection for database exploits to traffic destined to an SQL server address
object only. All other traffic would bypass this rule.
Client-to-Server/Server-to-Client
Many IDP Series attack objects are defined to implement detection in a direction of flow when attempting to match
traffic—any, client-to-server, server-to-client. Server-to-client attack objects are resource intensive. The direction of
an attack object can be viewed by opening the attack object and editing it and viewing the detection tab. To improve
IDP Series policy performance, implement server-to-client signatures with specific source/destination address objects
rather than “any” to direct relevant traffic to the signature.
Notifications
Notifications can impact IDP Series performance depending on the rate and type. Performance will vary based on the specific
IDP Series model and traffic levels, but log notifications should be minimized not only to improve performance, but to reduce
the noise in log reports to ease notice of relevant attacks. While increased notification levels are a must while diagnosing
attacks, under normal IDP Series usage, no notifications are recommended for minor, warning, and info severity attacks.
Additionally, packet logging is very resource intensive as packet captures are logged to disk. Pre-packet captures
require IDP Series appliances to capture ahead of attacks and should only be used when diagnosing false positives.
Application Identification
A new feature introduced in standalone IDP Series 4.1, Application Identification (AI) can identify applications running
on dynamic ports by matching patterns in the data stream.
AI allows detection of applications running on non-standard or unknown ports. Previously, signature decoders had
to have port specified with static protocol mapping. Many peer-to-peer (Bittorent, Kaaza) and chat (Skype, Yahoo
Messenger, and so on) applications are designed to use dynamic ports. AI determines application by traffic patterns
and then applies the appropriate signatures independent of the ports being used. It will match a signature based on
the first client-to-server or first server-to-client packet.
Application Identification improves application detection and reduces false positives by not having to rely on static
port definitions in the signature.
It is enabled by default in IDP Series 4.1. The setting to disable is in the Sensor Settings > Load Time Parameters
section within NSM. If AI is disabled, older signatures based on ports and context decoders are used.
Fine-tuning and Monitoring the Network
A key component to fine-tune the IDP Series deployment is to understand the sources and destinations of typical
network traffic patterns on your network and be able to identify unique network events. IDP Series appliances have
several tools that are valuable to analyze and monitor the network . These include Enterprise Security Profiler, SCTOP
Command-Line Utility, and Security Explorer.
Copyright © 2009, Juniper Networks, Inc. 17
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Security Profiler
The Profiler is best utilized in the initial deployment to learn about your network and its resources. The Profiler collects
a database of each unique event that occurs on your network, allowing you to identify the following:
• Hosts and servers
• Traffic ports and protocols used
• Layer 3, L4, and L7 data identifying applications, host operating systems, users, and services
Profiler is initially used to create a network baseline where it identifies each host, server, and software application that
regularly appears on the network. As part of establishing a baseline, Profiler builds a database of operating systems,
applications, versions, and other parameters to characterize normal traffic. After a network baseline is established,
Profiler should be configured to alert on network deviations. Profiler can send an alert when a new host or new
application appears on the network.
As an example, Profiler can be instrumental in identifying a new infection of a host with a network-scanning worm.
Profiler quickly identifies the network scan as a non-standard activity that can fire off an alert. Review of the Profiler
database can identify the specific IP and media access control (MAC) address of the infected host, allowing a quick
response to the malware.
Detailed notes on the configuration and application of the Profiler are in the NSM Administrator’s Guide.
Security Explorer
The Security Explorer is a graphical tool allowing the administrator to correlate network traffic based on data collected
in the Profiler, Log Viewer, and Report Manager. It displays several panes that graphically depict relations between
objects based on Peer IP, inbound/outbound services, and client/server profiles as well as attacks. The report tab can
be used to view top attacks, alarms, logs, destination IPs, attacks over time, and attacks by severity.
This correlation allows the administrator to learn traffic patterns within the network that are not immediately evident
and apply policy rules tailored to the context seen.
SCTOP Command-Line Utility
Monitoring IDP Series performance and utilization is easily done using the built-in command-line IDP Series utility
SCTOP. This utility is accessed via SSH in the IPS sensor (standalone IDP Series products only) as admin, and su to
root user. SCTOP is launched via the sctop command-line interface (CLI) command.
Snapshot of SCTOP options:
sctop help ‘h’ - Display this help ‘a’ - ARP/MAC table ‘i’ - IP flows ‘c’ - ICMP flows ‘u’ - UDP flows ‘t’ - TCP flows ‘r’ - RPC table ‘x’ - RPC XID table ‘s’ - Subscriber’s status ‘m’ - Memory statistics ‘l’ - Q-Module statistics‘e’ - Rulebase statistics‘g’ - Aggregate statistics‘k’ - Attack statistics‘p’ - Spanning tree protocol‘b’ - IP Action table‘z’ - Packet distribution‘d’ - Strip Chart‘f’ - Fragment chain‘w’ - HA status‘y’ - IDS cache statistics‘q’ - Quit the program
kernel v4.1.104571-idp41‘v’ - reverse sort order‘0’ - disable sorting‘1’ - sort by bytes/session‘2’ - sort by packets/session‘3’ - sort by expiration‘4’ - sort by service‘5’ - sort by dst port‘6’ - sort by src ip‘7’ - sort by dst ip‘8’ - sort by vlan
18 Copyright © 2009, Juniper Networks, Inc.
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
The following SCTOP displays are the most useful for IDP Series monitoring:
• s—Subscriber status: Very useful to monitor IDP Series network throughput, policy, traffic peaks, uptime, version
• k—Displays attack statistics in order of frequency of hits
• i, t, u—Displays IP, TCP, and UDP flows, respectively
• g—Displays Aggregate Statistics based on sessions per IP address
• z—Packet size distribution
Summary
By implementing the approach presented in this solution design guide, a security administrator can simplify the design
and maintenance of the IDP Series policy. Using the newly developed recommend policy and recommended action
features allows the administrator to leverage the knowledge and resources of the Juniper Networks Security team to
better utilize IDP Series appliances. As with any network tool, IDP Series Intrusion Detection and Protection Appliances
have to ultimately be customized to the specific network requirements. The detailed review of IDP Series features and
implementation recommendations presented in this guide will enable the administrator to decrease false positives and
improve overall performance while minimizing ongoing maintenance of IDP Series appliances.
Appendix A: Resources for Ongoing Maintenance
Daily Signature Updates
The Juniper Networks Security team constantly monitors for the latest vulnerabilities and threats and creates
signatures on a daily basis against these threats. Customers can subscribe to the signature bulletins from the support
Web page or the following link can be used directly:
www.juniper.net/alerts/subscribe.jsp?actionBtn=Modify
NSM can be configured to automatically download the attack database on a regular basis. This procedure is detailed in
the NSM Administrator’s Guide in the “Managing Devices > Managing the Attack Database” section.
Detector Updates
Upgrading the attack database in NSM is the one step for IDP Series maintenance. In addition, the Juniper Networks
Security team regularly updates the IDP Series Detector Engine. Updating the detector is done through NSM.
http://kb.juniper.net/KB9773
Detector Engine updates can contain fixes for the protocol decoders, so upgrading to the latest Detector Engine can
solve packet loss conditions and device crashes.
Improvements that go into a new detector include:
• False positive fixes
• Decoder fixes (fix false positives and false negatives)
• New contexts (improve accuracy and performance)
• New protocol decoders (improve performance and accuracy)
• Stability and memory improvements
8010068-001-EN Dec 2009
Copyright 2009 Juniper Networks, Inc. All rights reserved. Juniper Networks, the Juniper Networks logo, Junos, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other trademarks, service marks, registered marks, or registered service marks are the property of their respective owners. Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice.
EMEA Headquarters
Juniper Networks Ireland
Airside Business Park
Swords, County Dublin, Ireland
Phone: 35.31.8903.600
EMEA Sales: 00800.4586.4737
Fax: 35.31.8903.601
APAC Headquarters
Juniper Networks (Hong Kong)
26/F, Cityplaza One
1111 King’s Road
Taikoo Shing, Hong Kong
Phone: 852.2332.3636
Fax: 852.2574.7803
Corporate and Sales Headquarters
Juniper Networks, Inc.
1194 North Mathilda Avenue
Sunnyvale, CA 94089 USA
Phone: 888.JUNIPER (888.586.4737)
or 408.745.2000
Fax: 408.745.2100
www.juniper.net
To purchase Juniper Networks solutions,
please contact your Juniper Networks
representative at 1-866-298-6428 or
authorized reseller.
Printed on recycled paper
Copyright © 2009, Juniper Networks, Inc. 19
IMPLEMENTATION GUIDE - IDP Series Policy Design and Optimization
Known Issues Update
Juniper Networks Technical Assistance Center (JTAC) publishes this update once a month to update customers of
known issues and workarounds on the latest IDP Series releases. Customers can subscribe to Known issues Update
(KIU) from this page:
www.juniper.net/alerts/subscribe.jsp?actionBtn=Modify
Attack Object Information
Each signature is described at this link:
https://services.netscreen.com/restricted/sigupdates/nsm-updates/HTML/index.html
Customers can also refer to the Juniper Security RSS feed, as all updates are listed here:
https://services.netscreen.com/restricted/sigupdates/nsm-updates/updates.xml
The latest CVE to IDP signature mapping file is on the TAC Software Download Pages:
https://services.netscreen.com/restricted/sigupdates/nsm-updates/CVE-BID-mapping.csv
About Juniper Networks
Juniper Networks, Inc. is the leader in high-performance networking. Juniper offers a high-performance network
infrastructure that creates a responsive and trusted environment for accelerating the deployment of services and
applications over a single network. This fuels high-performance businesses. Additional information can be found at
www.juniper.net.
Recommended