Upload
randy-smither
View
215
Download
2
Embed Size (px)
Citation preview
1NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved.© 2001, Cisco Systems, Inc. All rights reserved.© 2001, Cisco Systems, Inc. All rights reserved.
NATO NATO Advanced Networking Workshop Advanced Networking Workshop
S4.2 Contemporary Network Management
September 18th, 2001
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 2© 2001, Cisco Systems, Inc. All rights reserved. 2© 2001, Cisco Systems, Inc. All rights reserved. 2
Buying a Network Management System should be easy…
Sigma Systems
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 3© 2001, Cisco Systems, Inc. All rights reserved. 3© 2001, Cisco Systems, Inc. All rights reserved. 3
ISO Architecture for Network Management
Configuration Configuration ManagementManagement
Fault Fault ManagementManagement
Security Security ManagementManagement
Performance Performance ManagementManagement
Accounting Accounting ManagementManagement
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 4© 2001, Cisco Systems, Inc. All rights reserved. 4© 2001, Cisco Systems, Inc. All rights reserved. 4
Planning &Planning &
OrganizingOrganizing
DesignDesign
ImplementImplement
Network Life Cycle
S
U
R
I
EC
TY
AnalyzingAnalyzing
ChangesChanges
MONITORINGMONITORING
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 5© 2001, Cisco Systems, Inc. All rights reserved. 5© 2001, Cisco Systems, Inc. All rights reserved. 5
TMN Open Reference Architecture Customer Interface
Fulfillment Assurance Billing
SalesSales OrderHandling
OrderHandling
ProblemResolutionProblem
ResolutionPerf./SLAReportingPerf./SLAReporting
Invoicingand RatingInvoicing
and Rating
Service Product Development and MaintenanceService Product Development and Maintenance
Network and Systems ManagementNetwork and Systems Management
NetworkPlanningNetworkPlanning
ElementManagement
ElementManagement
NetworkProvisioning
NetworkProvisioning
MaintenanceRestoration
MaintenanceRestoration
NetworkMonitoringNetwork
Monitoring
ServiceCreationServiceCreation
ServiceInventoryService
InventoryService
ProvisioningService
ProvisioningServiceQualityServiceQuality
MediationAggregationMediation
Aggregation
Programmable and Physical Network LayersProgrammable and Physical Network Layers
Plug-and-Play, Configuration, Policy, InstrumentationPlug-and-Play, Configuration, Policy, Instrumentation
Cisco Network DevicesCisco Network Devices
Inte
gra
tion
Bu
sIn
teg
ratio
n B
us
PartnerCisco
NetworkNetworkServicesServices
Data• CIM/DEN
Model• Caching/state• Repository
Data• CIM/DEN
Model• Caching/state• Repository
Security• Author/authent• RADIUS,
Kerberos, TACACS+, PKI
Security• Author/authent• RADIUS,
Kerberos, TACACS+, PKI
Location• Location• Registration• Naming
Location• Location• Registration• Naming
IP Address Mgmt• DNS• DHCP• Address mgmt.
IP Address Mgmt• DNS• DHCP• Address mgmt.
Workflow• Process
workflow• Application
integration
Workflow• Process
workflow• Application
integration
Customer Care
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 6© 2001, Cisco Systems, Inc. All rights reserved. 6© 2001, Cisco Systems, Inc. All rights reserved. 6
Agenda
• Motivation for Network Management
• Evolution of Basic Technologies
• Designing for Network Management
• Best Practices
• Policy Management
• Summary and Recommended Reading
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 7© 2001, Cisco Systems, Inc. All rights reserved. 7© 2001, Cisco Systems, Inc. All rights reserved. 7
• 80% say managing your network is significantly more important than 18 months before
• Why?
Your business relies more on the network
Your network is more complex than before
Your network is more visible than ever before
You can’t hire and keep enough good people
Network Management Challenge
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 8© 2001, Cisco Systems, Inc. All rights reserved. 8© 2001, Cisco Systems, Inc. All rights reserved. 8
IT Organization Challenge
Network Management Service Management
Utility Strategic Asset
Facilitate High ReliabilityLeverage the Organizational ResourcesMinimize Transmission Costs
Facilitate High ReliabilityLeverage the Organizational ResourcesMinimize Transmission Costs
Identifying opportunities to use Information Technology to help the corporation better compete
E-CommerceExtranets & VPNsVoIP
Identifying opportunities to use Information Technology to help the corporation better compete
E-CommerceExtranets & VPNsVoIP
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 9© 2001, Cisco Systems, Inc. All rights reserved. 9© 2001, Cisco Systems, Inc. All rights reserved. 9
Evolution of Network Management
• Networks are increasing in scale and complexity—there is a clear need for management functionality
• Management Technologies evolve along with the technologies and services deployed in networks
Network Traffic andNetwork Technology
Network Resources(Support Staff, $$)
Growth
Time
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 10© 2001, Cisco Systems, Inc. All rights reserved. 10© 2001, Cisco Systems, Inc. All rights reserved. 10
Heterogeneous Management Servers
xmlCIM xmlCIM
Device ID
Management Intranet
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 11© 2001, Cisco Systems, Inc. All rights reserved. 11© 2001, Cisco Systems, Inc. All rights reserved. 11
Agenda
• Motivation for Network Management
• Evolution of Basic Technologies
• Designing for Network Management
• Best Practices
• Policy Management
• Summary and Recommended Reading
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 12© 2001, Cisco Systems, Inc. All rights reserved. 12© 2001, Cisco Systems, Inc. All rights reserved. 12
SNMPManager
(CW 2000)
NetworkTime Protocol
NTP
NTP
NTP
CDP orILMI
CDP
ILMI
CDP CDP
IP
IP
IPIP
IPConnectivity
IPIP
MIBSNMP AgentMini-RMON
RMON-MIBCISCO-STACK-MIBBRIDGE-MIB...
MIBSNMP Agent
MIB—RMON 1 and 2SNMP Agent
Get, GetNext, Set, GetBulk
Responses, SNMP Traps
SNMPTraps/RMON
MIBSNMP Agent
Syslog
Syslog
Syslog Message
Syslog
Syslog
Network Management Technology Basics
Telnet
Telnet
TelnetTelnet
Telnet
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 13© 2001, Cisco Systems, Inc. All rights reserved. 13© 2001, Cisco Systems, Inc. All rights reserved. 13
(optional)
The Syslog Facility
ConsoleMessages RS-232
console
syslog 514/udp
Syslog Server
config logfile
facility severity level timestamp system log messagesystem log message
Severity Level Description
0 Emergencies
1 Alerts
2 Critical
3 Errors
4 Warnings
5 Notifications
6 Informational
7 Debugging
Text messages over UDP
Very basic reporting mechanism
CatOS CatIOS IOS
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 14© 2001, Cisco Systems, Inc. All rights reserved. 14© 2001, Cisco Systems, Inc. All rights reserved. 14
SNMP The Management Entity, Agents, and Protocol
• Management entity collects data by generating requests; this causes in-band traffic coexisting with production traffic
• Agents are information storehouses of object definitions provided in many Management Information Bases (MIBs)
• SNMP protocol is used to transport the information requests
SNMPSNMPAGENTAGENT
NetworkManagement
Station IP Network
SNMPManageable
Device
ManagementManagementEntityEntity
Get Request, Get-Next RequestGet-Bulk Request
Set Request
Get Response
Trap !
SNMP v1, SNMP v2
1000s ofDefined Objects
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 15© 2001, Cisco Systems, Inc. All rights reserved. 15© 2001, Cisco Systems, Inc. All rights reserved. 15
SNMPUnderstanding Community Strings
• SNMP Protocol Data Units (PDUs) are processed as per the access policy indicated by the community string
• Community strings are clear text and provide a trivial authentication mechanism
• Avoid using the well known defaults:
Read-only agent access: public
Read-write agent access: private
Frame Header
CRC
UDP Header
Port161
SNMPMessageIP
Header
Protocol NumberUDP (17) Packet Payload
Frame Payload
VersionCommunity
StringSNMP PDU
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 16© 2001, Cisco Systems, Inc. All rights reserved. 16© 2001, Cisco Systems, Inc. All rights reserved. 16
MIBs: Management Information Bases
• A MIB defines the variables that reside in a managed nodeDefined according to SMI (Structure of Management Information) rulesEach managed object is described using an object identifier defined in the SMI
• MIB I114 standard objectsObjects included are considered essential for either fault or configuration management
• MIB IIExtends MIB I185 objects defined
• Other standard MIBsRMON, host, router, ...
• Proprietary vendor MIBsExtensions to standard MIBs
SNMPAGENTSNMPAGENT
1000s of Manageable Objects DefinedFollowing Rules Set Out in the SMI Standards
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 17© 2001, Cisco Systems, Inc. All rights reserved. 17© 2001, Cisco Systems, Inc. All rights reserved. 17
• Hierarchically structured
• Each object uniquely identified
MIBsObject Identifiers
OID for System1.3.6.1.2.1.1
OID for System1.3.6.1.2.1.1
SNMPAGENTSNMPAGENT
Internet Activities Board (IAB) Administered
SNMP (11)SNMP (11)
Transmission (10)Transmission (10)
CMOT (9)CMOT (9)IP (4)IP (4)
Address Translation (3)Address Translation (3)
Interfaces (2)Interfaces (2)
System (1)System (1)
MIB-2 (1)MIB-2 (1)
EGP (8)EGP (8)
UDP (7)UDP (7)
TCP (6)TCP (6)
ICMP (5)ICMP (5)
Experimental (3)Directory (1) Management (2) Private (4)
Internet (1)
DOD (6)
Organization (3)
ISO (1)
...Unassigned (9118)Unassigned (9118)
Microsoft (311)Microsoft (311)
Enterprise (1)
Sun (42)Sun (42)
Apple (63)Apple (63)
Cisco (9)Cisco (9)
HP (11)HP (11)
IBM (2)IBM (2)
Proteon (1)Proteon (1)
Vendor Administered
Wellfleet (18)Wellfleet (18)
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 18© 2001, Cisco Systems, Inc. All rights reserved. 18© 2001, Cisco Systems, Inc. All rights reserved. 18
sysUpTime OBJECT-TYPE SYNTAX TimeTicks MAX-ACCESS read-only STATUS current DESCRIPTION "The time (in hundredths of a second) since the network management portion of the system was last re-initialized." ::= { system 3 }
What’s in a MIB?
MnemonicMnemonic
ParentParent OIDOID
How to Encode and Interpret this Variable
How to Encode and Interpret this Variable
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 19© 2001, Cisco Systems, Inc. All rights reserved. 19© 2001, Cisco Systems, Inc. All rights reserved. 19
Trap
Inform
Acknowledgement
Traps and Informs
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 20© 2001, Cisco Systems, Inc. All rights reserved. 20© 2001, Cisco Systems, Inc. All rights reserved. 20
Version 1Version 1 Version 2cVersion 2c Version 3Version 3
Informs NoNo YesYes YesYes
RMON/Event NoNo Yes*Yes* Yes*Yes*
Authentication CommunityCommunity CommunityCommunity UsersUsers
Privacy NoNo NoNo YesYes
IOS/CATOS SupportedSupported SupportedSupported SupportedSupported
NMS Support UbiquitousUbiquitous Pretty GoodPretty Good LimitedLimited
SNMP Version Differences
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 21© 2001, Cisco Systems, Inc. All rights reserved. 21© 2001, Cisco Systems, Inc. All rights reserved. 21
Example Tool using SNMP MIB Polling
• Monitors traffic load on network links based on SNMP statistics
• Generates real-time HTML traffic reports
• Monitor any SNMP variable you choose
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 22© 2001, Cisco Systems, Inc. All rights reserved. 22© 2001, Cisco Systems, Inc. All rights reserved. 22
Low LatencyLow Bandwidth
VoIP ERP Multimedia VPN Web/URL
Latency TolerantBursty Bandwidth
Network Must Provide Each Application With DifferentService Level Characteristics Simultaneously
Network Must Provide Each Application With DifferentService Level Characteristics Simultaneously
Traffic Management for Multiservice Networks
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 23© 2001, Cisco Systems, Inc. All rights reserved. 23© 2001, Cisco Systems, Inc. All rights reserved. 23
dod
mgmt
RMON
internet
mib-2
org
iso 1
.3
.6
.1
.2
.1
.16
RMONRMON 1 . 3 . 6 . 1 . 2 . 1 . 16 …1 . 3 . 6 . 1 . 2 . 1 . 16 …iso.org.dod.internet.mgmt.mib-2.rmon ...iso.org.dod.internet.mgmt.mib-2.rmon ...
tokenRing
eventscapture
filter
matrix
hostTopN
hosts
alarm
history
statistics
.1
.2
.3
.4
.5
.6
.7
.8
.9
.10
RMON-1 (RFC-1757)
RMON-1 (RFC-1757)
Token Ring (RFC-1513)Token Ring (RFC-1513)
probeConfig
usrHistory
alMatrix
alHostnlMatrix
nlHostaddressMap
protocolDist
protocolDir
.11
.12.13
.14.15
.16
.18
.19
.17
RMON-2 (RFC-2021)
RMON-2 (RFC-2021)
Remote Monitoring MIB
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 24© 2001, Cisco Systems, Inc. All rights reserved. 24© 2001, Cisco Systems, Inc. All rights reserved. 24
Example Tool using RMON Data
• Collects RMON data from intermediate devices
• Analyzes data for performance metrics
Netscout NGenius
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 25© 2001, Cisco Systems, Inc. All rights reserved. 25© 2001, Cisco Systems, Inc. All rights reserved. 25
NBARNetwork Based Application Recognition
• SW Feature in Routers
• Analyzes Data Portion of packets to identify applications
• Supports QoS deployment
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 26© 2001, Cisco Systems, Inc. All rights reserved. 26© 2001, Cisco Systems, Inc. All rights reserved. 26
Corp. HQ/Data CenterCorp. HQ/Data Center
SA Agent
Regional Aggregation
Regional Aggregation
Retail BranchRetail Branch
Field OfficeField Office
Retail BranchRetail Branch
Field OfficeField Office
• Synthetic traffic for various protocols
• Session Level Probe mechanism
• Generates availability and threshold traps
• Collects statistics
Service Assurance Agent
SA Agent
SA Agent
SA Agent
SA Agent
SA Agent
SA Agent
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 27© 2001, Cisco Systems, Inc. All rights reserved. 27© 2001, Cisco Systems, Inc. All rights reserved. 27
HTTPHTTP DLSwDLSw
Voice
Jitter
Voice
Jitter Packet
Loss
Packet
LossPathEcho
PathEcho
ICMPICMP
IOS-BasedService Assurance
Agent
TCPTCP
LatencyLatency
UDPUDP
LatencyLatency
DNS/
DHCP
DNS/
DHCP
Service Assurance Agent Operation Types
Increasing Service Value
Supports IP Precedence!!
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 28© 2001, Cisco Systems, Inc. All rights reserved. 28© 2001, Cisco Systems, Inc. All rights reserved. 28
Hop-by-Hop Response Time Report
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 29© 2001, Cisco Systems, Inc. All rights reserved. 29© 2001, Cisco Systems, Inc. All rights reserved. 29
SEQ 101
ACK 101SEQ 102SEQ 103SEQ 104
ACK 104SEQ 105
ACK 105
Example: FTPExample: FTPIdentify Application
Response TimeResponse Time
Packet Level Measurement
CNetwork Flight Time
Server LatencyClient Latency
Application Level Response Time
NNTPNNTP
COMPUSRVCOMPUSRV NOTESTCPNOTESTCP
DLSW_RDDLSW_RD ORACLSQLORACLSQL
DLSW_WRDLSW_WR REALAUDREALAUD
DNS_TCPDNS_TCP SMTPSMTP
DOOMDOOM SNA_TCPSNA_TCP
FTP-CTRLFTP-CTRL SOCKETSOCKET
FTP-DATAFTP-DATA SQLNET_NSQLNET_N
HTTPHTTP SUNRPC_TSUNRPC_T
HTTPSHTTPS TELNETTELNET
NB_DGM_TNB_DGM_T XWINDOWXWINDOW
NB_NS_TNB_NS_T
NB_SSN_TNB_SSN_T
NEWS_TCPNEWS_TCP
AOLAOL
SSSS
ART MIB Functionality
• TCP protocols only (1.0)
• Based upon well-known destination port
• Default protocols:
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 30© 2001, Cisco Systems, Inc. All rights reserved. 30© 2001, Cisco Systems, Inc. All rights reserved. 30
ART MIB Example of Reporting
• Web accessibleFor monitoring application and web flows from anywhere, anytime
• URL visibilityFor control of your site
• Proactive managementAlarm on responsiveness of the site or your mission critical applications
• Seamless real-time and historical
Current statistics with look back capability
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 31© 2001, Cisco Systems, Inc. All rights reserved. 31© 2001, Cisco Systems, Inc. All rights reserved. 31
Flow Data Exported to Management Application
Flow Data Exported to Management Application
NetFlow Defined
• Flows are defined by 7 keys:
Source Address
Destination Address
Source Port
Destination Port
Layer 3 Protocol
TOS byte (DSCP)
Input Interface
• Flows are unidirectional
• Flows are enabled on a per input-interface basis
• Flows can beconfigured “on-demand” or continuous
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 32© 2001, Cisco Systems, Inc. All rights reserved. 32© 2001, Cisco Systems, Inc. All rights reserved. 32
• Number of Flows• Flow Size Distribution• Number of Flows• Flow Size Distribution
• Packet Count• Byte Count• Packet Count• Byte Count
• Input Interface• Output Interface • Input Interface• Output Interface
• Type of Service• TCP Flags• Protocol
• Type of Service• TCP Flags• Protocol
• Source TCP/UDP Port• Destination TCP/UDP Port• Source TCP/UDP Port• Destination TCP/UDP Port
• Source IP Address• Destination IP Address• Source Prefix Mask• Destination Prefix Mask• Source AS Number• Destination AS Number
• Source IP Address• Destination IP Address• Source Prefix Mask• Destination Prefix Mask• Source AS Number• Destination AS Number
DeviceInterface
Application
RoutingandPeering
QoS
Usage
• Start Timestamp• End Timestamp• Call Duration
• Start Timestamp• End Timestamp• Call Duration
• Next Hop Address• Lost Datagrams• Next Hop Address• Lost Datagrams
TimeStamp
Usage
NetFlow Data Record per Flow
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 33© 2001, Cisco Systems, Inc. All rights reserved. 33© 2001, Cisco Systems, Inc. All rights reserved. 33
NetFlow Related Applications
Flow Profiling
Accounting/Billing
Network Planning
Network Monitoring
Flow Collectors
Flow Collectors
ManagementApplication
ManagementApplication
End-UserInformationEnd-User
InformationNetFlow/
Data ExportNetFlow/
Data Export
RMON ProbeRMON Probe
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 34© 2001, Cisco Systems, Inc. All rights reserved. 34© 2001, Cisco Systems, Inc. All rights reserved. 34
Evolution of Data Exchange Standards
• SQL interfaces subject to schema redefinition
• XML makes it easier to exchange data between computer systems
• Organizations rarely use a standardized set of tools
• Need to define a common data model!
• Structured data can be exchanged without APIs
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 35© 2001, Cisco Systems, Inc. All rights reserved. 35© 2001, Cisco Systems, Inc. All rights reserved. 35
CIM Schema v2.1CIM Schema v2.2CIM Schema v2.3
MOF Parser and Editor
CIM Specification V2.0
ExtensionSchema
SystemSystem
AppsApps
CoreCore
PhysicalPhysical(DEN)(DEN)
DeviceDevice
Logical Network
(DEN)
MetaModel
CIM Specification v2.1
UserPolicy(DEN)
• OutputHTML
SQL
Visio
ASCII
CIM Specification v2.2 CIM Schema v2.4
QoS(DEN)
IPSec(DEN)
DEN LDAP Mappings
CIM Components
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 36© 2001, Cisco Systems, Inc. All rights reserved. 36© 2001, Cisco Systems, Inc. All rights reserved. 36
Transporting CIM: XML!
• XML = eXtensible Markup Language
• Over HTTP, XML enables access toCIM objects
• Enables mixed vendor, distributed server environments!
<XML>CIM Data</XML><XML>CIM Data</XML>
HTTP/HTTPSHTTP/HTTPS
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 37© 2001, Cisco Systems, Inc. All rights reserved. 37© 2001, Cisco Systems, Inc. All rights reserved. 37
XML Components
• What makes up XML?
• XML document
• XML interpreter or parser
• Document Type Definition (DTD)
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 38© 2001, Cisco Systems, Inc. All rights reserved. 38© 2001, Cisco Systems, Inc. All rights reserved. 38
CIM
//////////////////////////////////////////////////////// // Device: nmcpw1601.cisco.com //////////////////////////////////////////////////////// instance of DEN_NetworkElement { DeviceId = "133"; CommonName = "nmcpw1601"; DNSName = "cisco.com"; Description = "";
CIM
CIM Example: Inventory Data
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 39© 2001, Cisco Systems, Inc. All rights reserved. 39© 2001, Cisco Systems, Inc. All rights reserved. 39
Agenda
• Motivation for Network Management
• Evolution of Basic Technologies
• Designing for Network Management
• Best Practices
• Policy Management
• Summary and Recommended Reading
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 40© 2001, Cisco Systems, Inc. All rights reserved. 40© 2001, Cisco Systems, Inc. All rights reserved. 40
Designing for Management Redundant Infrastructure
• High availability management
• Completely separates management from user data
• Management link is in separate subnet, VLAN, and switch
• Higher assurance for management data delivery during congestion or convergence
SNMP Manager
10.1.100.12 10.1.100.13 10.1.100.14
10.1.100.10 10.1.100.11
10.1.100.15
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 41© 2001, Cisco Systems, Inc. All rights reserved. 41© 2001, Cisco Systems, Inc. All rights reserved. 41
Management Station Performance
• How fast is fast, and how slow is slow?
• Check Browsers, Virus Scan Options, Java Releases….
• Customize Views
• Server CPU, Client RAM (and CPU)
• Be aware of the number of managed devices
• Be aware of the number of functions
• Don’t ask for information you won’t look at!
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 42© 2001, Cisco Systems, Inc. All rights reserved. 42© 2001, Cisco Systems, Inc. All rights reserved. 42
Service Mgmt
CiscoSecure HP NMM
QoS Policy Manager
DNS / DHCP
CiscoWorks Blue
Cisco VoiceManager
Integration and Growth Issues
• What happens when you need to run more applications?
Is the OS supported?
CPU or memory constraints?
Conflicting databases?
Conflicting ports used?
Multi-user access?
CustomerSpecific
MRTG
CW2000
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 43© 2001, Cisco Systems, Inc. All rights reserved. 43© 2001, Cisco Systems, Inc. All rights reserved. 43
Centralized Network Management Architecture
Enterprise Network
Site C
Site B
Site A
Centralized Database
Central NMS
NMS Queries
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 44© 2001, Cisco Systems, Inc. All rights reserved. 44© 2001, Cisco Systems, Inc. All rights reserved. 44
Hierarchical Network Management Architecture
Enterprise Network
Local Query
Local Query
Local Query
Client NMS
NMS Communication
Site C
Site B
Site A
ClientNMS
Client NMS
Central DB
Server NMS
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 45© 2001, Cisco Systems, Inc. All rights reserved. 45© 2001, Cisco Systems, Inc. All rights reserved. 45
Distributed Network Management Architecture
Enterprise Network
Local Query
Local Query
Local QueryLocal DBC
Peer NMS
NMS Communication
Site C
Site B
Site A
Local DBC
Peer NMS
Local DBC
Peer NMS
Local DBC
Peer NMS
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 46© 2001, Cisco Systems, Inc. All rights reserved. 46© 2001, Cisco Systems, Inc. All rights reserved. 46
Micromuse NetCool Architecture
G
Info ServerInfo Server
G
Trouble TicketTrouble Ticket
SNMPSNMPCMIPCMIP
M
ASCIIASCII(TL1)(TL1)
M
LogfilesLogfilesDBDB
M
APIAPI
M
FW-1FW-1
M
FusionFusion
M
ISMISM
M
NTSMNTSM
M
Motif/NT Desktop
Event List
Infoive View
WWW Server
Jeld
Web Browser
Event List
G RDBMS
Info ServerInfo ServerDE-DUPLICATION
CNM ViewCNM View
G
Automations
Actions Triggers
External actions
Internal actions
ReporterReporter
ImpactImpact
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 47© 2001, Cisco Systems, Inc. All rights reserved. 47© 2001, Cisco Systems, Inc. All rights reserved. 47
Internet OSS
Element Management and Network Management Framework
Integrated Mgmt Applications
Network Elements & Intelligent Agents
…
Intelligent Network Services
Au
tho
rization
Au
thn
tication
Pro
vision
ing
Fau
lt Mg
r
DH
CP
DN
S
Qo
s po
licy
Billin
g S
rv
Directo
ry
Ban
dw
idth
Integration BUS/Middleware Services
Integration Bus/ Middleware / Northbound APIs
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 48© 2001, Cisco Systems, Inc. All rights reserved. 48© 2001, Cisco Systems, Inc. All rights reserved. 48
Agenda
• Motivation for Network Management
• Evolution of Basic Technologies
• Designing for Network Management
• Best Practices
• Policy Management
• Summary and Recommended Reading
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 49© 2001, Cisco Systems, Inc. All rights reserved. 49© 2001, Cisco Systems, Inc. All rights reserved. 49
Monitor Critical Links – forget the rest
• Define key infrastructure aggregation ports ( )
• Setup statistics collection (RMON)
• Monitor “away” from the core
• Enable traps for link failure and thresholds
• Monitor for performance and fault conditions
Remote Offices
Corp Network
Servers
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 50© 2001, Cisco Systems, Inc. All rights reserved. 50© 2001, Cisco Systems, Inc. All rights reserved. 50
NTP helps correlate information
• Defined in RFC 1305
• Used to synchronize system clocks on network devices with an authoritative time source
• Essential for manual troubleshooting via Syslog
• Client/Server unicast or multicast options
NTP
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 51© 2001, Cisco Systems, Inc. All rights reserved. 51© 2001, Cisco Systems, Inc. All rights reserved. 51
Use two Clock sources
NTP
RTR Ac75xx
RTR B
RTR 1 ... ... RTR n
Authoritative Clockntp.nasa.gov (143.232.55.5)
ntp server 143.232.55.5ntp server 204.34.198.40ntp peer 192.168.100.2ntp peer 192.168.100.3ntp update-calendar
RTR C
ntp server 143.232.55.5ntp server 204.34.198.40ntp peer 192.168.100.1ntp peer 192.168.100.3
ntp server 143.232.55.5ntp server 204.34.198.40ntp peer 192.168.100.1ntp peer 192.168.100.2
Authoritative Clocktick.usnogps.navy.mil (204.34.198.40)
ntp server 192.168.100.1ntp server 192.168.100.2ntp server 192.168.100.3
STRATUM 2
STRATUM 3
Time Negotiation Time Negotiation
Internet
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 52© 2001, Cisco Systems, Inc. All rights reserved. 52© 2001, Cisco Systems, Inc. All rights reserved. 52
AAA – who can do what?
• Authentication, Authorization, and Accounting
• TACACS+ available in routers and switches—allows for centralized username/password/priv administration
• Removes the requirement of having to config hundreds of routers/switches when a user leaves
• Allows for accountability when each user has their own login ID
• AAA implementation case study
http://www.cisco.com/univercd/cc/td/doc/cisintwk/intsolns/aaaisg/index.htm
AAA/TACACS+
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 53© 2001, Cisco Systems, Inc. All rights reserved. 53© 2001, Cisco Systems, Inc. All rights reserved. 53
DNS – know what you’re looking at
• At a minimum put your router loopback addresses and switch sc0 interface address in DNS
• Set hostname to match DNS nodename
• Forward/reverse lookups for interfaces?
DNS
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 54© 2001, Cisco Systems, Inc. All rights reserved. 54© 2001, Cisco Systems, Inc. All rights reserved. 54
Limit SNMP Abuse
• SNMP should only be accessible to NMS
• Use ACLs where appropriate
• Use SNMPv3 where available
• Limit available SNMP Data with “Views”
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 55© 2001, Cisco Systems, Inc. All rights reserved. 55© 2001, Cisco Systems, Inc. All rights reserved. 55
Community Strings Privacy
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 56© 2001, Cisco Systems, Inc. All rights reserved. 56© 2001, Cisco Systems, Inc. All rights reserved. 56
SNMP Views
enterprises
rttmon
interfaces
bgp
ipRouteTable
mib-2
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 57© 2001, Cisco Systems, Inc. All rights reserved. 57© 2001, Cisco Systems, Inc. All rights reserved. 57
SNMP Views
enterprises
rttmon
interfaces
bgp
ipRouteTable
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 58© 2001, Cisco Systems, Inc. All rights reserved. 58© 2001, Cisco Systems, Inc. All rights reserved. 58
Conserve Bandwidth
snmpwalk ofipRouteTable
Snmp-server ViewEnabled
Cisco 2621 w/ 64MB RAM and 4000 routes (EIGRP)snmpwalk would have run for 25 ½ minutes unrestricted
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 59© 2001, Cisco Systems, Inc. All rights reserved. 59© 2001, Cisco Systems, Inc. All rights reserved. 59
Conserve Device Resources
• Restrict access to certain MIBs
• Some NM apps poll IP route tables and ARP caches—this can cause high CPU load on low-end routers with many route entries
• Use “snmp-server views” statements
SNMP Access
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 60© 2001, Cisco Systems, Inc. All rights reserved. 60© 2001, Cisco Systems, Inc. All rights reserved. 60
Polling vs. Notifying
• Polling: NMS asks for status
• Notifying: Device actively notifies NMS of problems
• Two types of notifications
Trap—unreliable, no state retained
INFORMs
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 61© 2001, Cisco Systems, Inc. All rights reserved. 61© 2001, Cisco Systems, Inc. All rights reserved. 61
• Be Careful!
• Set polling interval wisely
• Bandwidth issues on lower speed links
Cost of Queries
Network
% of Bandwidth Utilized
Polling Interval in Seconds
# o
f P
oll
ed S
tati
on
s
10 50 25 12.5 8.3
20 100 50 25 16
30 150 75 37 25
5 10 20 30
Example:1 manager, multiple managed devices64 Kb access link1 Request = 1KB packet (avg.)1 Poll = getreq + getresp = 2KBAssume 1 object polled/managed device
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 62© 2001, Cisco Systems, Inc. All rights reserved. 62© 2001, Cisco Systems, Inc. All rights reserved. 62
Cost of Traps
• No queries
• But you may need to poll for other reasons (performance metrics)
• SMART polling engines can really make the difference!
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 63© 2001, Cisco Systems, Inc. All rights reserved. 63© 2001, Cisco Systems, Inc. All rights reserved. 63
Benefit of Traps
• Use trap-based polling
• Use RMON to define Traps
• Use RMON to set Thresholds
• Use RTT-Mon Traps for Timeouts, Thresholds, Connection Changes
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 64© 2001, Cisco Systems, Inc. All rights reserved. 64© 2001, Cisco Systems, Inc. All rights reserved. 64
WAN
Overload!Overload!
DeviceDuplicates
Limit the Amount of Information
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 65© 2001, Cisco Systems, Inc. All rights reserved. 65© 2001, Cisco Systems, Inc. All rights reserved. 65
Fault Correlation
Remove Duplicates and Correlate
WAN
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 66© 2001, Cisco Systems, Inc. All rights reserved. 66© 2001, Cisco Systems, Inc. All rights reserved. 66
Hierarchical Mechanisms
Fault Correlation
Fault Correlation
Fault Correlation
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 67© 2001, Cisco Systems, Inc. All rights reserved. 67© 2001, Cisco Systems, Inc. All rights reserved. 67
Security vs. Trust in the Network
• Ease of access vs level of security is always a tradeoff
• Every network management feature can be viewed as a security vulnerability
Manageabilty, Ease of Access Concerns
Security
Ease of Access
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 68© 2001, Cisco Systems, Inc. All rights reserved. 68© 2001, Cisco Systems, Inc. All rights reserved. 68
Management Traffic
• In-band clear text
• In-band encrypted
• Out-of-band
What Options for Securing It?
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 69© 2001, Cisco Systems, Inc. All rights reserved. 69© 2001, Cisco Systems, Inc. All rights reserved. 69
Management Protocol Security
• SNMP
• TELNET
• RCP
• HTTP/XML
• TFTP
• CORBA, other special/proprietary, etc.
Cleartext Transmissions
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 70© 2001, Cisco Systems, Inc. All rights reserved. 70© 2001, Cisco Systems, Inc. All rights reserved. 70
Medium Trust Environment
• Higher concern for protecting managed devices from unauthorized access
• Standard cleartext-based protocols may still be acceptable
• Restrict access to devices as appropriate
access lists / ip permit lists for SNMP, TELNET
AAA for device access via TELNET
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 71© 2001, Cisco Systems, Inc. All rights reserved. 71© 2001, Cisco Systems, Inc. All rights reserved. 71
Low Trust Environment
• Some protocols have secure option
SNMP: SNMPv3
TELNET: SSH
HTTP: SSL/HTTPS
RCP: SSH/SCP
• But what about ?
TFTP : ?
CORBA: ?
Encryption of Management Traffic Needed
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 72© 2001, Cisco Systems, Inc. All rights reserved. 72© 2001, Cisco Systems, Inc. All rights reserved. 72
Low Trust Environment
• IP Sec / VPN Tunnels
• Can cover ALL management protocols
• Useful for connections across public WAN between sites
• Possible consideration for management of individual devices (if all devices support IPSec)
Encryption of Management Traffic Needed
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 73© 2001, Cisco Systems, Inc. All rights reserved. 73© 2001, Cisco Systems, Inc. All rights reserved. 73
Network Management
•Network management subnet for all NMS hosts and tools
•Security point to control access to subnet
•Firewall
•VPN aggregation point
Firewall
NMS
Corporate Intranet
VPN
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 74© 2001, Cisco Systems, Inc. All rights reserved. 74© 2001, Cisco Systems, Inc. All rights reserved. 74
Firewall Issues
• Need to consider not only traffic between management workstation and devices, but also between management workstation and clients (management users)
• May be possible to filter based on ports
• Some products break—tools choose free ports at random (CORBA, some other client and server architectures)
Try telling firewall to permit larger port range from management station
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 75© 2001, Cisco Systems, Inc. All rights reserved. 75© 2001, Cisco Systems, Inc. All rights reserved. 75
Firewall Issues
• NAT—no general solution for SNMP
• Common workaround is multihome management station or DMZ when necessary for one server to manage both “inside” and “outside” addresses
NAT
DMZ
NMSOutside
Inside
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 76© 2001, Cisco Systems, Inc. All rights reserved. 76© 2001, Cisco Systems, Inc. All rights reserved. 76
Agenda
• Motivation for Network Management
• Evolution of Basic Technologies
• Designing for Network Management
• Best Practices
• Policy Management
• Summary and Recommended Reading
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 77© 2001, Cisco Systems, Inc. All rights reserved. 77© 2001, Cisco Systems, Inc. All rights reserved. 77
Define your Policies
• Policies are Goal Statements• Implementing Policies: Conditions and Actions• Conditions
Packet headerExternal conditionsUser
• ActionsFilter rulesEncryption requirementsQuality of service requirements
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 78© 2001, Cisco Systems, Inc. All rights reserved. 78© 2001, Cisco Systems, Inc. All rights reserved. 78
SyntheticSynthetic ObservedObservedSampling MethodSampling Method
Embedded AgentsEmbedded Agents External ProbesExternal ProbesCollection MethodCollection Method
Device/LinkDevice/Link End-to-End/PathEnd-to-End/PathScope of MeasurementScope of Measurement
UserUser NetworkNetworkPerspective of MeasurementPerspective of Measurement
Define Methods and Metrics
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 79© 2001, Cisco Systems, Inc. All rights reserved. 79© 2001, Cisco Systems, Inc. All rights reserved. 79
Corp. HQ/Data CenterCorp. HQ/Data CenterRegional
AggregationRegional
AggregationRetail
BranchRetail
Branch
Service Provider Domain 1
Service Provider Domain 2
Enterprise Domain
Enterprise Domain
Enterprise Domain
Other DomainsNetwork HardwareWorkstation HardwareApplication SoftwareEtc.
Defining Demarcations
SA Agent
SP1
SA Agent
SA Agent
SP2
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 80© 2001, Cisco Systems, Inc. All rights reserved. 80© 2001, Cisco Systems, Inc. All rights reserved. 80
Example Policy
If service is HTTPif destination is S
if source is Hservice level = Premiumpermit
else if source is N1 or N4permit
if source is N4use tunnel
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 81© 2001, Cisco Systems, Inc. All rights reserved. 81© 2001, Cisco Systems, Inc. All rights reserved. 81
Policy-Based Networking
Directory Enabled Networking - Why?
Network Device Layer
IP Routing Protocols
Operating System Services
Applications
OSPF
BGP4
PIM
PGM
L2TP MPLS other...
SAP
Oracle
Voice
Video
DistanceLearning
Conferencing
Name Resolution Location
Authentication Authorization
Directory
Operating System Services
Applications
SAP Call
Center
Voice
Video
DistanceLearning
Conferencing
Name Resolution Location
Authentication Authorization
Directory
DEN ServicesQoS
Voice
DNS
DHCP Security
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 82© 2001, Cisco Systems, Inc. All rights reserved. 82© 2001, Cisco Systems, Inc. All rights reserved. 82
Rapidly create, provision and deploy Rapidly create, provision and deploy advanced networking services on a per user advanced networking services on a per user basisbasis
Centralized management of network resourcesCentralized management of network resources
Single network logonSingle network logon
Personalized network servicesPersonalized network services
Easy access to advanced network servicesEasy access to advanced network services
Develop network-aware applications using Develop network-aware applications using standard development interfaces and toolsstandard development interfaces and tools
Protect mission-critical trafficProtect mission-critical traffic
Simplify and enhance network management Simplify and enhance network management and provisioningand provisioning
Benefits of Directory Enabled Networks
Enterprise Enterprise CustomersCustomers
Service Service ProvidersProviders
End-UsersEnd-Users
Application Application DevelopersDevelopersD
ire
cto
ry E
nab
led
Ne
two
rk S
erv
ice
sD
ire
cto
ry E
nab
led
Ne
two
rk S
erv
ice
s
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 83© 2001, Cisco Systems, Inc. All rights reserved. 83© 2001, Cisco Systems, Inc. All rights reserved. 83
Directory Protocols
• LDAP—standards-based query/update
• Kerberos—standard token-based authentication
• ADSI—Active Directory Service Interface (Microsoft AD)
• NDS/NDK—Novell Directory Services
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 84© 2001, Cisco Systems, Inc. All rights reserved. 84© 2001, Cisco Systems, Inc. All rights reserved. 84
CLI, SNMP, COPS
QPM Architecture
Data, voice, video applications
RSVP
LDAPv3
Directories• Active Directory,
Sun/Netscape, NDS,...
CiscoWorks 2000
Import device data
DiffServ
Cisco / 3rd party apps• Cisco CNR DHCP,...
QPM MgmtConsoles
Distributed QPMPolicy Servers
QPM Server• policy
database
Cisco Intelligent Network
• Policy & configuration management via CLI and COPS
• DiffServ and RSVP QoS standards
• Directory-enabledUser-based policies
Export policies
DEN / CIM compliant
• CiscoWorks 2000 device import
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 85© 2001, Cisco Systems, Inc. All rights reserved. 85© 2001, Cisco Systems, Inc. All rights reserved. 85
Common Open Policy Service
• Benefits of COPS
Policing & aggregate policies for RSVP
Multi-vendor, standards-based interoperability
Simplified support of new / upgraded devices
Policy abstraction of device specifics
• Standards
COPS-RSVP is a standard
COPS-PR not yet IETF RFC
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 86© 2001, Cisco Systems, Inc. All rights reserved. 86© 2001, Cisco Systems, Inc. All rights reserved. 86
Agenda
• Motivation for Network Management
• Evolution of Basic Technologies
• Designing for Network Management
• Best Practices
• Policy Management
• Summary and Recommended Reading
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 87© 2001, Cisco Systems, Inc. All rights reserved. 87© 2001, Cisco Systems, Inc. All rights reserved. 87
Summary
• Network Management is key to productivity
• Networks evolve – so do NMS technologies
• Design your NMS to support your goals
• Choose suitable architectures and tools
• Define Methods and Metrics
• Integrate
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 88© 2001, Cisco Systems, Inc. All rights reserved. 88© 2001, Cisco Systems, Inc. All rights reserved. 88
Recommended Reading
• Performance and Fault Management, Paul Della Maggiora et al. 2000, Cisco Press, ISBN 1-57870-180-5
• SNMP, SNMPv2, SNMPv3 and RMON 1 and 2, Third Edition, by William Stallings Addison Wesley Longman, Inc.
• Network Management: A Practical PerspectiveLeinwand and Fang Conroy
• Network Management: Principles and PracticeSubramanian
• How to Manage Your Network Using SNMP: The Networking Management PracticumRose and McCloghrie
NCM-1012973_05_2001_c1 © 2001, Cisco Systems, Inc. All rights reserved. 89© 2001, Cisco Systems, Inc. All rights reserved. 89© 2001, Cisco Systems, Inc. All rights reserved. 89
Some useful Links
• http://www.telecommagazine.com/
• http://www.osswatch.com/
• http://www.billingworld.com/
• http://www.tmforum.org/
• http://www.ietf.org/
• http://www.ietf.org/html.charters/wg-dir.html#Operations_and_Management_Area
• http://dmtf.org/
• http://www.simple-times.org/
• http://www.snmpworld.com/
• http://www.stardust.com/policy/index. htm
• http://dmoz.org/Computers/Software/Networking/Network_Performance/RMON_and_SNMP/
• http://joe.lindsay.net/webbased.html
• http://joe.lindsay.net/javamgmt.html
• http://netman.cit.buffalo.edu/index.html