SLAC’s Networks

1

Prepared by: Les CottrellSLAC, for SLAC Network & Telecommunications groups

Presented to Kimberley ClarkeMarch 8th 2011

SLAC’s Networks

Outline• Phone upgrade

• Core network & offsite connections

• Cell phone coverage, mobility

• Wireless, visitor subnet

• Monitoring LAN & WAN

• Gigamon

• VPN upgrade

• IPv6, IPAM

• Conclusions2

Philosophy• Support getting the science done (safely)

– The science is the mission

• Uniformity of design (where possible)– Define standardized solutions & apply repeatedly– Limit vendors, technologies used– Leverage existing OCIO staff expertise

• Engineered for robustness (e.g. redundancy)– OCIO is not staffed for 24/7 coverage– “Throwing smart (dedicated) people at issues” works as long

as you do not throw them too often

• Powerful, easy to use monitoring3

Central phone system• Designed for low cost ($15/phone/month) , high reliability (1

unscheduled system fail in 22 years – loss power)

• End of life: parts are 1988 vintage, last major update 2000– 4000 phones, ~ 50% are non user (e.g. wall, conference room,

FAX, emergency …, so can stay analog)• Evolutionary upgrade phone system using existing

infrastructure (phone sets, closets, UPS, cabling) where possible to reduce costs and ensure maintainability while we:– Enable VoIP– Enable unified communications

• Email/vmail integration, presence, mobility, SMS …4

Network Scale• 70 major buildings,

• Single site, but lots of worldwide collaborations

• 300 layer 2 capable devices, 50 layer 3

• 15K end devices, 30K ports,

• Support:– science (open high performance worldwide), – business (protected, e.g. HR, finances ..), – controls & monitoring systems (local HVAC, accelerator), – desktops with local & internet access – visitors

5

6

Local Area network• Core network: highly reliable, supports 10Gbps

connections for: – high performance computing clusters, offsite, and buildings

(edge) switches, – Redundancy for power, routers, power supplies etc.

• Most wired desktops can be/are enabled for 100Mbps connections, we are upgrading to 1Gbps to the desktop for major buildings.

• Segmenting and rationalizing subnets– Private (RFC1918), Internet access, printers– Subnet set/switch, removing flat earth– Improved security, isolation of problems & performance

7

Accelerator Control network• The SLAC LINAC is operated via an IP based control

network.• About 4 mile long, about 80 individual network switches,

4000 switch ports• Routed centrally, dual redundant routers and links to each

switch• Uses IP multicast technology for real-time feedback and

control at 120Hz• Deterministic latency design: all traffic for each pulse must

be delivered within 1ms• Centrally designed and maintained: the entire network is

based on only two platforms: Cisco 6509 for core routing and switching, and stackable Cisco 3750G switches for access. 8

Wide Area Network Access• Off site links: multi 10Gbps links

– ESnet most production and also dedicated circuits (using MPLS) to BNL for ATLAS

– Stanford and CENIC/Internet2

• One physical path down Sand Hill Rd AT&T conduits with IRU – SRCF 2nd redundant path

• ACLs at borders

9

Mobility• WiFi: most buildings covered ~ 160WAPs

• Open access, not authenticated: ease of use

• No privileged access to SLAC resources

• Visitor subnet: no servers, block inbound connections

10

Cell phones• Coverage outside good: on site macro sites

for T-Mobile, Sprint, Metro-PCS and AT&T. Verizon going in across the street

• In buildings: most are penetrated from outside.– Installed BDAs in a few heavily shielded buildings– Pico cell in one area

• Pagers at end of life (atrophied ’60s technology)

11

Monitoring• Critical enabler for network and desktop admins• LAN: lookup routers, switches, ports, hosts, hosts for person, MAC & IP addresses, VLANs, provide:

– History, uilization, temp, cpu, power use, weather maps, idle ports, topology

• WAN: collaborations worldwide, E2E pingER & perfSONAR (multi NRENs)

• GigaMon: capture packets outside border on 10Gbps links and inspect

12

Security• Improved security via ACLs, firewalls,

• New VPN infrastructure going into place using IPSEC,

• Easy to use visitor network, reasonable security– private VLANs, – blocking of in-bound sessions and outbound SMTP– Blocking of outbound SMTP

•

13

Future• Developing new roadmap for service types with

differing security requirements:– science; business; guest/visitors; SLAC general

networks (desktops etc.); internal networks such as controls, data acquisition

• being ready to address IPv6 when DoE demands it– Network equipment IPv6 capable

• better IP address management with delegation,

• Mobile computing and unified communications

•  

14