Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Confidential – Internal Use Only
Legacy Microwave to IP Microwave Conversion
6/11/13
Confidential – Internal Use Only
• The Kansas City Southern Railway (KCSR) had a need to modernize an outdated microwave system in preparation for a PTC deployment
• Microwave system is made up of 48 sites than runs through 5 states
• Primarily paths run North/South from Kansas City to Beaumont and New Orleans
• Multi year project• Phase I included sites South of Shreveport• Phase II included sites North of Shreveport
Legacy to IP Microwave Conversion
Confidential – Internal Use Only
Phase I Tower Locations
3
Confidential – Internal Use Only
Phase II Tower Locations
4
Confidential – Internal Use Only
Legacy Microwave Backbone
5
• T1’s were used for back feed on the legacy system
160 MHz160 MHz
160 MHz
T1 Back Feed
Kansas City Data CenterShreveport
T1 Back. Feed
Confidential – Internal Use Only
• New system needed to be capable of handling PTC data transfer requirements
• New microwave radios needed to be implemented with failover capability in mind
• Tower infrastructure needed upgrading where for enhanced reliability
• Needed to be deployed with a minimal impact to the business
Key Objectives
Confidential – Internal Use Only
• Sites needed to be surveyed• Had to determine what equipment would stay after the
upgrade• A one for one comparison, existing endpoint/channel
interface to IP interface, was performed on the entire network • Will the remaining legacy equipment work with the new
equipment being installed
Interface considerations
Confidential – Internal Use Only
• There was legacy equipment that needed to remain post cutover not capable of communicating over IP
• Had to find an interface solution– Distributed Power (DP) repeaters– Wind speed indicators– Cameras– Base Communication Package (BCP) communication to backoffice.– Dispatch radios
Non IP Equipment
Confidential – Internal Use Only
DP Repeaters • Relays communication between locomotives in a train consist
• CPU for two zones located at master end, limited equipment located at remote end, each end equipped with one multichannel transmitter and four independent receivers at 450 Mhz
• OLD – Used E&M channels via mux equipment. Mod/Dem for data transmission, E&M for PTT and Channel Selection
• NEW – Duplicated configuration using E&M interfaces on routers.
DATE:SCALE:
DRAWN BY:REV: 4/11/13 gm
APPROVED BY:
D R AWING NO.:
7/12/12NONE
XXX-XXXXWIRING DIAGRAM
Rich Mt and HoratioDP Repeater
gm
Kansas City Southern
DP MSTRMDF
Rich Mt Horatio
0/1/0
CISCO
10 Base T0/1/1
0/2/0
0/2/1
SB
DEM
M
3C55A1
3C6R5A7T
8
3C95A2
3C10R5A7R
3C10T
3C135A3
3C14T
3C175A4
3C18R5A6T
3C20R
3C12R
3C14R
3C195A5
SB
DEM
M
SB
DEM
M
SG/E
D
M
SG/E
3C8R
3C7
3C6T
3C12R
3C11
3C12T
3C16R
3C15
3C16T
3C20R
3C19
3C20T
3C18T
3C14T
3C10T
3C8T
SG
MOD
SB
SG
MOD
E
SB
SG
MOD
E
SB
SG
MOD
E
SB
E
3C17DEM
CH 1
CH 2
CH 3
CH 4
Zone B Rx 1
Ch Sel Bi t 0
Zone B Rx 2
Ch Sel Bi t 1
Zone B Rx 3
Zone B Tx 4
Zone B Rx 4
PTT
MOD
MDF
-48V
24
Bits 11 = Ch 1 452.925Bits 01 = Ch 2 452.950Bits 10 = Ch 3 457.925Bits 00 = Ch 4 457.950
1011
1516
23
78
45
12
18
19
SB
Signal is received at Horatio on the corrsponding ch. according to the chart. The signal is sent to Rich Mt. Master DP repeater. Repeater then selects TX freq via ch. select bits 0 & 1. See chart. Then master keys tx via M lead on ch 4. Signal goes out on mod of ch. 4
and is transmitted on the correct frequency.
3C6T
175A6R
5A8
3C18T
-48V
6B4R
Tx Key Return
Ch Select Return
SG3C8R
SG
+48V GND
K3
1
9
2/0/0 Chan 1 2/0/1 Chan 2 2/ 0/ 3 Chan 4
+
-
+
-
+
-
6
4
8
11
13
16
5.6K1
9
6
4
8
13
5.6K1
96
4
8
13
5.6K
16
+
-
-48V
yellow yellow Green
PTTFreq Dat a bit "0"
Normally Off
F-9
Act ive Positive Act ive PositiveAct ive Posit ive
11
Active Positive
Freq Dat a bit "1"
Active Positive
16
11
K2K1
External Circuit Boiard Notes:
T0 DP Repeater T0 DP Repeat er T0 DP Repeater
All relays shown in the picked positionUsed at remote (zone B) location only.
RX Ch 123 DB
BIT 0Ch Sel
23 DB
5A7T
F8 1 Amp
SG
23 DB
23 DB
RX Ch 2
BIT 1Ch Sel
RX Ch 4
TX Ch 4
Tx Key
RX Ch 3
-48V
5A6T
5A4
5A6T+48V
5A3
5A2
5A7R
5A5
Ch 4
5A6R
DP slave
Act ive Positive
L M
N O
P S
L
M
N
O
P
S
J1 DB-25 12 pairJ6B DB-25 12 pair
5A6R
6A7T GND +V
SG +48V
5A8T
5A8R
SG +48V
SG +48V
Relays needed for required hard ground.
5A1
3C16R
CISCO
0/1/0
0/1/1
0/2/0
0/2/1
3D44D4
4D5
4D6
4D7
4D8
(FB3)
3D5
3D6
3D7
3D8
6B4T
3D4
4B213B21
4B223B22
4B233B23
4B243B24
PROT
PROT
PROT
PROT
4D33D3
4D53D3
4D73D7
4D93D9
Confidential – Internal Use Only
Wind Speed Monitor• OLD – Used a GDC
MUX channel on each end to transmit Asynchronous Data
• NEW – Now use a GDC ADT (Asynchronous Data Transport) module to convert to IP, bookend solution
APPROVED BY:SCALE: NONE
DRAWING NO.
BLOCK DIAGRAM
Kansas City SouthernDATE: DRAWN BY: RDO
XX-XXX-XXXX
Kansas City
Wind Speed Indicator
REV: 09/12/2012 RDO
Kansas City - Beaumont Bridge
Beaumont Bridge
01/25/2010
Console 4
RS-485
RS-232
RS-485 to RS-232
9600 Baud
4th Floor Comm Room
Young Model 05103Wind Monitor
Young Model 06201Wind Tracker
BBox ConverterRS-232 to RS-485 BBox Converter
Young Model 32400Serial Interface
RS-485
RS-2329600 Baud
MW Router
GDC ADT (SA)
Switch to Network
IP Network
GDC ADT (Shelf Mt)
Async N81
Async N81
P7
P1
P1
10.102.168.129 /28
NW B
NW B
Confidential – Internal Use Only
Video Camera• OLD – Used an
Enerdyne Codex and a GDC MUX channel on each end to carry video and control signals
• NEW – Now use a Pelco 300T video transmitter and a Pelco 300R video receiver to transport video and control signals via IP, bookend solutionAPPROVED BY:
SCALE: NONE
DRAWING NO.
BLOCK DIAGRAM
Kansas City SouthernDATE: DRAWN BY: RDO
XX-XXX-XXXX
Kansas City 12th Street
CCTV Camera over SMW IP
REV:
Camera
Light
Kansas City - Beaumont Bridge
Beaumont Bridge
04/23/2009
Console 4
CID Desk
CCTV
Tower
PC
Box
MONITOR
CCTVMONITOR
Enclosure
300T
300R
10 Base T
VideoRS-422
KBD
Video
Video
Video
RS-422
DVR
Video
KBD
4800 Baud
4800 Baud
General Settings
Video Settings
Pelconet 300T Settings at Beaumont Bridge:
Frame Skip Ratio: 2
CID PC runs Client software with DVR
110 VAC
Control
Control
4th Floor Comm Room
LDR
KBD set for Camera 1KBD AUX 1 controls light.
Set date and time.
Video Stamp: Neches River Bridge and date/time.
Field Mode: Interlaced
MW Router
10 Base T
P8
Intraframe Distance: 0Video Quality: ManualData Rate 100KPreset Parameter Name: High Res (4CIF)
Network SettingsIP Address Subnet MaskDefault Gateway :Video Transmission: TCP port 80
Com 14800 N 8 1 (controls pan/tilt/zoom)
9/12/2012 RDO
Switch to Network
Confidential – Internal Use Only
Centralized Traffic Control (CTC)• BCP located at the tower
sites communicate with WCP at signal locations
• Allows the centralized dispatch office the ability to control traffic flow remotely using signals
• Critical system that supports safe fluid train operations
12
160 MHz160 MHz
Dispatch Center
BCP BCP
WCP WCP
Confidential – Internal Use Only
Centralized Traffic Control (CTC)
• OLD – Used a channel on the GDC MUX to a synchronous serial port on the BCP
• NEW – Now use a router serial interface to a PPP (point to point protocol) serial interface on the BCP
SCALE: NONEAPPROVED BY:
REV: 08/22/12 RDO
DRAWING NO.
Kansas City Southern
CABLE DETAIL
DATE: D R AWN BY:
BCP Field LocationBCP WIRING DETAIL
gm
5
3
6
8
4
2
7
20
DB25M to RJ45
WHT
RED
BLK
YEL
BRN
GRN
ORG
BLU
XXX-XXX-XX
Safetran BCP
H3C / Hauwei
Tx D ata Out
R x Data In
RTSCTS
Not Used
Not Used
Not Used
Common
4
2
5
7
3
1
6
8
DB25M to RJ45 Jack Adapter
DB25M RJ45 JACK
J 1 Port
+12VDC
GND
J 2 Port
Serial Port
RS 232
Jack Adapter
Note:BCP RS232 port must be set to PPP Mcast
Use straight through cable
P/N : FA770 (Black Box)
5/8/2012
OR CiscoRouter
Confidential – Internal Use Only
Dispatch radios• Use Motorola Quantar
analog dispatch radios at tower locations
• OLD ‐ Communicated over E&M channel to Avtec outposts located in central office
• NEW ‐ Avtec outposts were moved to tower locations to support Scout deployment and allow for disaster recovery
SCALE: NONEA PPROVED BY:
REV:
DRAWING NO.
Kansas City Southern
CABLE DETAIL
DATE: D R AWN BY:
North MWOUTPOST WIRING DETAIL
RDO04/26/2012
8
6
9
14
7
3
12
15
DB25M to RJ45 Jack Adapter
WHT
RED
BLK
YEL
BRN
GRN
ORG
BLU
XXX-XXX-XX
AVTEC OUTPOST (IP to 4Wire)
4WIRE TONE CONTROL RADIODISPATCHER'S VHF
TO IP NETWORK
R AD IO GND
PTT COM
PTTCOR
R X LO
TX LO
TX HI
R X HI
4
2
5
7
3
1
6
8
} R AD IO R X AUDIO (0 bdm) } R AD IO TX AUDIO (0 bdm)
DB25M to RJ45 Jack Adapter
DB25M RJ45 JACK
R AD IO 1 PORT
ETHERNET
MDF
+12VDCEarth GND
GND
Typ ically Wht/Blu Pair Typ ica lly Wht/Grn Pair
P/N : FA770 (Black Box)
Confidential – Internal Use Only
Frequency Coordination• ComSearch performed
frequency coordination• All links were analyzed• This is an example data
sheet from Plain Dealing, LA to Shreveport, LA
• Helps determine possible interference issues with outside entities and legacy paths
• Found interference issues with some legacy paths, temporarily decreased new transmitter power until cutover was complete
Microwave Path Data Sheet Page 23 of 23 COMSEARCH
19700 Janelia Farm Boulevard, Ashburn, VA, 20147 (703)726−5651 www.comsearch.com
PCN Date: 08/19/2011 Prev Job Num: Job Number 110819COMSTH02 RCN Number: 11081917
Administrative Information PLAIN DEALI LA SHREVEPORT LACity/County /Bossier Shreveport/CaddoStatus / License Basis Engineering Proposal / PRIMARY OPERATION Engineering Proposal / PRIMARY OPERATION Call Sign KJN89 KJN88 Licensee Code S0047P S0047P Licensee Name Kansas City Southern Railway Compan Kansas City Southern Railway Compan Radio Service / Station Class MG −− Microwave Industrial/Business Pool FXO −− Fixed
Site Information Latitude (NAD 83) 32 ° 56’ 1 6.0 " N 32 ° 32’ 27.0" N Longitude (NAD 83) 93 ° 43’ 5 5.0 " W 93 ° 50’ 10.0" W Ground Elevation (m/ft−AMSL) 142.30 / 466.9 62.20 / 204.1 Antenna Structure Registration # 1054953 1054958
Path Azimuth (°) 192.533 12.477 Path Length (km / miles) 45.091 / 28.018
Transmit Antenna A64130 A64130 Manufacturer ANDREW CORPORATION ANDREW CORPORATION Model P8−65D P8−65D Gain(dBi) / Beamwidth(°) / Tilt(°) 42.3 / 1.30 / −0.28 42.3 / 1.30 / −0.02 Centerline (m / ft − AGL) 89.92 / 295.0 68.58 / 225.0
Receive Antenna Same As Transmit Manufacturer Model Gain (dBi) / Beamwidth (°) Centerline (m / ft − AGL)
Diversity Receive Antenna A64130 A64130 Manufacturer ANDREW CORPORATION ANDREW CORPORATION Model P8−65D P8−65D Gain (dBi) / Beamwidth (°) 42.3 / 1.30 42.3 / 1.30 Centerline (m / ft − AGL) 76.20 / 250.0 59.44 / 195.0
Radio Information TEMZ47 TEMZ47 Manufacturer CERAGON NETWORKS CERAGON NETWORKS Model IP10−6−T10−64QAM−4 IP10−6−T10−64QAM−4 Model Description IDU T 64QAM IDU T 64QAM Emission Designator / Modulation 10M0D7W 64 QAM 10M0D7W 64 QAM Loading 1 CH DIG 44730.000 1 CH DIG 44730.000 Stability (%) 0.001 0.001
Nominal Coordinated Maximum Nominal Coordinated Maximum Power (dBm) 32.0 32.0 Received Level (dBm) −38.3 −38.3 EIRP (dBm) 67.4 68.4
Fixed Loss: Tx / Common (dB) 0.0 / 6.9 0.0 / 5.9 Free Space Loss (dB) 142.1
Transmit Frequencies (MHz) 6785.0000H(5) 6625.0000H(5)
Confidential – Internal Use Only
T1 Interfaces • It was determined during 1 for 1 evaluation that legacy wireless T‐1 circuits needed to be maintained after cutover, supporting backup for leased T‐1 circuits
• The solution was to emulate T‐1 circuits across the ethernetnetwork
• This was accomplished using Cisco routers with T‐1 interfaces
• Main issue to overcome was determining the correct clocking environment for the T‐1s over multiple router hops
CISCO 2941 T1 Interface
APPROVED BY:SCALE: NONE
DRAWING NO.
BLOCK DIAGRAM
Kansas City SouthernD A TE : 04/04/2012 DRAWN BY: R D O
XX-XXX-XXX
KC and Sport Cisco 2941TDM T1 Interface Detail
REV:
IP Network
Kansas City 11th Street Shreveport
GD C TMS
GD C TMS
GD C TMS
GD C TMS
Cisco 2941Lpbk1
Cisco 2941
Lpbk1
A
A
A
A
A Adapter Deta i l (Black Box PN FA770)
C EM RX
STD CAT5 Patch
STD CAT5 Patch
STD CAT5 Patch
STD CAT5 Patch
C EM TX
Router
TX D ATA ATMS ACC
TX D ATA BR X D ATA AR X D ATA B
T
T
R
R
RJ45 F DB25 M
142
42
5
1
163
T1 0/0C EM 0/0CLK Line
T1 0/0C EM 0/0CLK INT
T1 0/1C EM 0/1CLK Line
T1 0/1C EM 0/1CLK INT
ROUTER
C L OCK SOURCE
BLUORGREDGRN
Confidential – Internal Use Only
PBX Tie Trunks
• Legacy tie trunks using T‐1 had to be supported with the Ethernet solution
• We used a Cisco call manager with T‐1 interfaces to convert the legacy T‐1 trunks to IP
• Analog sections of the MW network already had MPLS circuits and VOIP phones, now supported via IP microwave
APPROVED BY:SCALE: NONE
DRAWING NO.
Kansas City SouthernDATE: 08/2 7/ 20 12 DRAWN BY: R D O
XX-XXX-XXX
SHREVEPORT
Interface Detail
REV:
IP Network
Shreveport
PBX T-1 1
Cisco ?
Cisco 2911
T1 0/0
T1 0/1
PBX T-1 2
Kansas City Call Manager
11th Street
12th Street
Cisco 4507
g 0/0
150B
12
4
B4
W
5
O
21
5 W
WO
BW
17
18
21
22
Cisco 2911 Voice Gateway
Blade 7Port 7
Confidential – Internal Use Only
Equipment Racks• Rack layout drawings
were made before any physical work
• Different layouts required based on building type and equipment requirements
• Height to top of rack kept consistent for measurement of flex waveguide needed
• Provided for uniform installation
FUSE PANEL
SHELF
RFU RFU RFU RFU
H3C Router
ALARM UNIT
CABLE MGMT
Rep MW Rack Site Mux RackS'port MW Rack
N/W N/W S/E S/E2 21 1
IDU S/E
RFU RFU
MW Rack
21S/E S/E
BUILDING ALM
RF PANEL
RFU RFU
21N/W N/W
PATCH PANEL
FAN ASSY
CABLE MGMT
222324
192021
4546
43
PATCH PANEL
17
15
FUSE PANEL 46FUSE PANEL 46FUSE PANEL 46
IDU N/WIDU S/E
RF PANEL
CABLE MGMT
FAN ASSY
1718
RFU RFU RFU RFUN/W N/W S/E S/E
1 12 2
IDU N/WIDU S/E
RF PANEL
CABLE MGMT
FAN ASSY2021
15
22
31
IDU N/W
RF PANEL
CABLE MGMT
FAN ASSY
IDU N/W
RF PANEL
CABLE MGMT
FAN ASSY202122
31
18
16
14
222324
32
19
1618
Confidential – Internal Use Only
• Researched radio manufactures to understand their interface options
• Technical considerations such as path reliability projections or bandwidth limitations and frequency limitations eliminated some vendors
• Selected Ceragon IP‐10 all indoor, 32 dbm TX, native Ethernet, 64QAM
IP Radios
Confidential – Internal Use Only
• Phase I– General DataComm (“GDC”) multi‐plex units were being used for re‐
routing and T1 interfaces on the legacy microwave system– Chose to use Huawei AR 4549 routers recommended by GDC for the IP
routing
• Phase II– KCSR converted to Cisco core network devices before this phase began– Implemented Cisco 3925 routers instead of the Huawei routers in the
first phase– Utilize Cisco certified network engineers on staff
Network Routers
Confidential – Internal Use Only
• Advertised in industry magazine to solicit vendors• Generators needed to work with current transfer switches
where possible• Sized for 25% capacity growth• Needed a minimum of 3 alarm contacts (generator run, loss of
AC power, fail to start)• Outdoor model (current indoor units overheated and were
starved for oxygen if building ventilation failed)• We had used Generac back up power systems in the past and
were happy with the units
Generators
Confidential – Internal Use Only
GeneratorsNew Generac QT025A 25kWGenerators replaced
Confidential – Internal Use Only
• A site package, containing all necessary equipment, hardware, and drawings was created and delivered to each location to assist technicians during the install and cutover
• All cabling, power connections, and grounding terminations were completed before equipment was put in service
• Racks were staged at field locations to minimize down time• Utilized diversity antennas during cutover
– One set of antennas used for the legacy radios, second set used for new radios
– Kept the legacy microwave in service until cutover complete
Installation
Confidential – Internal Use Only
• Equipment configured and tested in lab environment
• Equipment racks were built in central office
Installation
Confidential – Internal Use Only
Site Deployment• Ceragon IP‐10 radios• 1 + 1 HSB configuration
Confidential – Internal Use Only
Antenna Configuration During Cutover
26
New Digital Path
Old Analog Path Old Analog Path
New Digital Path
Confidential – Internal Use Only
Space Diverse Configuration Post Cutover
27
New Digital Path
Space Diversity Paths
New Digital PathMain Antennas
Diverse AntennasSpace Diversity Paths
Confidential – Internal Use Only
Tower with space diverse antennas
Confidential – Internal Use Only
IP Communications backbone ‐Microwave
• Replaced the T1 point to point back feed circuit with MPLS
• Added flexibility for routing traffic to multiple locations if needed
29
160 MHz 160 MHz 160 MHz
MP
LS B
ack Feed
Data Center Shreveport
Confidential – Internal Use Only
Phase I Timeline
215Feb ‘10 Mar ‘10 May ‘10 June ’10 Sep ‘10 Jan ‘11Aug ‘09 Dec ‘09 Jan ‘10
Phase I kickoff
Get vendor quotes for microwave radios and network routers
Complete configuration of radios, routers and other support equipment
Site preparation for generators complete
Generator installs complete
Generators arrive
Microwave radios and network equipment arrive
Back feed circuits installed
Complete assembly of racks
2015????? ??????? ??????Feb ‘11 Mar ‘11 ??????
Start generator installs
Lab testing complete, equipment move to field locations
Field installations complete
Start cutover to IP radios
Cutover Complete
Old equipment removed from field locations
Phase I Complete
Confidential – Internal Use Only
Phase II Timeline
215Aug ‘11 Sept ‘11 Dec ‘11 Jan ’12 Feb ‘12 March ‘12April ‘11 June ‘11 July ‘11
Phase II kickoff
RFP’s for generators, network routers and microwave radios goes out
Back feed circuits installed
Vendor selection of radios, routers and generators. Prepare sites for generators.
Microwave radios arrive
Network routers arrive
Start generator installs
Complete assembly of equipment racks
Generator installs complete
Project Complete 2015July ‘12 Aug ‘12 –
Jan ‘13 Jan ‘13 Mar ‘13 April ‘13April ‘12 May ‘12 June ‘12
Site preparation for generators complete. Generators arrive.
Complete configuration of microwave radios, routers and other support equipment
Lab testing complete. Equipment moved to field locations
Field installations complete
Waiting for completion of packet switch replacement project
Start cutover to IP radios
Cutover Complete
Old equipment removed from field locations
Confidential – Internal Use Only
• Gained an understanding of Ethernet microwave radios– Reduced latency– Better bandwidth utilization– Scalable
• Realized that the data network had to be prepared so it could handle “communications” traffic
• Gained an understanding of dynamic routing (OSPF). Before this project we only used static routing in the network.
• Realized the importance of getting design help from data network engineers. Didn’t have adequate network engineering support on the first phase.
Lessons Learned
Confidential – Internal Use Only
• Not all legacy equipment that had to say on‐site was IP capable and not all vendors offer IP solutions
• Had to design interfaces between some legacy equipment and new IP radios
• Had to learn new diagnostic and trouble shooting techniques• Network Engineering and Communication Engineering teams
have to work more closely together • New radios are IP devices on the network, have to become
more security aware
Lessons Learned