Web-centric Business NRIC V – February 27, 2001

NRIC V Council Meeting - February 27, 2001

Focus Group 2.B1

Network Reliability - Data Reporting & Analysis

2

NRIC V – February 27, 2001 Agenda

•Governance

•Membership

•Charter

•Key Deliverables

Governance

Focus Group 2Focus Group 2Network ReliabilityNetwork ReliabilityChair: Brian Moir, Chair: Brian Moir,

ICAICA(202) 331-9852(202) 331-9852

Focus Group 2.B2on Data Reportingand Analysis forPacket Switching

Chair: Paul HartmanHartman Associates

(913) 893-9722

Focus Group 2.A2on Best PracticesPacket Switching

Chair: Karl Rauscuer(908) 582-5306

Focus Group 2.B1on Data Reporting

and AnalysisChair: P.J. Aduskevicz,

AT&T(908) 234-5790

Focus Group 2.A1on Best Practices

Chair: Rick HarrisonTelcordia

(732) 758-2013

NRIC V

4

NRIC V – February 27, 2001 Charter

I. Implement a voluntary one year trial with participation by Internet Service Providers,

CMRS, satellite, cable, and data networking service providers to alert National

Communications System/National Coordinating Center for Telecommunications

(NCS/NCC) of outages that are likely to have significant public impact.

Criteria for determining types of outages to be reported will be developed by each industry segment building

on the recommendations from NRIC IV.

The process for reporting data during the voluntary trial will be documented and implemented.

Analysis of the data from the trial will be conducted by a neutral third party.

Industry associations will be requested to encourage participation by their constituents.

At the completion of the trial period, an evaluation of its effectiveness will be conducted and a

recommendation made.

II. Evaluate outage-reporting requirements and guidelines currently used by wireline

carriers to improve the quality of outage reporting.

III. Evaluate and report on the reliability and availability of the PSTN utilizing NRSC

quarterly reports.

5

NRIC V – February 27, 2001 Membership

Name Company Name Company

PJ Aduskevicz AT&T Jim Lankford SBC OperationsEd Ballington BellSouth Norb Lucash USTAMichael Caloyannides Microtek Systems Spilios Makris TelcordiaRick Canaday AT&T Clyde Miller NortelWilliam Check NCTA Brian Moir ICAWayne Chiles Bell Atlantic Clayton Mowry SIABernie Farrell NCS David Opferman MotorolaPerry Fergus Booz Allen & Hamilton Gary Pellegrino Verizon WirelessJudy Glatz AT&T Bonnie Petti Verizon WirelessEd Hall ATIS Karl Rauscher LucentRick Harrison Telcordia Arthur Reilly Cisco SystemsJohn Healy Telcordia Ira Richer The Telesis GroupDean Henderson Nortel Networks Harold Salters PCIASuzy Henderson SBC Andy Scott NCTALynn Johnson Boeing Ron J. Shuster SprintRick Kemper CTIA John Todd NCSBill Klein ATIS Angela Wehmeyer AT&T Broadband

2.B1 - Data Reporting and Analysis Team

Industry Segment "Alert Situation" Criteria Outage Examples

CMRS WirelessA system level failure affecting wireless customers service and preventing new calls for 30 minutes or

more.

Wireless Mobile Telephone Switching Office (MTSO) failure.

Paging Multiple Switch isolated from PSTN

Cable TelephonyA failure that would cause a loss of cable

telephony service to 30,000 or more customers for 30 minutes or more. (reported through 63.100)

Failure of Head Ends (Class 5 switch, etc) which serves a minimum of 30,000 telephony

customers.

ISPsA failure that would cause a loss of service to a

large number of customers for 30 minutes or more.

A Domain Name Server (DNS) failure.

Satellite TelephonyA failure that causes loss of service to 30,000 or

more customers for 30 or more minutesLEO - Loss of multi customer shared earth

station.

GEO - Failure of transponders.

Data Networking Including Broadband Access

A failure that causes a loss of service to 30,000 or more customers for 30 or more minutes.

Multiple Asychronous Transfer Mode (ATM) Switch failures.

NRIC V Focus Group 2.B1Voluntary Trial CriteriaNRIC V – February 27, 2001

7

NRIC V – February 27, 2001 Key Deliverables

I. Implement a voluntary one year trial with participation by Internet Service Providers, CMRS, satellite, cable, and

data networking service providers to alert National Communications System/National Coordinating Center for

Telecommunications (NCS/NCC) of outages that are likely to have significant public impact.

Voluntary Trial Status

Recommendation to NRIC IV to proceed with trial predicated on FOIA exemption

February 22, 2001 MOU (Memorandum of Understanding) signed by NCS/NCC and FCC ensuring that

proprietary information provided under the agreement is protected against unauthorized use or disclosure

February 23, 2001 NRIC V Chairman sent letter to industry association representatives re-inforcing

commitment to voluntary trial and explaining MOU

March 9 Subcommittee Agenda item to confirm that all barriers to participation have been removed

Criteria for data scrubbing by NCS complete

Flowchart of reporting process documented by team and distributed to industry association

representatives

Subcommittee to review extending trial completion dates now that MOU has been signed

Make recommendation to

NRIC V on Outage Reporting

for those not governed by 63.100

8

NRIC V – February 27, 2001 Voluntary Trial Process

Outage Occurs

Determine if Outage Meets Vol.

Trial Criteria

Fax to NCS/NCCOn ( / / )

Create Initial Report within 3 Days –Mark Company

Proprietary

ConductRoot Cause

Analysis

IdentifyBest Practices

Fax Final Reportwith in 30 Days

NCS/NCC Logs Report

Scrub Data per Criteria Established

by 2.B1

Share Scrubbed Data (e.g. NIPC)

Handoff scrubbed data required by

2.A2 (Best Practices for Packet)

Does 2.B2 want to use data?

Analyze Scrubbed Data:•By Type/Segment•Impact•Frequency•Root cause for Commonality

Provide status at NRIC V Council

meetings

Make Recommendations

Monitor Progress

Concur or Provide Input on

Recommendations

Develop Final Report to Include Recommendations

Service Provider NCS/NCC Subcommittee 2.B 1

NRIC V

If yes

• Service Provider – Companies who provide CMRS, Satellite, Cable Telephony, ISP and Data Services• NCS/NCC (National Communication System/National Coordinating Center) - Joint industry-government organization and mechanism by which the federal government and telecommunications industry respond to national security and emergency preparedness (NS/EP) telecommunications service requirements.• Subcommittee 2.B1 – A subcommittee of the NRIC V Network comprised of representatives of industry associations, consultants, and providers for wireline and non-wireline services and equipment.• NRIC V – A federal advisory committee that is chartered to evaluate and report on the reliability of public telecomm network services in the US including the reliability of packet switched networks.

If no Local Root Cause

Analysis

Initial Final

Aggregate And

Send Data

Initial Final

File Final Report

Determine whether request to share data should

be made

Take NoFurtherAction

If yes

If no

Grant Permission

to Share Data?

If no

If yes

9

NRIC V – February 27, 2001

Box a: Reporting Carrier/Service Provider Box b: Contact Person

Box c: Telephone Number of Contact Person Box d: Start Date (mm/dd/yy of Incident)

Box e: Start Time of Impact (at outage location; 24-hour clock) Box f: Geographic Area Affected

Box g: Estimated Number of Customers Affected Box h: Types of Services Affected (if applicable)

Box i: Duration of Outage

Hrs. ____ Min. ____

Box j: Apparent or Know n Cause

Box k: Name of Equipment Involved [OPTIONAL FIELD] Box l: Type of Equipment Involved [OPTIONAL FIELD]

Box m: Specif ic Part of Netw ork Involved

Box n: Methods Used to Restore Service [OPTIONAL FIELD]

Box o: Steps Taken to Prevent Recurrence

NON-WIRELINE VOLUNTARY NETWORK OUTAGE

INITIAL REPORTING TEMPLATE

TEMPLATE

10

NRIC V – February 27, 2001

Final Report field Scrub Action

Reporting Carrier/Service Provider

Delete and create outage tag (Industry segment identifier, i.e., CMRS, Satellite,Cable Telephony, Data Services, and ISPs, plus a unique numerical identifier)

Contact Person Scrub Out

Start Date Unchanged

Start Time of Impact Unchanged

Geographic area affected Scrub Out

Estimated number of customers affected Unchanged

Types of Services Affected (if applicable) Unchanged

Duration of Outage (hours and minutes) Unchanged

Apparent or known cause Unchanged

Name of Equipment Involved [OPTIONAL] Scrub Out

Type of Equipment Involved [OPTIONAL] Scrub Out

Specific Part of Network Involved Unchanged

Methods Used to Restore Service [OPTIONAL] Unchanged

Steps Taken to Prevent Recurrence Unchanged

Root Cause and Trouble Found [OPTIONAL] Unchanged

Applicable Best Practice [OPTIONAL] Unchanged

Voluntary Trial Outage Report - Scrubbed Data Criteria

Agreement with Team – For Data Handoff

11

NRIC V – February 27, 2001 Key Deliverables

II. Evaluate outage-reporting requirements and guidelines currently

used by wireline carriers to improve the quality of outage reporting.

Revised Network Reliability Steering Committee Charter to Reflect intent to

“improve” reliability verses previous charter to “monitor”

Changed baseline for FCC Reportable Service Outages from first year to cumulative

going forward

Began conducting special analysis on detected trends (for example, single failure

affecting large geographic areas)

Began performing analysis on Emergency Services Outages

FCC working to implement 63.100 electronic filing and putting template into “user

friendly” format. (Due End of 2Q01)

12

NRIC V – February 27, 2001 Key Deliverables

III. Evaluate and report on the reliability and availability of the PSTN

utilizing NRSC quarterly reports.

Utilize NRSC to report data analysis from 63.100 process

Outage results assembled and analyzed by Network Reliability Steering

Committee

Quarterly and annual results for overall outages and sub categories available

on web site: http://www.atis.org/atis/nrsc/nrschome.htm

Will incorporate analytics from Voluntary Outage Reporting Trial for cable,

satellite, wireless, ISPs to extend Network Reliability Reporting

NRSC – 3rd Quarter Results

FCC Reportable Service Outages (by number of events)

4Q922Q93

4Q932Q94

4Q942Q95

4Q952Q96

4Q962Q97

4Q972Q98

4Q982Q99

4Q992Q00

0

10

20

30

40

50

60

35

49 48

40

2838

42

41

Nu

mb

er o

f O

uta

ges

Quarter

Red

Yellow

Green

3540

4542

40

47 4844

47

35

4448

52

41

484443

46

34

475243

38 39

55

Reliability Reporting

NRSC – 3rd Quarter Results

FCC Reportable Service Outages (by outage index)

4Q922Q93

4Q932Q94

4Q942Q95

4Q952Q96

4Q962Q97

4Q972Q98

4Q982Q99

4Q992Q00

0

100

200

300

400

500

600

700

800

900Red

Yellow

Green

211

504553

439 376

675

300287

Ou

tag

e In

de

x

Quarter

301346

412

331

401440

376

458

246

249

422

414

411362

510

326

519622

223286

345

481

305

470

364

Reliability Reporting

NRSC – 3rd Quarter Results

Procedural Error Attributed Outages(by number of events)

4Q922Q93

4Q932Q94

4Q942Q95

4Q952Q96

4Q962Q97

4Q972Q98

4Q982Q99

4Q992Q00

0

5

10

15

20

25

30

35

Red

Yellow

1113

6

Green

9

Quarter

Nu

mb

er o

f O

uta

ges

1416

1312

18 20 19

26

9

13

9

1113 13 13

11 12

10

20

15 16

25

18 19

15 11

1716

29

Reliability Reporting

NRSC – 3rd Quarter Results

Procedural Error Attributed Outages(by outage index)

4Q922Q93

4Q932Q94

4Q942Q95

4Q952Q96

4Q962Q97

4Q972Q98

4Q982Q99

4Q992Q00

0

50

100

150

200

250

300Red

Yellow

62

30

55

93

Green

90132

Quarter

Ou

tag

e In

dex

107

112

184

126

171

229

29

67

9882

122

129

54 53

119

82

198

130

207 206

7094

5656

133

172

143

Reliability Reporting

Network Reliability Steering CommitteeSignificant Results Through 3Q00

Historical average number of outages in the third quarter is 48.3, the highest of any quarter

3Q00 overall outage frequency was the highest ever (55) although it does not exceed third quarter control limits

Tandem Switch Outages and Procedural Error Outages had highest frequency to date

Tandem Switch CO Power and Procedural Error outages were all in the “Red” region

CO power outage frequency may be attributable to season effects as previously identified by NRSC which are statistically significant