Upload
pmi2011
View
235
Download
0
Embed Size (px)
DESCRIPTION
Citation preview
1
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
Deep Kamal Singh
27 September 2013
2
Introduction
• Changes in System are Inevitable
• Uncertainty, Variations, Deviations are Un Avoidable
• Prime objective of Business – Profit & Cost Control
• Factors that control cost of variation/deviation – Number of interdependent systems/services disrupted– Knowledge Gap between cross functional domains– Time spent in reaching a decision
• Exponentially proportional to Stake holders involved• Directly proportional to number of services/systems
disrupted
• Need of the hour, Business Need
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
3
• Organizations are facing a complex and
risky environment–A “do more with less”– “Tight” economic climate–Expanding global priorities–Necessity to enable innovation
The Strategic Approach
Identify All Services
Identify Revenue of
every service
Define Threshold
Form Crisis Management
Team
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
Identify All Impacted Services
Estimate outage & Calculate Revenue
Consolidate Options
Decide• Delay• GO Live• Fall Back
Planning
Execution - Handling Crisis
Organizations are facing a
complex and risky environment– A “do more with less”– “Tight” economic climate– Expanding global priorities– Necessity to enable
innovation
4Presentation Title
Knowledge Area Processes
Initiating Process Group
Planning Process Group Executing Process GroupMonitoring
&Controlling Process Group
Closing Process Group
Project Integration Management
4.1 Develop Project Charter
4.2 Develop Project Management Plan
4.3 Direct and Manage Project Execution
4.4 Monitor and Control Project Work 4.6 Close
Project or Phase 4.5 Perform Integrated
Change Control
Project Scope Management
5.1 Plan Scope Management 5.5 Validate Scope 5.2 Collect Requirements 5.6 Control Scope 5.3 Define Scope 5.4 Create WBS
Project Time Management
6.1 Plan Schedule Management
6.7 Control Schedule
6.2 Define Activities 6.3 Sequence Activities 6.4 Estimate Activity Resources 6.5 Estimate Activity Durations 6.6 Develop Schedule
Project Cost Management
7.1 Plan Cost Management 7.4 Control Costs
7.2 Estimate Costs7.3 Determine Budget
Project Quality Management
8.1 Plan Quality Management 8.2 Perform Quality Assurance
8.3 Perform Quality Control
Project Human Resource Management
9.1 Plan Human Resource Management
9.2 Acquire Project Team
9.3 Develop Project Team
9.4 Manage Project Team
Project Communications Management
10.1 Plan Communications Management
10.2 Manage Communications10.3 Control Communications
Project Risk Management
11.1 Plan Risk Management
11.6 Monitor and Control Risks
11.2 Identify Risks 11.3 Perform Qualitative Risk Analysis11.4 Perform Quantitative Risk Analysis11.5 Plan Risk Responses
Project Procurement Management
12.1 Plan Procurement Management
12.2 Conduct Procurements 12.3 Control Procurements
12.4 Close Procurements
Project Stakeholder Management
13.1 Identify Stakeholders
13.2 Plan Stakeholder Management
13.3 Manage Stakeholder Engagement
13.4 Control Stakeholder Engagement
PMI PMBOK™ V.5
5
Agenda
Introduction
Activities and Phases
Being prepared for uncertainty
Handling uncertainty
Theory in practice
In Conclusion
Q & A
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
6
Activities and Phases
• Complexity Assessment is first step to prepare for
uncertainty
• Concise definition of Decision makers for every stages
• Knowing Phases of activity – Strategic approach to take
next step under unplanned outage/failure depends upon the
phase– 3 phases of any activity in simplistic view
• Preparation • Execution• Operation
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
7
Activity Complexity
1 2 3 4 5 6 7
Complexity 1 4 9 16 25 36 49
Teams 1 2 3 4 5 6 7
5
15
25
35
45
55
Activity Complexity Vs. Systems Involved
Co
mp
lex
ity
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
• Complexity of activity– Exponentially proportional to
number of Inter-dependent systems involved
– Directly proportional to number of cross functional teams involved
– Directly proportional to Allocated duration versus tasks list ratio
8
Impact of failure during Activity phases
Presentation Title
Planning Phase
• Delayed Activity timelines
• Budget overhead
• Resources re-alignment
• Impact on cost to business
• Man hours invested in
preparation, development,
planning, testing etc.
• Business loss due to delayed
launch
• Delay overhead cost on
logistics
• Cascading delay on other
planned activities
Execution Phase
• Increased service downtime
Sudden revenue loss
• Impact on cost to business
• Direct revenue loss
• Customer dissatisfaction
Unpredictable future damages
• Increased calls to customer
care and support systems
• Resource working in
extended hours
Unpredictable future issues.
• If Fallback is decided Total
investment sunk.
Operation Phase
• Live Service downtime
Sudden revenue loss
• Impact on cost to business
• Direct revenue loss
• Customer dissatisfaction
Unpredictable future damages
• Increased calls to customer
care and support systems
• Unplanned additional Man
hour investment
• Resource working in
extended hours
Unpredictable future issues.
9
Agenda
Introduction
Activities and Phases
Being prepared for uncertainty
Handling uncertainty
Theory in practice
In Conclusion
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
10
Activity Planning
• The present day Approach– Preparation, Review and approval of detailed step wise
execution plan• Estimation of planned outage duration.
– Business teams analyzes and calculate total cost of activity to business • Including cost of planned outages, consequential costs,
logistics, HR overhead and so on.– Every step of activity is executed, monitored and controlled if
everything goes ‘as per plan’
• Shortcomings – We can not predict what may go wrong and thus
we can not be 100% ready for unplanned outages
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
11
Planning for uncertainty
Being prepared for something that can not be planned – During
Preparation Phase
• Identification of all system and services
• Evaluation of revenue earnings of each services
• Formation of Crisis management team
• Unanimous view of Project managers, Activity owners, Team
leads and Business teams over – Revenue potential of each service– All factors that impact business for each service
• Documentation of the unanimous view – “ Service Impact
Matrix”
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
12
Preparing Service impact matrix
• “Service impact matrix” - single document of reference to assess
impact to business during outage
– List all services which are going to be affected due to the
planned activity
– Include subscriber/user/consumer base of every service
– Include time based distribution of revenue earnings of each
service,
– Include direct and indirect cost to business for each service if
failure is encountered at any time duration
• Service Impact Matrix must be finalized, reviewed and approved by
all stake holders.
– Saves time to estimate cost of outage to business during crisis
– Prevents lengthy review/approval process
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
13
Example Service impact matrix
SNO Service NameSubscriber
Base(In Mn)
Hourly Revenue Potential [in 100K Rs.] Indirect cost to business
H01 H02 H03 H04 H… H21 H22 H23 H24Customer Care Overhead
Dependent Services impact
Customer Satisfaction Impact
1 Voice Calls 14.34 …
1.1 Local Home network 14.34 …
1.2 Local cross network 14.34 …
1.3 National Long Distance 09.60 …
1.4 International Long Distance 00.12 …
2 SMS 14.34 …
2.1 Local 02.21 …
2.2 National 02.21 …
2.3 International 00.01 …
3 Data Usage 01.90 … 4 Paper Recharges 08.14 … 5 ETOPUP 11.89 … 6 USSD Services 14.34 …
6.1 Subscriber Info 14.34 …
6.2 Subscription management 04.30 …
6.3 VAS Services 01.80 …
7 Subscriber Life Cycle 14.34 …
7.1 Subscriber Activation 0.03 …
7.2 Subscriber Churn 00.01 …
7.3 Service management 00.01 …
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
14
Agenda
Introduction
Activities and Phases
Being prepared for uncertainty
Handling uncertainty
Theory in practice
In Conclusion
Q & A
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
15
Identifying affected services and subscriber base
In case of any unexpected outage we need to find out total impact
to business, thus
• First and foremost, get update from all teams and consolidate
list of services that are affected.
• Classify level of disruption for each impacted service– Intermittent, Partial or Total
• Estimate outage duration and dependency factors for the
impacted service
• Consolidated outage stats for each impacted service in a single
view for comparison and correlation.
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
Considering the revenue impacting factors associated with the activity, below equation can be deduced
If
R(S1), R(S2)…..R(Sn) = Avg. revenue per user per minute of Service1,2,…n
O(S1),O(S2)…..O(Sn) = Total estimated outage duration of Service1,2,…n
Sb(S1),Sb(S2)….Sb(Sn) = Estimated percentage of affected subscriber base during outage of Service1, 2,….n
Su(S1), Su(S2)…..Su(Sn) = Total users of Service1, 2, ….n
Then
Total loss to business during service outage =
∑ [ R(Sn) x O(Sn) x Sb(Sn) x Su(Sn) / 100 ]
16
Deciding next course of action
• Identify impacted services
• Calculate Revenue Impact– Consolidate outage stats of all affected
services– Correlate available data with corresponding
details in Service Impact Matrix
• Assess Variable and Dependency factors
• Mathematically deduce all possible next steps
to formulate “The Decision Matrix”
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
17
Formulating Decision matrix
• The Decision matrix – Mapping of Options vs. Risk vs. Revenue Impact vs.
Variance factor – Crisis Management team to prepare
• Cross functional managers and tech leads to tabulate
domain specific next step of actions
• Categorization of options– Track 1 :: Go-Live– Track 2 :: Delay– Track 3 :: Fall back
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
18
Finalizing Decisions
• Ignore – impossible, unattainable, high risk options– Rule out options which are
• least possible, • have highest risk, • have highest number of variable factors
• Final time estimation– Get ‘Time to implement’ from respective teams– Readiness Confirmation – technically and logistically
• Final out put of crisis management team:-
Decision matrix with final concluded set of options and details of
‘time to implement’ presented to apex decision makers
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
19
Agenda
Introduction
Activities and Phases
Being prepared for uncertainty
Handling uncertainty
Theory in practice
In Conclusion
Q & A
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
20
Case study : Activity Details
• Billing System upgrade in 14 telecom circles of a leading telecom operator in India
• The activity requires complete outage of billing system for 8 hours, Since billing
system stays unavailable, all connected services (as listed) also face subsequent
downtime:– Voice calls – Local / National Long distance / International long distance– SMS– Data Browsing– Real-time data charging– Recharges – Voucher recharge and E-topup– USSD– Unified Subscriber life cycle management – Activation, Churn, daily jobs, and other offline process.– Business reporting – MIS
• The complex activity involves changes at many functional systems – High level of coordination and control required between Cross functional
technical teams.– Due to numerous Inter-dependent systems, outage of one system or service
may result in consequential outage of other services also
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
21
Case study 1
• Subscriber : 5.4
• Activity: Core billing system upgrade
• Issue during execution: ETOPUP not working with new version– Important to note here that multiple rounds of testing on test-bed and rehearsals were
conducted which showed no sign of issues earlier.
• Technical teams started investigating and fix the issue ASAP.
• Business team kept waiting for updates from Technical team to decide next step.
• All possible next actions were greatly debated without reaching a final conclusion– ETOPUP : DELAY
• Given estimated time to fix the issue was 2 Hours– Revenue assurance team : FALL BACK
• Cancelling complete activity and reinstating old version billing system– Billing system team : GO LIVE
• leaving ETOPUP team to fix “their” issue separately
• Business team: absence of conclusive data No decision was taken
• Finally after 95 minutes of extended downtime, ETOPUP issue was resolved and then upgraded
version of billing system was made live.
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
22
Evolution of strategic Approach
• Learning from failure – What took time to reach a decision?– Lack of information to take a decision
• “What to do next” is Biggest Time eater– Lack of predefined delay threshold for critical milestones– Lack of designated managers to manage crisis
• Biased Decision Making is biggest Hindrance to consensus• Absence of a dedicated Crisis management team.
– Lack of clarity on activity owners, team leaders, business owners and decision makers• Dedicated team lead/managers always suggest biased options which
favor their own team– Lack of methodical approach to ensure quick decision making
The learning drawn from events of site 1 became foundation for evolution of
strategic approach which is discussed in this presentation.
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
23
Case study 2
• Subscriber Base : 9.28 million
• Activity Details: Same activity as Case study 1
• The Issue: Once new version of billing system was made live for test traffic– It was identified that some of call handling units were not functioning
• When call routed on those units voice calls, video calls, SMS, USSD were failing
– Other units were functioning without any issues
• Crisis management team swung in action and as a first step it was deduced
that 20% of units were not functioning– This meant Voice calls, SMS and USSD requests were impacted by 20%.
Important to note that the Crisis management team was doing all coordination
and fact gathering with cross functional technical and business teams, the
Technical teams were dedicatedly focused to resolve the issue.
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
Continued…
24
Case study 2
• Total Impact identified Service impact matrix referred Assessment– Revenue loss when Billing system is made live with 20% failures in 3 service– Cost of delaying the launch – Which meant 100% subscriber base will not be able to
use ANY service until the issue is fixed– Cost of fallback was also compared with above two costs
• 3 Tracks Options (Delay, Go Live, Fallback ) consolidated – Presented to Business owners and Function heads
• Decision Taken– Within the duration of 45 minutes allocated for system testing before launch –
Business team successfully reached a decision to remove those 20% units from network (so call will not land on them) and make the system Live as per schedule
Even with unexpected variations encountered during critical launch
schedule, the strategic approach helped reaching a right decision
methodically
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
25
Agenda
Introduction
Activities and Phases
Being prepared for uncertainty
Handling uncertainty
Theory in practice
In Conclusion
Q & A
Strategic approach to ascertain accurate decisions after unplanned service outage in Telecom operations
26
Identify impacted services
Identify impacted subscriber/user base• Estimate
probable service outage duration for each impacted service
Evaluate indirect, additional and future cost to
business due to service outages
Formulate list of possible next steps
or possible options.
Identify variable (uncertain) factors
and Risk associated with each concluded
option.
Prepare Option vs. Risk vs. Revenue Impact vs. Variance factor matrix having each of 3 tracks• Track 1 - Go-
Live• Track 2 – Delay• Track 3 – Fall
backEvaluate time to
implement for every option.
Present Decision makers with Tracks
containing concluded set of
options and details of ‘time to
implement’ for each option.
Conclusion
27
Thank You
Deep Kamal Singh
Project Manager – BSS APAC
Comverse Network Sys.
+91-9971003585
Shay Shargal, PMP
CEO, RBS Projects
+972-52-23-11-11-0
www.rbsprojects.co.il
Past President – PMI Israel Chapter