18
Page | 1

Presentation by namit

Embed Size (px)

Citation preview

Page 1: Presentation by namit

Page | 1

Page 2: Presentation by namit

s

Title of the Paper

Next Generation Program and Project Managers – How the role is evolving in the era of Lean and Cloud computing

Theme

Mantra for Innovative Project Management

Keywords

Program Management, Project Management, Roles and Responsibilities, Change

Abstract

Next Generation Project/Program Manager in a Product Development setting. With the onset of SMAC

(Social, Mobile, Analytics and Cloud) there has been a new felt need for the project and program

managers to transform themselves. Delivering projects on time, with satisfactory quality and within budget

is no longer good enough. Project Managers need to go beyond the obvious and seek new ways to add

value to project teams and to the business. There are three ways in which project and program managers

can do this 1) By being Lean and Agile Champions - Doing development iteratively is now not a new

concept but making sure what is being developed right or not is becoming more and more import. In that

sense project managers need to not only imbibe lean principles but become Lean champions 2) By being

the Voice of the Customer - Often project managers are biased by technology challenges and the

development decisions taken/inspired are more from technology challenges. Project managers need to

think more in terms of what would be in the benefit of customers 3) By bringing in Analytics - Project

cycles no longer end at acceptance testing and shipping the product. What happens to the product after

the release is more important. How is the product getting used, what features are getting used, what

issues are frequently coming up etc. need to be collected and analyzed. The feedback collected needs to

be made part of the product life cycle itself.

Authors

Namit Agrawal – Senior Program Manager, Adobe

Sandeep Grover – Senior Program Manager, Adobe

Page | 2

Mantra for leveraging Emerging TrendsNamit AgrawalSenior Program ManagerAdobe Systems

Page 3: Presentation by namit

Table of ContentsIntroduction................................................................................................................................................3

Program Management Evolution at Adobe...........................................................................................3

Lean Development................................................................................................................................3

The Lean Cycle.................................................................................................................................4

Customer Delight..................................................................................................................................5

Data Driven Analysis............................................................................................................................6

Customer Lifecycle Funnel..............................................................................................................7

Cohort Analysis.................................................................................................................................7

Product Scorecards..........................................................................................................................8

Globalization Metrics........................................................................................................................9

Product Locale Analysis...................................................................................................................9

Forum Analytics..............................................................................................................................10

Support Cases by Customers.......................................................................................................11

Trial Downloads..............................................................................................................................12

Net Promoter Score Surveys.........................................................................................................12

Conclusion...............................................................................................................................................13

References..............................................................................................................................................13

Page | 3

Page 4: Presentation by namit

Introduction

This paper covers key trends in Program Management that are evolving at Adobe as the company

transforms from a shrink wrap, product based business model to a cloud based, subscription business

model. Program Management has played its part in this transformation by reinventing the role and being

more in tune with the realities and requirements of the business. This paper context is from a software

product development company perspective but some of the takeaways can be applied to an IT service

bespoke development environment. Experiences shared here are some of the authors own experiences

or the experiences shared by other Program Managers from different parts of the business.

Please Note: Data/charts used in this paper are indicative/sample data to protect privacy of our

customers.

Program Management Evolution at Adobe

The only constant in life is change. Everything about software development has changed and changed

fast. From waterfall to agile, from shrink wrapped to cloud, from desktop to devices, from no social to

social, from multiple year release cycles to multiple times a day releases. With this rapidly changing

environment program management has to change and adapt as well.

Traditionally Program/Project Managers have been focused mainly on

1) Managing Cost , Schedule , Delivery , Quality

2) Cross Functional Team Management

3) Stakeholder Management

4) Upstream/Downstream Communication Management

5) Dependency and Risk Management

6) Coordination ( Cross Geo, Cross Team, Intra Team)

With the advent of Agile this has changed. Agile brings in principles of self-managed team and several of

these aspects have now been taken over by engineering teams themselves. Program Managers are

increasingly taking the role of resolving outside team control impediments. This has allowed Program

Managers to also look at other value adds they can bring to the teams. Based on our own experiences

and by talking to other Program Managers we have identified three main areas where we feel Program

Managers have stepped up and started to add value to the business.

Lean Development

The concept of lean comes from manufacturing. Resources are limited and they need to be optimized and

waste needs to be minimized (a la Toyota philosophy). This require making sure we are

Page | 4

Page 5: Presentation by namit

Figure 1

manufacturing/developing the right things. Lean movement is geared towards that. It encourages to

develop incrementally and take frequent customer feedbacks to ensure we are on the right track. This

requires a mindset change from the days when project cycles used to be more than one year long.

Program Managers can help here by becoming lean coaches and empowering the teams to move fast,

move independently and move in the right direction by eliminating wasteful practices. Program Managers

can help create/contribute, maintain and enhance a companywide Lean framework (examples of artifacts

which are part of the framework would be training recordings, templates, samples, case studies etc.)

which new and existing teams can leverage. In essence,they can democratize the lean start up process.

Make it available to anyone, anytime, anywhere. Majority of Program Managers at Adobe today have

attended Lean workshops and are now in the process of taking their teams on “Do it right and do what is

right” path. Several product teams have been able to move to monthly and quarterly release cycles by

imbibing these principles and thus providing continuous value to the customer. A subscription based

model creates low stickiness because exit barrier for the customer is lowered. Therefore by providing

continuous value to the customer we have been able to keep our attrition rates to better than planned

numbers.

The Lean Cycle

Figure 1 illustrates the Lean Cycle some of the product teams have started to use at Adobe.

Build a Hypothesis– At the start of a release or when a new product is conceived, product team builds a

hypothesis. This hypothesis has inherent assumptions/intuitions which need to be validated by taking real

customer feedback. The hypothesis identifies the customer pain we wants to solve and how we want to

solve the pain. This can also be referred as the use cases we wanted to address.

Build Experiments – Experiments are created to validate the hypothesis. Experiments could be as

simple as prototypes created to demonstrate

intended user benefits. Experiments help cut the

waste as they are not effort intensive. Product

team instincts would often translate into

experiments.

Quantify –Qualitative (unstructured, anecdotal,

revealing and hard to aggregate) and

Quantitative(involves numbers and statistics and

provide hard numbers but less insight) feedback

on experiments needs to be captured. This

concept is also referred as data driven product

Page | 5

Page 6: Presentation by namit

development where measurement aspects are defined upfront and built into the product from the very

beginning.

Analyze – this feedback should then be analyzed and the output of this phase is a set of learnings.

Mostly Program Managers have used spreadsheets to collect the data and ran specific analytical tools

like Cohort Analysis or Pivot tables to draw inferences.

Learn – Post analysis team knows if the hypothesis was correct (in this case team decides to persevere

with the hypothesis) or not (in these case team decides to pivot or change course). Any adjustments

required can be feeded into the next iteration and this whole process is repeated at regular intervals.

The Lean cycle allows teams to quickly develop a product which can be monetized rather than the

traditional approach of trying to monetize what was developed.

Customer Delight

Faster release cycles and data driven product development approach requires one to be in constant

touch with customers (greater than ever before). Product teams are busy developing the right product and

this has opened up the opportunity for Program Managers to participate and drive the customer delight

cycle (Figure 2)

Validate Often - As discussed in Lean principles, Program Managers need to ensure we are reaching out

to customers on a regular basis for validation of work being done. Ideally this frequency should be same

as the duration of one development iteration but a few iterations can also be clubbed depending upon the

nature of the project. Customers own time taken to absorb incremental builds and investment of time to

test and provide feedback needs to be factored in validate often decisions.

Satisfaction Surveys - Periodically satisfaction surveys should be rolled out to get pointed customer

feedback on areas which need improvement. In our company environment this is typically done once a

quarter.

Proactive Response - Program Managers

need to keep a watch on customer

communication and ensure prompt and

comprehensive response. The number of

channels via which customer can

communicate directly or indirectly with us

has increased – email, web, phone, social

media, forums etc. therefore the need to

monitor and consolidate all communication

Page | 6

Page 7: Presentation by namit

channels. Often a response may require cross team, cross functional coordination where Program

Managers are best suited to get a prompt response back to the customer.

Engage - the customer in the process of development right from the start. Engagement can be by way of

Beta/Prerelease programs, newsletters, webinars, events, emails etc. Engagement drives the quality of

feedback being received from the customers and adds a sense of ownership and trust in the community.

Partner - Program Managers should treat customers as their partners to the success and thus there is

some element of relationship building/rapport building which should happen. Program Managers should

identify counterparts on customer side and build the bridges to make this happen.

Data Driven Analysis

Program Managers should enable the teams to takedata based decision and not subjective/intuition/gut

based decisions. To this extent Program Management should keep an eye on various data capture

points, get the data and build insights. Data should be sliced/diced and analyzed. Peter Drucker famously

said – if you can’t measure it you can’t manage it (or scale it)

For several product teams, analytics instrumentation is now part of user story/product backlog. Analytics

is used to corroborate key hypothesis, understanding patterns, analyzing errors and resolving workflow

issues. Measuring makes us accountable. Teams are forced to confront inconvenient truth rather than be

guided purely by vision/gut/hunch/subjective opinions.

Below are a few examples to illustrate how program management has enabled the team to usedata

driven analysis concepts.

Customer Lifecycle Funnel

Few teams have started using Customer Lifecycle funnels (Figure 3). These funnels track the numbers of

customers who move from Acquisition to Referral as a % of total customers acquired. Greater the number

of customers who are not only revenue generating but also refer products to others (a concept we also

call as Net Promoter Score) the better it is for product teams.

Acquisition –

this is the number

Page | 7

Page 8: Presentation by namit

of customers ‘acquired’ via customer acquisition process (marketing campaigns, advertising,

social media etc.)

Activation – number of customers who actually sign up to use the product and register the

product and launch it

Retention – is determined by repeat usage. Customers who launch the product more than once, login

more than once, use more than one feature more than one time. It is not necessary that the customer is a

paying customer to qualify as a retained customer. Several products today are available in freemium

model whereby the base products could continue to be used for free.

Revenue – is determined by how many customers are paying customers

Referral – is the number of customers who have brought new customers by way of referrals. This set of

customers are the most engaged and valued customers.

Cohort Analysis

Some of the program Managers have successfully used Cohort analysis (Figure 4). A Cohort is a logical

collection of customers having the same set of characteristics. A sample Cohort analysis done on

Creative Cloud subscribers for measuring usage, conversion and retention over time.

Figure 4

Page | 8

Page 9: Presentation by namit

Product Scorecards

Another example of data analysis for one of the products – A product scorecard was created to measure

most launches, active use, conversions and retention (Figure 5)

Figure 5

Scorecards - Product Scorecards                    Product ABC - App Most Launched Cohort View - Active Use, Conversion and Retention at 7,30,60 Days

2013-42

2013-43

2013-44

2013-45

2013-46

2013-47

2013-48

2013-49

2013-50

2013-51

Free- 7 Day FunnelFree members who launch in 7 Days

18660

19092

19762

19706

19409

19981

18809

19064

19921

19307

% of members who launch in 7 Days 30% 30% 30% 29% 29% 29% 29% 29% 29% 29%Active Free Members (2+Launch) 5594 5757 5901 5780 5790 5500 5442 5800 5507 3456% Members who Launch in 7 Days

1.10%

1.20%

1.10%

0.80%

0.90%

1.00%

1.00%

1.00%

0.90%

1.10%

Convert in 7 Days 202 234 208 165 173 207 196 184 182 213

Fee-30 Day FunnelFree members who launch in 7 Days

18660

19092

19762

19706

19409

19981

18809

19064

19921

19307

% of members who launch in 7 Days 30% 30% 30% 29% 29% 29% 29% 29% 29% 29%

Page | 9

Page 10: Presentation by namit

Active Free Members (2+Launch) 5594 5757 5901 5780 5790 5500 5442 5800 5507 3456% Members who Launch in 7 Days

1.10%

1.20%

1.10%

0.80%

0.90%

1.00%

1.00%

1.00%

0.90%

1.10%

Convert in 7 Days 202 234 208 165 173 207 196 184 182 213

Globalization Metrics

Examples of Metrics which Adobe Globalization team captures to support product teams in their product

Globalization efforts.

Localization Revenue by country

Localization Revenue by app language

English vs. non-English revenue

Localization spend vs. revenue from

localization versions

Downloads by app language

Paid vs. Trial by language for Int’l

customers

Country prioritization / product

prioritization

Cost of Adding a new language

Cost per word added in the UI

Cost of re-work (string

churn/certifications)

Localization content readership (Product

Help)

Volume of outsourced effort (Cost)

Volume of words localized across all

languages

Leveraging Technology efficiency

NPS Score

CSAT Score

Product specific quality measures

Product Locale Analysis

Another example of Analytics done by Globalization team (Figure 6) wasto measure locale wise number

of downloads. Such metrics is helpful to product teams to optimize spend per locale. Teams can now take

informed decisions.

Figure 6

Page | 10

Page 11: Presentation by namit

In some of the product teams, Program Managers have brought in data game storming techniques to

enable product teams to define critical factors for success and create metrics dashboards. Gamification of

innovation and dashboarding makes the whole process engaging and fun. Some teams have

experimented with online mediums like www.innovationgames.com to bring in gamification to innovation

process. Program Managers have also acted as facilitators and coaches in the process.

Forum Analytics

An example from forums tracking(Figure 7). Here customer inputs are taken from forum participation in

terms of how many new threads were there, how many views were there, how many were responded.

Program Manager creates a voice of customer report where top 10 issues in terms of views is highlighted

and categorized in terms of product areas impacted. This report is then reviewed with engineering on a

monthly basis and tangible action items are discussed and updated in the form of backlog items which are

then prioritized by Product Management.

Figure 7

Discussion User Total Total Thread

Page | 11

Page 12: Presentation by namit

Thread Names View helpful_answercorrect_answer Reply Link

Thread Topic 1 _name 34 4 2 12 URLThread Topic 2 _name 45 5 3 32 URLThread Topic 3 _name 23 8 6 24 URL

Support Cases by Customers

Below example (Figure 8) shows how number of support cases from customers is tracked. Orange boxes

indicate customer stopped using the product. This metrics was studied along with the revenue brought in

by each of that customer and that helped the engineering team to prioritize escalations work. This also

helped forecast how many escalations the team should be factoring in their release plan. Learning here

was to not look at the number of support issues in isolation but also look at the revenue impact and

strategic importance of the customer to the business.

Figure 8

Company Name

1/2010

2/2010

3/2010

4/2010

1/2011

2/2011

3/2011

4/2011

1/2012

2/2012

Grand Total

A 22 10 38 12 20 15 14 5 9 1 146

C 23 25 12 23 11 8 3 7 5   117

C 49 35 4 2 3 5 3   7 2 110

D 7 6 11 19 25 14 9 17     108

E 24 9 17 21 14 4 3   1   93

F 9 10 21 7 8 7 3 6 2 2 75

G 8 22 5 8 9 4 3 4 5 3 71

H 16 7 4 8 5 5         45

I         2 3 4 8 16 9 42

J 3 3 4 7 6 2 5 8   3 41

K 8 7 5 9 5 2 1       37

L 3 4 1   4 8 8 1 2 3 34

Trial Downloads

Page | 12

Page 13: Presentation by namit

This example (Figure 9) shows actual product trial downloads were captured along with number of visits

to the login page. Observation here was that that the number of actual downloads was less than 50% of

visits to the login page which sends a message to the product team for taking corrective actions.

Figure 9

Net Promoter Score Surveys

This example (Figure 10) is for the Net Promoter Score surveys rolled out to customers to determine

likelihood of them to promote our products to others

Figure 10

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8

Customer 1 7 5 4 3 4 5 5 4.3

Customer 2 8 4 5 4 2 4 4 3.8

Customer 3 9 5 3 5 5 5 4 4.5

Customer 4 9 4 5 5 3 4 4 4.2

Customer 5 6 4 2 3 2 3 2 2.7

Customer 6 8 4 4 4 2 2 4 3.3

Conclusion

Page | 13

Page 14: Presentation by namit

Changing business and work environment has opened up a lot of new opportunities for Program

Managers to contribute beyond the regular project management & rather participate as an equal

business stakeholder. At Adobe, Program Managers are focusing on three key areas – (1) bringing in

lean development to the teams (2) bringing in customer advocacy i.e.getting the customer involved in

all aspects of product development and (3) enabling the teams to take data driven decisions.

This is how we see that the program management will evolve (or has already evolved) in coming days

& we should be ready to embrace it.

References

The Lean StartUp – Eric Ries , Penguin Publication , 2011

Lean Analytics – Alistair Croll and Benjamin Yoskovitz , O Reilly Publication , 2013

Adobe Intranet ( Not accessible outside Adobe)

www.innovationgames.com

Page | 14