Upload
quentin-shelton
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
Using Talent Development Reporting Principles (TDRp) to Run Learning Like a Business
2013 Skillsoft Perspectives Orlando May 13, 2013
• Dave Vance, Executive Director, CTR
• Dawn Campbell Borland, TD Bank
2Workshop Objectives
May 13, 2013Skillsoft Perspectives
At the conclusion of the workshop, you will be able to:
• Articulate the principles of TDR and its benefits to your organization
• Identify the type of measures you should gather in your organization to populate the statements and reports
• Explain the Talent Development statements and reports and why each is important
• Understand how the statements and reports can help you rum learning like a business
• Appreciate the change that will be required to implement TDRp and run learning like a business
• Get started!
3Agenda & Guidance
May 13, 2013Skillsoft Perspectives
• Overview» TDRp background and context
» Running Talent like a business
» Executive reporting framework
• Break
• The Specifics» Recommended measures
» TDRp Statements & Reports
• Managing the Change
• Learn More
• Wrap Up
• Our guidance to you
» Maintain an aspirational view
» Think about what you can do in 2013 to make progress
The perfect is
the enemy
of the good.
--- Voltaire
4Interactive Introductions
May 13, 2013Skillsoft Perspectives
• Organization size (<100K employees, >100K employees)
• Sector (Private, public)
• Organization (L&D, HR, Business)
• Role (Individual contributor, manager/director, senior leader)
• Experience with Executive Reporting (yes, no)
• # of L&D Employees (<250, >=250)
• Structure of L&D (centralized, decentralized, hybrid)
• Common Evaluation Platform (Yes, No)
5
TDRp Background & Context
May 13, 2013Skillsoft Perspectives
6Why TDRp?
May 13, 2013Skillsoft Perspectives
• Need for commonly-agreed upon standards for measuring, reporting, and managing human capital
• Need to run talent like a business: deliver planned, agreed-upon results that help achieve company goals
• Need to clearly show alignment and value of human capital initiatives
• TDRp is as much about management as it is about measurement» Measurement, analysis and
reporting are all means to the end
» The end is effective, efficient management to deliver important outcomes
7Talent Development Reporting Principles (TDRp)
• Bring standardization and consistency to talent development for » Internal reporting
» Demonstrating the value of human capital
» Running talent like a business
• Answer the common questions» What to collect
» How to define it
» How to report it
» What to do with it
• Leveraging work done by others
May 13, 2013Skillsoft Perspectives
8A Brief History of the Movement
2010 2011 2012 2013+
May 13, 2013Skillsoft Perspectives
• Launched in Oct by Kent Barnett, CEO of Knowledge Advisors and Tamar Elkeles, VP of Learning & Communication at Qualcomm
• Industry thought leaders engaged
• Leading industry practitioners form Executive Council
• Knowledge Advisors provides initial funding
• Development of TDRp for L&D
• First whitepaper approved by Council
• Presentation at Learning Analytics Symposium
• TDRp extended to all key talent processes
• Presented to over 1500 people
• ASTD ICE and Singapore
• Creation of Center for Talent Reporting (October)
• New website goes live
• Membership• Credentialing,
Accreditation available
• Workshops• Webinars• User group
conference • SHRM/ANSI
standard
9The Center for Talent Reporting:The Home of TDRp
• Established October 2012» Not-for-profit, 501c(6) organization (used for trade associations)
• Funded by sponsors and members
• Governed by nine-member Board of Directors» Including a Standards Committee to provide continued guidance on
standards and reporting and an Advisory Council
• Mission» Develop and implement reporting standards for human capital
» Business model: Others to provide consulting and products
May 13, 2013Skillsoft Perspectives
Plans for 2013 10
May 13, 2013Skillsoft Perspectives
• Two-day workshops offered in 2013» Feb 14-15 Denver
» Mar 27-27 Irvine
» Jun 6-7 Alexandria, VA
» Oct 15-16 Atlanta
• Webinars offered monthly» Intro to TDRp
» Implementation CSFs
» Implementation Guidance (members only)
• First TDRp conference planned for October 17
• Certification» Individuals
» Vendors providing software products employing TDRp
• Accreditation» Organizations implementing TDRp
» Consultancies providing services
• SHRM will establish a taskforce in 2013 to consider recommending TDRp for L&D as a SHRM/ANSI standard
11
SponsorsWe want to thank the following sponsors for their support of the Center for Talent Reporting.
Center For Talent Reporting www.centerfortalentreporting.org
FOUNDING ORGANIZATION
SILVER SPONSORS
BRONZE SPONSORS
9-May-13
12
Running HR (Talent) Like a Business
May 13, 2013Skillsoft Perspectives
13
• Why does the talent function exist?
• Why do organizations invest in:» Learning and Development
» Leadership Development
» Capability Management
» Performance Management
» Talent Acquisition
» Total Rewards
First: Agree on Our Role
May 13, 2013Skillsoft Perspectives
14First: Agree on Our Role
• Help our organizations achieve their goals» More quickly
» At lower cost
» More effectively
• Best accomplished by running HR like a business focusing on:» Outcomes (Are we doing the right things?)
» Effectiveness (How well?)
» Efficiency (How much and at what cost?)
May 13, 2013Skillsoft Perspectives
1. Identify the business goals
2. Align HR initiatives to business goals and plan the initiatives carefully
3. Execute and report with discipline
4. Measure and evaluate
May 13, 2013Skillsoft Perspectives
Four Steps to Run HR Like a Business 15
16Step 1: Identify Business Goals
May 13, 2013Skillsoft Perspectives
• Proactive, strategic exercise» Meet with CEO or business
unit leader
» Learn about next year’s goals and priorities
» Learn who the sponsors are
• Results from Step 1» List of CEO-prioritized
SMART goals and sponsors
• We need specific, measurable goals expressed in terms of change from the previous year
• Example» The goal is to increase sales from
$50M to $55M- Goal: Increase sales by $5M or
10%
» The goal is to increase the employee engagement score from 60 points to 63 points- Goal: Increase engagement by 3
points (don’t say by 5% since some may interpret that to be 5 percentage points)
17Step 2: Align HR Initiatives to Business Goals; Plan the Initiatives Carefully
May 13, 2013Skillsoft Perspectives
• Meet with sponsors to understand» Business need in greater depth, the
challenge, past success & failure
» HR’s role (if any)
• If HR has a role, agree with sponsor on» Specific initiatives, programs,
systems, or processes
» Target audience, location, completion dates
» Number of participants
» Roles and responsibilities for sponsor and HR
» Measures of success: Expected impact of initiative on business goal
• Expected impact of HR on business goal» This is the contribution of just the
talent initiative(s) on the business goal
» In L&D terms, it is the expected isolated impact of learning. In other words, it is a forecast of the level 4 (impact)
» Our aim: a reasonable, roughly right forecast that is mutually agreed upon between the sponsor and HR. It needs to be “close enough” to help us make the right decision about whether the initiative is worth pursuing
» Both parties need to own it
18
• The business goal: 10% increase in sales for next year» L&D and the sponsor agree training could help achieve the 10%
increase
» Specific programs, target audiences, completion dates are agreed
» What portion of the 10% increase might reasonably come from this agreed-upon program?
» Perhaps 20%.
» Then the expected isolated impact of sales training on the goal is 20% x 10% = 2% increase in sales due to training
Step 2: Align HR Initiatives to Business Goals; Plan the Initiatives Carefully
May 13, 2013Skillsoft Perspectives
19
• Reach agreement on the expected isolated impact» Ask sponsor to list all drivers of higher sales
» Now ask sponsor to prioritize the drivers
» Then ask sponsor to list the expected % contribution of each. Work the percentages until they add up to 100%
» The percentage next to the sales training initiative is what we are looking for. This is the expected isolated impact of training on achieving the business goal.
» Note that the expected impact came from the sponsor, not HR
• If impractical or impossible to get a % contribution, assign a High, Medium, or Low impact based on where the sales training fell in the prioritized list.
• Or agree on a proxy for impact and success
Step 2: Align HR Initiatives to Business Goals and Plan the Initiatives Carefully
May 13, 2013Skillsoft Perspectives
20
• Start with a list of drivers
• Potential drivers to increase sales by 10%• Hiring two new salespeople
• Growth in economy
• Opening new sales office
• New salesperson incentive system
• New advertising campaign
• Growth in market share due to new products
• Failing competitor
• New sales tracking system
• Consultative sales and product features training
Step 2: Example of Impact Determination with Sponsor
May 13, 2013Skillsoft Perspectives
21Step 2: Example of Impact Determination with SponsorPrioritize and assign contribution %Prioritized Drivers %
contributionHR Initiative?
1. Growth in economy 25%
2. Consultative sales and product features training
20% Yes
3. New salesperson incentive system
15% Yes
4. Growth in market share due to new products
10%
5.Hiring two new salespeople 10% Yes
6. Opening new sales office 5%
7. New advertising campaign 5%
8. Failing competitor 5%
9. New sales tracking system 5%
May 13, 2013Skillsoft Perspectives
Prioritized list of drivers to increase sales by 10%
Prioritized list of drivers with percentage contribution to increase sales by 10% (These add to 100%)
Notice that HR initiatives in total are expected to contribute 45% or almost half to the goal of increasing sales by 10%
22
• Manage your execution to deliver planned outcomes, effectiveness, and efficiency
• Review progress monthly within HR» Year-to-date results versus plan
» Will you make plan?
» If not, what is the forecast?
» What action should you take?
• Review progress at least quarterly with CEO, senior leaders, sponsors» Focusing on most important business goals, key measures
Step 3: Execute and Report with Discipline
May 13, 2013Skillsoft Perspectives
23Step 4: Measure and Evaluate
May 13, 2013Skillsoft Perspectives
• Execute your measurement and evaluation strategy.
• Did you achieve your goals?
• If not, what can you learn? What can you improve?
• If you exceeded goals, what can you learn?
24Review: Four Steps to Run HR Like a Business
May 13, 2013Skillsoft Perspectives
1. Identify Business Goals
2. Align HR initiatives to business goals and plan it carefully
3. Execute and report with discipline
4. Measure and evaluate
25Our Experience in Running HR Like a Business
May 13, 2013Skillsoft Perspectives
• Conceptually not difficult
• Challenging in practice
• Requires new skills for some, confidence to apply them for others
• Some common objections» No standards in place to
provide guidance
» No internal processes to support it
» Not comfortable with business-like approach
• TDRp can help
26Exercise
May 13, 2013Skillsoft Perspectives
• Turn to your neighbor» Identify 3 ways that you are
running L&D like a business
» Identify 3 areas where you need to improve your processes to more effectively run L&D like a business
• Be prepared to share one idea with the group
27
Executive Reporting Framework
May 13, 2013Skillsoft Perspectives
28TDRp: Designed to Facilitate the Management of Human Capital
May 13, 2013Skillsoft Perspectives
• Alignment of human capital initiatives to organization goals
• Upfront agreement on the expected impact and other measures of success
• Monthly reports for management of initiatives and operations
• Quarterly reports for the Department Heads (like the CLO), SVPHR, CEO to assess progress and manage to planned results
• Focus on three types of measures: Outcomes, Effectiveness, Efficiency
29The Target Audience for TDRp
May 13, 2013Skillsoft Perspectives
Talent leaders and managers
Those responsible for programs, people, and budgets
Senior talent leaders SVP of HR, CLO, Head of Talent Acquisition, Heads of other talent processes (e.g leadership development, total rewards, capability management)
Senior organizational leaders
CEO, CFO, EVPs, SVPs, governing boards
Different reports are required for different audiences
30Operational vs Executive Reporting
May 13, 2013Skillsoft Perspectives
Operational Reporting Executive Reporting
Reporting on specific programs
A holistic picture of the business
Reporting on what happened
Reporting on what happened and projections for the year
Reporting results only Setting goals and reporting results against goals
Identifying specific improvement opportunities
Identifying how L&D is contributing to the business and how it can provide even greater value
Talking to ourselves using L&D terminology
Engaging business leaders in business oriented discussions about L&D performance
31Reports Are Designed to Answer:
May 13, 2013Skillsoft Perspectives
• What is the goal or plan?» Organization goal (e.g., 10% increase in sales)
» Impact of initiative on it (e.g., 20% contribution or 2% increase in sales)
• What key programs are planned to achieve it?
• How are we doing? (Year-to-date results versus plan)
• Are we going to make plan?(What is the forecast?)
• Focus: Managing the function versus merely Presenting results
32The Four Assumptions
May 13, 2013Skillsoft Perspectives
1. Primary purpose of human capital initiatives and processes: build organizational capability that enables the organization to achieve its goals or achieve its goals more quickly or at lower cost.
2. Human capital initiatives and processes should be aligned strategically to the goals of the organization
3. Business environment characterized by significant uncertainty; plans are made with the best information available. Waiting for absolute certainty and perfection is not an option.
4. Recommended reports and the underlying data will be used appropriately by competent, experienced leaders to manage the function to meet agreed-upon goals and to continuously improve.
33The Eight Principles
May 13, 2013
1. Employ concise and balanced measures reported in a consistent and clearly defined format
2. Produce and communicate Executive Reporting with a frequency and thoroughness to enable appropriate management of the function
3. Include actionable recommendations in all Executive Reporting
4. Maintain data integrity and completeness
5. Employ appropriate analytical methods
6. Provide the impact and value or benefit of programs and initiatives whenever appropriate
7. Capture and report the full costs of Talent
8. Support continuous improvement of executive reporting and the underlying data bases
Skillsoft Perspectives
34The TDRp Process
May 13, 2013Skillsoft Perspectives
1. Use standard definitions to extract and convert data into three types of measures- Outcome
- Effectiveness
- Efficiency
2. Create three standard statements with these measures- Outcome
- Effectiveness
- Efficiency
3. Create three types of customized reports from these statements- Summary
- Program (Initiative)
- Operations
35Executive Reporting Focus: Standard Measures
Data Sources and Systems
Systems and processes for organizing data, calculating
measures
Specific methodology
Statements
Standard Measures
Data
Reports
System Wide Analytics
Program/ Initiative Analytics
Standard Measures: • Efficiency measures: How much? How many? At what cost? • Effectiveness measures: How well?• Outcome measures: What is the impact on the business?
May 13, 2013Skillsoft Perspectives
36Executive Reporting Focus: Three Statements
Data Sources and Systems
Systems and processes for organizing data, calculating
measures
Specific methodology
Statements
Standard Measures
Data
Reports
System Wide Analytics
Program/ Initiative Analytics
Three Statements:• Outcome Statement• Effectiveness Statement• Efficiency Statement
May 13, 2013Skillsoft Perspectives
37Executive Reporting Focus
Data Sources and Systems
Systems and processes for organizing data, calculating
measures
Specific methodology
Statements
Standard Measures
Data
Reports
System Wide Analytics
Program/ Initiative Analytics
Three Reports:• Summary Report• Program Report• Operations Report
May 13, 2013Skillsoft Perspectives
38Executive Reporting FocusSummary Conclusions, Actionable
Recommendations, Issues for Further Analysis
Data Sources and Systems
Systems and processes for organizing data, calculating
measures
Specific methodology (e.g. Phillips, Brinkerhoff)
Scope of TDRp
Statements
Standard Measures
Data
Reports
System Wide AnalyticsProgram/ Initiative
Analytics
May 13, 2013Skillsoft Perspectives
Skillsoft Perspectives
Executive Reporting Process 39Senior Executives Talent Development Executives
Executive Reports
Statements
Note: Data sets can be organized by processes
and/or efficiency,
effectiveness & outcomes
Data Sources
Executive Reporting ProcessG
uid
ing
Prin
ciples
Financial Systems
Evaluation, EOS Systems
Non-Financial, non-TDR Systems
Others: HRIS, LMS, CRM, ERP
Outcomes Effectiveness Efficiency
Talent Development Summary Report
Quarterly
Talent Development Program Report (s)
Monthly
Talent Development Operations Report (s)
Monthly
EffectivenessStatement (s)
OutcomeStatement
EfficiencyStatement (s)
Talent Development Processes
Talent Acquisition
Leadership Development
Learning & Development
Capability Management
Performance Management
Total Rewards
Extract, convert and calculate Standard Measures
Data Sets
May 13, 2013
40
Recommended Measures
May 13, 2013Skillsoft Perspectives
41Optimizing Outcomes
Efficiency
Effectiveness
Optimized Outcomes
Optimize
SET LEARNING GOALS MONITOR & REPORT
ANALYZEIMPROVE
42Categories of Success Measures
• Effectiveness measures: Measures how well a task/process/function is performed
• Efficiency measures: Measures the extent to which time or effort is well used for the intended task or purpose (e.g. output relative to the input)
• Outcome measures: Measures the relationship and/or value of a program or activity to strategic business goals
May 13, 2013Skillsoft Perspectives
43L&D Efficiency Measures
Types Examples
Activity • # courses available• # classes delivered• # learners attending training (total and unique)• # hours of training consumed
Utilization / reach • Course fill rate• No shows• % of target audience reached
Cost • Cost of development/ delivery• External costs (vendors, course libraries)• Travel costs
Time • Development time• Time from request to fulfillment
May 13, 2013Skillsoft Perspectives
These measures answer the questions: how much, when, how long?
44What Other Efficiency Measures Do You Use?
•
•
•
•
•
May 13, 2013Skillsoft Perspectives
45L&D Effectiveness Measures
Types Examples
Level 1: Reaction • Courseware• Instructors/ Online delivery
Level 2: Learning • Pre vs post test difference• % of tests passed
Level 3: Application • % of learning applied (or not applied, “scrap”)• Perceived improvement in job performance
Level 4: Business Results
• Perceived impact on business results (post or follow up)
• Actual business results
Support • Manager support
Level 5: Value • Value for time spent
May 13, 2013Skillsoft Perspectives
These measures answer the questions: how well?
46What Other Effectiveness Measures Do You Use?
•
•
•
•
•
May 13, 2013Skillsoft Perspectives
47Outcome Measures
Type Examples
Financial Revenue, cost of goods sold, operating expenses, profit, margins, growth
Customer: satisfaction, loyalty
Satisfaction, loyalty, share of wallet, market share, complaints, referrals, reference accounts
Operational Productivity, product quality, new product development, patents
Employee Engagement, turnover (voluntary, involuntary), leadership quality index
May 13, 2013Skillsoft Perspectives
These measures answer the questions: what difference are we making?
48What Outcomes Do Your Programs Support?
• Sales?
• Productivity or cost reduction?
• Quality or safety?
• Compliance?
• Engagement? Retention? Attraction?
• Others?
May 13, 2013Skillsoft Perspectives
49How Do You Determine What to Measure?
May 13, 2013Skillsoft Perspectives
• L&D outcome measures» Reflected in the top strategic
goals of the company
» Reflected in important operational goals supported by L&D
» Are measured and reported by the business
• Efficiency and effectiveness measures» Reflect the needs of L&D
executives to manage the function efficiently and effectively in pursuit of- Company goals
- Department goals
» Are actionable
» Provide insightsBe open to new measures. Just because you haven’t measured
something before doesn’t mean it’s not worth measuring
50Build a Measures Library
May 13, 2013Skillsoft Perspectives
• What is it? The single source of truth of your measures» Based on TDRp measures library
and consistent with standard industry definitions
» Can contain other measures relevant to your industry or company (particularly with respect to outcome measures)
• Why is it important?» Ensures a common language
» Creates accountability
» Builds maturity
• What’s in it?» The measure, type and definition
» Criticality of the measure
» Frequency of data collection/availability (e.g. monthly, quarterly, annually)
» Source of data (system captured, manual) and data owner (organization or individual)
» Ease of accessing the data (availability, security)
» Quality of the data
• Who owns it?» A standards body that maintains,
updates and publishes the library
51Characteristics of a Good Measures Library
A Solid Library: Description
Measures what is important Focuses on key performance indicators of real value to the organization
Is predominantly quantitative
Most measures can be expressed as objective values
Is clearly defined and mutually understood
Defined by and/or agreed to by process owners or governance body
Encourages appropriate behavior
Balanced to reward productive behavior and discourage game playing
Is multi-dimensional Balances between efficiency, effectiveness & outcomes; balances rear-view mirror and forward looking measures
Articulates ownership Data quality and integrity is owned by specific individuals for specific measures
May 13, 2013Skillsoft Perspectives
52Considerations for Identifying Measures
May 13, 2013Skillsoft Perspectives
• Take time to understand the types of measures available » Review the measures library for
the relevant talent process
» Think about what each measure tells you and how it might provide insights
• Consider proxy measures» To simplify the process
» To advance your progress
• Not everything is equally critical. Create a prioritization matrix
• Criticality» High = Critical to demonstrate
tangible outcomes. Typically reserved for strategic, costly or visible programs
» Medium = Important to demonstrate outcomes. Helps make tradeoffs among alternative programs.
» Low = Won’t provide sufficient information for demonstrating outcomes
53Practitioner’s Perspective on Building a Measures Library
May 13, 2013Skillsoft Perspectives
54Exercise
May 13, 2013Skillsoft Perspectives
• Take 5 minutes
• For each category (efficiency, effectiveness and outcomes), identify 1-2 measures that you aren’t using today
• Identify how each measure could enable you to better manage the business of L&D
• Be prepared to share with the larger group
55Question: Are there any measures unique to e-learning?
•
•
•
•
May 13, 2013Skillsoft Perspectives
56TDRp Measures Summary
• Question: What measures are right for my organization?
• Answer:» Outcome measures that relate to key company goals
» Effectiveness measures that support company and department goals
» Efficiency measures that support company and department goals
• Target 3-5 for each to start
• Check Questions» Why are we collecting this?
» What will we do with it? Who will use it?
May 13, 2013Skillsoft Perspectives
57
TDRp Statements
May 13, 2013Skillsoft Perspectives
Yes, but what are your goals?
Skillsoft Perspectives
Executive Reporting Process 58Senior Executives Talent Development Executives
Executive Reports
Statements
Note: Data sets can be organized by processes
and/or efficiency,
effectiveness & outcomes
Data Sources
Executive Reporting ProcessG
uid
ing
Prin
ciples
Financial Systems
Evaluation, EOS Systems
Non-Financial, non-TDR Systems
Others: HRIS, LMS, CRM, ERP
Outcomes Effectiveness Efficiency
Talent Development Summary Report
Quarterly
Talent Development Program Report (s)
Monthly
Talent Development Operations Report (s)
Monthly
EffectivenessStatement (s)
OutcomeStatement
EfficiencyStatement (s)
Talent Development Processes
Talent Acquisition
Leadership Development
Learning & Development
Capability Management
Performance Management
Total Rewards
Extract, convert and calculate Standard Measures
Data Sets
May 13, 2013
59The Statements
• Three standard statements» Outcome
» Effectiveness
» Efficiency
• Uses standard measures but choice of measures depends on organization
• Summary statements show» Last year’s actual
» Plan (or goal) for this year
» Year-to-date results
• Detail statements show» Monthly, quarterly, trend data
» Granularity
» Without plan
May 13, 2013Skillsoft Perspectives
60Considerations in Creating the Outcome Statement
May 13, 2013Skillsoft Perspectives
• Requirements» Summary statement
» Lists organization’s top goals in priority order
» Identifies impact of talent initiative on goal
» Four columns- Last year’s actual (LYA)
- Current-Year Plan
- YTD Actual
- YTD Actual as % of Plan
• Options» Quantitative versus qualitative
» Isolated impact versus proxy
» Detailed statements
Statements
61Outcome Statement for L&DQuantitative & Qualitative with Proxies
May 13, 2013Skillsoft Perspectives
2011Priority Business Outcomes and Impact of Learning and Development Initiatives Actual Plan Jun YTD % of Plan
1 Revenue: Increase Sales by 20% Corporate Goal or Actual % 10% 20% 17% 85% Impact of L&D Initiatives: 25% contribution to goal % 1% 5% 4% 80%
2 Engagement: Increase Engagement Score by 3 Points to 69.4% (1) Corporate Goal or Actual Points 1 pt 3 pts 1.9 pts 63% Impact of L&D Initiatives High/Medium/Low Low Low Low
3 Safety: Reduce Injuries by 20% Corporate Goal or Actual % 10% 20% 15% 75% Impact of L&D Initiatives: High High/Medium/Low Medium High High
4 Comply with New Regulations Corporate Goal or Actual % in compliance 100% 100% 100% 100% Impact of L&D Initiatives: Essential Essential Essential Essential
5 Retention: Improve Retention of Top Performers by 5 Points to 90% Corporate Goal or Actual Points -3 pts 5 pts 2 pts 67% Impact of L&D Initiatives None planned
Learning and DevelopmentSample High-level Business Outcome Statement for the Private Sector with Mixed Impacts
Results through June
2012
Skillsoft Perspectives
62Experience with Outcome Measures
• Use of forecasted, isolated impacts of learning?» Quantitative
» Qualitative like high/medium/low or essential?
• Proxies?
May 13, 2013
63Considerations in Creating the Effectiveness Statement
May 13, 2013Skillsoft Perspectives
• Start with effectiveness measures directly related to the initiatives in support of organization goals
• Add effectiveness measures which relate to other organizational and department goals (if any)
• Requirements» Must be some effectiveness
measures
» Columns: LYA, Plan, YTD, YTD as % of Plan
• Options» Selected measures
» Grouping and labeling
Effectiveness Statement for L&D
May 13, 2013Skillsoft Perspectives
64
2011Actual Plan Jun YTD % Plan
Level 1: Employee (All programs and initiatives) Quality % top two boxes 80.0% 85.0% 85.6% 100.6% Amount learned % top two boxes 81.9% 85.0% 90.4% 106.4% Intent to apply % top two boxes 80.9% 85.0% 86.5% 101.8% Impact estimate (select programs) % top two boxes 82.8% 85.0% 85.4% 100.5% Value estimate (select programs) % top two boxes 77.8% 85.0% 85.3% 100.4% Total for Level 1 Average of measures 80.7% 85.0% 86.6% 101.9%
Level 1: Sponsor (Select programs) % top two boxes 75.0% 80.0% 77.0% 96.3%
Level 2 (Select programs) Score 78.0% 85.0% 83.0% 97.6%
Level 3 (Select programs) % who applied it 61.0% 75.0% 78.0% 104.0%
Sample High-level Effectiveness Statement Results through June
For 2012
Learning & Development
65Detailed Effectiveness Statement for L&D
May 13, 2013Skillsoft Perspectives
What’s added?• Quarterly data• Breakout measures
within the quality category
• Regional level data
Skillsoft Perspectives
66Experience with Effectiveness Measures
• How many use L1? L2? L3?
• How many set goals for them?
• How many actively manage through the year?
May 13, 2013
67Considerations in Creating the Efficiency Statement
May 13, 2013Skillsoft Perspectives
• Start with efficiency measures directly related to the initiatives in support of organization goals» Number of participants
» Number of employees
• Add efficiency measures which relate to other organizational and department goals
• Requirements» Must be some efficiency
measures
» Columns: LYA, Plan, YTD, YTD as % of Plan
• Options» Selected measures
» Their grouping and labeling
Efficiency Statement for L&D
May 13, 2013Skillsoft Perspectives
68
2011Actual Plan Jun YTD % Plan
Total Participants Number 109,618 147,500 67,357 46%Total Unique Participants Number 40,729 45,313 36,998 82%
Courses Taken by Type of Learning ILT only % of total 56% 43% 46% 107% vILT only % of total 3% 9% 8% 87% E-learning only % of total 35% 40% 40% 98% Blended % of total 6% 8% 7% 91% Total courses % of total 100% 100% 100% 100%
Utilization E-learning Courses Available Number 60 74 70 95% Taken by more than 20 Number 50 72 64 89% % taken by more than 20 % 83% 97% 91% 94%Reach % of employee reached by L&D % 71% 95% 72% 76%
Investment L&D Expenditures Million $ 15.8 20.2 10 49% Opportunity cost reduction Thousand $ 63 295 168 57% Development & Delivery cost reduction Thousand $ 28 50 18 36%
Sample High-level Efficiency StatementResults through June
For 2012
Learning and Development
69Detailed Efficiency Statement for L&D
May 13, 2013Skillsoft Perspectives
What’s added?• Monthly data• More detailed
measures• Breakdown by
type of learning
70Experience with Efficiency MeasuresHow many report
• Number of participants?» Unique and total?
» Set goals?
• Number of courses, hours?» Set goals?
• Progress against completion dates?
• Utilization rates?» Classrooms? Instructors? System usage? Reach?
» Set goals?
• Development/delivery/opportunity cost reductions?
May 13, 2013Skillsoft Perspectives
71
Skillsoft Perspectives
Exercise:Start to Create Efficiency Statement
May 13, 2013
• List the efficiency measures you want to report
• Most efficiency statements will include: » Number of participants
» Number of courses or classes or hours (perhaps segmented type of learning)
» Reach (% of employees touched by learning)
» Cost and/or cost reduction
» Utilization ratios of courses, classes, instructors, or rooms
72Practitioner’s Perspective on Using the Three Types of Measures
May 13, 2013Skillsoft Perspectives
73TDRp Statements Summary
• Question: What do we do with the measures?
• Answer» Organize them into three standard statements
» Key measures from the statements will be used in the reports to manage the function
» Statements provide additional detail
May 13, 2013Skillsoft Perspectives
74
TDR Reports
May 13, 2013Skillsoft Perspectives
Skillsoft Perspectives
Executive Reporting Process 75Senior Executives Talent Development Executives
Executive Reports
Statements
Note: Data sets can be organized by processes
and/or efficiency,
effectiveness & outcomes
Data Sources
Executive Reporting ProcessG
uid
ing
Prin
ciples
Financial Systems
Evaluation, EOS Systems
Non-Financial, non-TDR Systems
Others: HRIS, LMS, CRM, ERP
Outcomes Effectiveness Efficiency
Talent Development Summary Report
Quarterly
Talent Development Program Report (s)
Monthly
Talent Development Operations Report (s)
Monthly
EffectivenessStatement (s)
OutcomeStatement
EfficiencyStatement (s)
Talent Development Processes
Talent Acquisition
Leadership Development
Learning & Development
Capability Management
Performance Management
Total Rewards
Extract, convert and calculate Standard Measures
Data Sets
May 13, 2013
76The Reports
May 13, 2013Skillsoft Perspectives
• Three executive reports for the active management of human
capital to deliver planned results
• Constructed from the three statements just as management reports
in an organization are constructed from underlying financial and
activity reports
• The three executive reports:
» One targeted for use with senior executives
(CEO, SVP, etc.)
» Two targeted for use with talent executives
and program managers (those with specific
responsibility for programs, budget and people)
77The Reports (continued)
May 13, 2013Skillsoft Perspectives
• Three levels of reports» Summary Report (for CEO)
» Program Report (for CLO)
» Operations Report (for CLO)
» Highly customized, pulling the most important measures from the statements
• Executive-level reports show» Last year’s actual
» Plan (or goal) for this year
» Year-to-date results
» Forecast for this year
• Detailed reports for managers may show» Monthly, quarterly, trend data
» Granularity
» Without plan or forecast
78Bringing it All Together: The Summary Report
May 13, 2013Skillsoft Perspectives
• Audience: senior business and HR executives
• Most significant measures from the effectiveness, efficiency and outcome statements aggregated at the highest level
• Measures will vary by organization and year to year based on organizational priorities
• Generated at least quarterly
• Must be read holistically to obtain an accurate picture of progress against goals
Senior Business Executives
OutcomeStatement
Learning Effectiveness
Statement
Talent Development Summary Report
Quarterly
EffectivenessStatement (s)
OutcomeStatement
EfficiencyStatement (s)
79Considerations in Creating the Summary Report
May 13, 2013Skillsoft Perspectives
• Requirements» Summary report
» Top goals for organization impact by talent initiatives
» Most important effectiveness and efficiency measures
» Columns: - LYA
- Plan
- YTD
- YTD as % of Plan
- Forecast
• Options» Quantitative versus qualitative
» Isolated impact versus proxy
» Detailed report
» Names of measures, how they are grouped, group names
New!
Skillsoft Perspectives
80L&D Summary Report
May 13, 2013
Home
with Quantitative Impacts
2011 For 2012 Impact of Learning and Development Initiatives Actual Plan Jun YTD % of Plan ForecastRevenue: Increase Sales by 20% Corporate Goal or Actual % 10% 20% 17% 85% 20% Impact of L&D Initiatives: 25% contribution to goal % 1% 5% 4% 80% 5%
Engagement: Increase Engagement Score by 3 Points to 69.4% (1) Corporate Goal or Actual Points 1 pt 3 pts 1.9 pts 63% 2.5 pts Impact of L&D Initiatives High/Med/Low Low Low Low Low
Safety: Reduce Injuries by 20% Corporate Goal or Actual % 10% 20% 15% 75% 20% Impact of L&D Initiatives High/Med/Low Med High High High
Effectiveness Participant Feedback % favorable 80% 85% 87% 102% 85% Sponsor Feedback % favorable 75% 80% 77% 96% 78% Learning Score 78% 85% 83% 98% 84% Application rate % who applied it 61% 75% 78% 104% 79%
Efficiency % of courses that are traditional classroom % 56% 43% 46% 107% 42% E-learning course utilization rate % taken by 20+ 83% 97% 91% 94% 97% % of employees reached by L&D % 71% 95% 72% 90%
L&D Investment L&D Expenditure Million $ $15.8 $20.2 $9.9 49% $20.2 Opportunity cost reduction Thousand $ 63 295 168 57% 325 Development & Delivery cost reduction Thousand $ 28 50 18 36% 50
Sample Summary Report for the Private Sector
Results through June
Learning and Development
81Creating the Summary Report
May 13, 2013Skillsoft Perspectives
• Copy your three goals and impact of learning from the Outcome Statement
• Copy 3-4 effectiveness measures from your Effectiveness Statement
• Copy 3-4 efficiency measures from your Efficiency Statement
82The Program Report
May 13, 2013Skillsoft Perspectives
• Purpose: Manage key programs to deliver planned results
• Audience: Senior Talent Executives
• Generated for each major Talent program» Includes measures necessary for
senior talent leaders and managers to manage program, initiative, or process results on a monthly basis.
» Typically includes outcome measures relevant to the program and the key effectiveness and efficiency measures.
Senior Talent Executives
OutcomeStatement
Learning Effectiveness
Statement
Talent Development Program Report
Monthly
EffectivenessStatement (s)
OutcomeStatement
EfficiencyStatement (s)
83Considerations in Creating the Program Report
May 13, 2013Skillsoft Perspectives
• Requirements» Organizational goal the
initiative supports
» Impact of initiative on goal
» Sponsor’s name
» Key effectiveness measures
» Key efficiency measures
» In either rows or columns- Plan, YTD actual, YTD as % of
Plan, Forecast
- For all listed outcome, effectiveness, and efficiency measures
• Options» Level of detail (simple,
medium, complex or multi year)
Simple Program Report for L&D (page 1)
May 13, 2013Skillsoft Perspectives
84
Enterprise Goal: Reduce Injuries by 30% in 2012 Sponsor: Swilthe, VP of Manufacturing Expected Impact of Learning: High
Program Goals Level 1
Metric Value Employee Sponsor Level 2 Level 3Deliver Phase 1 Courses for Factory A Unique Participants 3,000 80% 90% 90% 65%
Total Participants 6,000Develop Phase 2 Courses for Factory B Complete by 5/30 3 90%Deliver Phase 2 Courses Unique Participants 1,000 80% 90% 90% 70%
Total Participants 3,000 ==== ==== ==== ==== ====
Total 80% 90% 90% 68% Courses Developed Number 3 Unique Participants Number 4,000 Total Participants Number 9,000Cost (not including opportunity cost) $250Thousand $
Sample Simple Executive Program ReportFor a Safety Initiative
Learning and Development
Simple Program Report for L&D(page 2)
May 13, 2013Skillsoft Perspectives
85
June Year-To-Date Results and Forecast
Year-to-Date Results Current ForecastEnterprise Goal: Injuries Reduced by 20% from First Six Months of Last Year 30% Reduction
Impact of Learning: High (Based on Level 1 and 3 results and anecdotal feedback) High
Current Forecast% of Level 1 % of
Metric Value Plan Employee Sponsor Level 2 Level 3 Value PlanDeliver Phase 1 Courses Unique Participants 2,800 93% 85% 88% 95% 62% 3,200 107%
Total Participants 5,542 92% n=1872 n=3845 n=270 6,300 105%Develop Phase 2 Courses Complete by 5/30 3 100% 90% 3 100%Deliver Phase 2 Courses Unique Participants 100 10% 80% 90% 92% 61% 1,100 110%
Total Participants 284 9% n=39 n=98 n=7 3,200 107% ==== ==== ==== ==== ==== ==== ==== ====
Total 83% 89% 94% 62% Courses Developed Number 3 100% 3 100% Unique Participants Number 2,900 73% 4,300 108% Total Participants Number 5,826 65% 9,500 106%Cost (not including opportunity cost) Thousand $ $178 71% $255 102%
Sample Simple Executive Program ReportFor a Safety Initiative
Learning and Development
86The Operations Report
May 13, 2013Skillsoft Perspectives
• Purpose: Manage Talent operations to deliver planned results
• Audience: Talent Executives and managers
• Pulls key measures from the Efficiency Statement
• Adds a column for forecast
Senior Talent Executives
OutcomeStatement
Talent Development Operations Report (s)
Monthly
EfficiencyStatement (s)
87Considerations when Creating the Operations Report
May 13, 2013Skillsoft Perspectives
• Select the most important efficiency measures from the efficiency statement
• Requirements» Key efficiency measures that
will be actively managed on a monthly basis
» Columns for LYA, Plan, YTD, YTD as % of Plan, Forecast
• Options» Names of the measures
» How the measures are grouped
» Names of the groups
88Operations Report for L&D
May 13, 2013Skillsoft Perspectives
2011 For 2012 Actual Plan Jun YTD % Plan Forecast
Total Participants Number 40,121 50,000 22,013 44% 48,000
Unique Participants Number 20,263 23,000 18,407 80% 24,000
Courses Taken by Type of Learning ILT only % of total 56% 43% 46% 107% 44% vILT only % of total 3% 9% 8% 87% 9% E-learning only % of total 35% 40% 40% 98% 40% Blended % of total 6% 8% 7% 91% 7% Total courses % of total 100% 100% 100% 100% 100%
Utilization E-learning Courses Available Number 60 74 70 95% 74 Taken by more than 20 Number 50 72 64 89% 72 % taken by more than 20 % 83% 97% 91% 94% 97%
Reach % of employee reached by L&D % 85% 88% 72% 88% 88%
Program Management Courses Total Developed Number 22 36 24 67% 35 Number Meeting Deadline Number 16 33 21 64% 33 % Meeting Deadline % 73% 92% 88% 95% 94% Total Delivered Number 143 178 167 94% 178 Number Meeting Deadline Number 89 160 139 87% 155 % Meeting Deadline % 62% 90% 83% 85% 87% Cycle Time Performance consulting Days 39 30 33 67% 32 Design and Development for ILT Days 56 45 44 109% 45 Design and Development for WBT Days 48 40 39 113% 39
Sample Executive Operations ReportResults through June
Learning and Development
89TDRp Reports Summary
• Question: How do the measures help run learning like a business?
• Answer» Key measures are shown in three executive reports
» The Summary Report is used by the CEO, governing body and CLO to manage the function at a high level on a quarterly basis
» Program Reports are used by the CLO, Directors, and Program Managers to manage program execution on a monthly basis
» The Operations Report is used by the CLO and Directors to manage key efficiency measures on a monthly basis
May 13, 2013Skillsoft Perspectives
90Practitioner’s Perspective on Executive Reporting
May 13, 2013Skillsoft Perspectives
Value Dashboard – monitor, manage and communicate training results to key audiences
91
Managing the Change
May 13, 2013Skillsoft Perspectives
92Making it Happen
• Technically, TDRp is pretty straight forward
• It’s about gathering relevant data and displaying it in a succinct way on reports for executives
• But if it’s so easy, why don’t we see TDRp (or some variant thereof) implemented within most Talent functions?
May 13, 2013Skillsoft Perspectives
TDRp requires a fair amount of change in
the way the L&D function operates.
And let’s face it, change can be
difficult.
93Exercise: What’s Changing?
May 13, 2013Skillsoft Perspectives
• Identify and share some changes that you believe organizations will experience as a result of TDRp
94Innovation Adoption Curve
May 13, 2013Skillsoft Perspectives
InnovatorsEarly Adopters
Early MajorityLate MajorityLaggards
Th
e C
has
m
I don’t understand why we need to do this. No one is asking for this data
This is a good idea. With support, I’m willing to move forwardIt sounds like a good idea, but
we have too many other things that we need to tackle right now. I’ll put this on the agenda for next year. “We need to act now. If we
don’t, we risk our ability to positively impact the business and become a true business partner
Individuals are at different points on the adoption curve. Sustained change happens when employees ‘cross the chasm’.
94
95The Change Model
May 13, 2013Skillsoft Perspectives
From John P. Kotter: Leading Change
DrivingChange
8. Make ChangeStick
7. Don’t let up
5. Empower Action6. Create Quick Wins
4. Communicatefor Buy-In
3. Get theVision Right
2. Build theLeadership Team
1. Create a Senseof Urgency
People start telling each other: “Let’s go, we need to change things!”
A group powerful enough to guide the change is formed and they work together well
The team develops the right vision and strategy for the change effort
People begin to buy into the change; this shows in their behavior
More people feel able to act and do act on the strategy
Momentum builds as people try to fulfill the strategy; fewer people resist
People make wave after wave of changes, fulfilling the vision
New and winning behavior continues despite the pull of tradition and turnover of change leaders
96Create a Sense of Urgency
• Risks of doing nothing?» Senior L&D leaders will lack a
holistic view of L&D performance
» L&D leaders will not have the information to prioritize investment decisions and funding allocation
» L&D leaders will lack a systematic approach to improving training results
» The L&D function will have a lower impact on business outcomes
» L&D will continue to be viewed as a cost center and will be subject to ongoing cost cutting
• Opportunity if you act?» L&D will build collaborative
partnerships with the business ensure it focuses on the most critical priorities, drives application and produces measureable results
» Senior L&D leaders run training like business including- An annual business plan
- Agreed upon KPIs
- Goals and performance targets
» L&D will systematically improve training results and their impact on business outcomes
» L&D will have the data, tools and methods to communicate its value to the business
May 13, 2013Skillsoft Perspectives
96
97Sponsors vs Stakeholders?
May 13, 2013Skillsoft Perspectives
• Sponsor: responsible to the CEO for delivering the business goal » The only person with whom you can
have a meaningful discussion about the impact of talent on the business
» Usually a direct report to the CEO at most one level further down (e.g. SVP of Sales, President of Manufacturing)
• Stakeholder: An individual who has a "stake" in the program » Can influence it
» Wants to know if it is successful
» Is directly affected by it
» Typically includes
- Head of the HR function (like the CLO)
- Managers/staff in the HR function responsible for the initiative
- Managers/staff in the client function (like Sales) responsible for the initiative
- May include the SVP of HR
98The Anatomy of Change
May 13, 2013Skillsoft Perspectives
Re
spo
nse
Skeptical
Enthusiasm
This is taking time Results aren't
visible
Is it worth it?
Maybe not a bad idea
It works!
The Dark Night
of theInnovator
Optimism
Pessimism
TimeInvestment Phase Pay-off Phase
Start to see pay-offs
Brass Bands and Fireworks
99
Implementing TDRp
May 13, 2013Skillsoft Perspectives
100Ten Steps to Implementing TDRp
May 13, 2013
1. Socialize TDRp concepts
2. Commit to implement
3. Engage a guiding team
4. Develop a multi-year plan
5. Identify business goals and establish common measures with sponsor(s) and stakeholders
6. Identify goals for each measure including the contribution of Talent to the business goals
7. Create draft statements and reports» Outcome Statement
» Effectiveness and Efficiency Statements
» Summary Report
» Program Reports
» Operations Report
8. Manage results to plan; conduct sponsor and stakeholder reviews
9. Review and improve
10. Integrate into business process
Skillsoft Perspectives
TDRp Implementation Journey101
Time
Create Awareness
(2) Commit to implement
(1) Socialize TDRp concepts
(5) Implement: Identify business goals and establish measures
(6) Implement: Identify goals
(7) Implement: Create draft statements & reports
(8) Manage: monthly/quarterly reviews
Institutionalize
Advocate, Champion
De
gre
e o
f S
up
po
rt f
or
Ch
an
ge
May 13, 2013
Expand, refine
(10) Integrate into business processes
( 9) Review: improve our processes
(4) Plan: Establish multi-year plan
This is a change process. Build capability over time
(3) Engage a guiding team
Skillsoft Perspectives
102Critical Success Factors
Talent Business Plan
LeadershipResults Drive
Action
Resources
Governance / Alignment
Data
May 13, 2013Skillsoft Perspectives
Technology &
Integration
Leadership• Exec Reporting driven
from the top (the department head)
• Talent leaders engage in ongoing dialogue with senior business leaders
Governance /Alignment• Align with business needs• Establish Talent priorities
Talent Business Plan• Driven by business priorities• Includes efficiency, effectiveness
& outcome measures with SMART goals
Results • Inform annual
planning process• Drive continuous
improvement
Data• Access to relevant data• Confidence in data quality • Availability matches reporting cadence
Technology • Systems and tools
enable scalable, repeatable process
• Drill down capability to explore anomalies
Resources• Bandwidth &
accountability• Knowledge & skills• Credibility with the
business
103Balance Supply with Demand
• Assess your organization’s strengths, weaknesses and blind spots
• In your implementation plan:» Leverage your strengths
» Develop a plan to mitigate your weaknesses
» Determine how you will reveal and address your blind spots
• Balance the supply of data with the demand (pull) for information that drives accountability and continuous improvement
May 13, 2013Skillsoft Perspectives
104Track Your Progress
May 13, 2013Skillsoft Perspectives
Stage Description Plan Actual Score CommentsForecasted Completion
Pre Awareness
1 Socialize TDRp concepts
2 Commit to implement
3 Engage a guiding team
4 Develop a multi-year plan
5 Establish measures
6 Identify goals
7 Create statements & reports
8 Manage monthly/quarterly reviews
9 Improve TDRp processes
10 Integrate into business processes
Post Expand, refine, recalibrate
Post InstitutionalizePost Advocate and Champion
Total Score 0% 0%
Completion Date or Progress
Internal note: Each stage has a maximum value of 100. Completion of implementation activities = 100. Maximum score is 1400
TDRp Progress Assessment
105TDRp Progress Assessment: Example
May 13, 2013Skillsoft Perspectives
Stage Description Plan Actual Score CommentsForecasted Completion
Pre Awareness Nov-11 Nov-11 100
1 Socialize TDRp concepts Feb-12 Feb-12 100
2 Commit to implement Mar-12 Mar-12 100
3 Engage a guiding team Apr-12 Apr-12 100
4 Develop a multi-year plan May-12 May-12 100
5 Establish measures Jun-12 75% 75Measures established but in review with stakeholders Jul-12
6 Identify goals Jul-12 60% 60Goal discussions completed with all but 3 stakeholders Aug-12
7 Create statements & reports Aug-12 0% 0 Not yet begun Oct-12
8 Manage monthly/quarterly reviews Sep-12 0% 0 Oct-12
9 Improve TDRp processes Jan-13 0% 0 Jan-13
10 Integrate into business processes May-13 0% 0 May-13
Post Expand, refine, recalibrate Oct-13 0% 0 Oct-13Post Institutionalize Feb-14 0% 0 Feb-14Post Advocate and Champion Jun-14 0% 0 Jun-14
Total Score 635% 45%
Completion Date or Progress
Internal note: Each stage has a maximum value of 100. Completion of implementation activities = 100. Maximum score is 1400
106P
rog
ress
to
op
tim
izin
g
trai
nin
g’s
val
ue
Measurement and Evaluation Strategic Roadmap
Reactive M&E Strategic M&E
Fully Integrated
‘Level 1’ Ad Hoc Approach
We are here
Skillsoft Perspectives
107
Learn More
May 13, 2013Skillsoft Perspectives
108CenterforTalentReporting.org
• Introduction to TDRp White paper
• Over 500 measures with definitions
• Over 60 sample statements and reports
• Detailed written guidance
• Sign up for email
May 13, 2013Skillsoft Perspectives
Skillsoft Perspectives
109The CTR Website
May 13, 2013
www.centerfortalentreporting.org
For L&D, start here. Go to Learning & Development to find all the whitepapers, definition of terms document, sample statements and reports, and detailed implementation guidance
110CTR Website: L&D Resources
May 13, 2013Skillsoft Perspectives
Tips & Tools Constructing reports: L&D Summary Report, Executive
Program Report, Outcome Report Customizing TDRp for your organization The L&D Summary Report and the Written Executive Report Multiple Learning Programs in Support of One or More Goals Outcome Discussions with the Stakeholder
Accelerate Your Mastery 111
May 13, 2013Skillsoft Perspectives
• Two-day workshops offered in 2013» Jun 6-7 Alexandria, VA
» Oct 15-16 Atlanta
» Use code SKWKSP1 to get $200 discount
• Webinars offered monthly» Intro to TDRp
» Implementation CSFs
» Implementation Guidance (members only)
• First TDRp conference planned for October 17
• Certification» Individuals
» Vendors providing software products employing TDRp
• Accreditation» Organizations implementing
TDRp
» Consultancies providing services
112Become a Member of theCenter for Talent Reporting
• Benefits» Access to Enhanced Content on the website
- Formulas and references for measures
- Excel versions of sample statements and reports
- Detailed implementation guidance
» $300 discount on workshops and conference
» Priority for TDRp certification
• Investment» Only $299
May 13, 2013Skillsoft Perspectives
113Wrap Up: Objectives Review
May 13, 2013Skillsoft Perspectives
By now, you should be able to:
• Articulate the principles of TDR and its benefits to your organization
• Identify the type of measures that are right for your organization to populate the statements and reports
• Explain the Talent Development statements and reports and why each is important
• Understand how the statements and reports can help you rum learning like a business
• Appreciate the change that will be required to implement TDRp and run learning like a business
114Advice:Just Get Started!
• Identify where you might begin to implement» One project
» One business unit or region
• Start small and grow» Begin with 3-5 measures from each group
» Start with what you already have
• Don’t wait for perfection, new data systems» You can do much manually
» You will learn as you go
• So, just start!
May 13, 2013Skillsoft Perspectives
115Contact Information
• Dave Vance: Executive Director» Email: [email protected]» Phone: (970) 460-0837
• Peggy Parskey: Assistant Director» Email: [email protected]» Phone: (323) 931-6589
• Dawn Campbell Borland» Email: [email protected]
May 13, 2013Skillsoft Perspectives
116
Skillsoft Perspectives
Appendix: Job Aids
May 13, 2013
117Sample Measures for L&D
May 13, 2013Skillsoft Perspectives
Type Purpose Examples
Efficiency Demonstrate the ability of the learning function to deliver learning solutions:• Cost effectively• In a timely manner• That reach the target audience• That use learning resources wisely
• Costs to design, develop and deliver content
• Course activity and volume• Time from request to deployment • Utilization of courses
Effectiveness Demonstrate the ability of the learning function to deliver learning solutions:• Aligned with business needs• Satisfy learners• Producing increased learning, new behaviors
and improved performance• Providing value to the business
• Level 1: Courseware quality, delivery effectiveness, learning effectiveness
• Level 2: Amount learned• Level 3: % of learning applied on the
job• Level 4: Job impact• Level 5: ROI or ROE
Outcomes Demonstrate the ability of the learning function to deliver learning solutions:• That produce tangible business outcomes• Result in high levels of satisfaction• Increase employee engagement and
retention
• Financials: revenue growth, profitability, cost of goods sold
• Customer satisfaction and loyalty• Operational: space utilization,
inventory turns• Employee engagement, turnover
118
Skillsoft Perspectives
The Outcome Statement
The Question The Answer
What is it? Depicts the key goals / desired outcomes (results) of the organization along with the impact the initiatives are expected to have on achieving those outcomes
Who is the audience? Senior HR Leaders
Why do it? The Business Outcome Statement depicts the value of the learning organization to the business
What reports does it feed?
• Quarterly Summary Report• Monthly Program Report
May 13, 2013
119
Skillsoft Perspectives
The Effectiveness Statement
The Question The Answer
What is it? Focuses on how effectively or how well the business outcomes are achieved.
Who is the audience? Senior HR Leaders
Why do it? The effectiveness statement provides insight into areas for continuous performance improvement of learning programs
What reports does it feed
• Quarterly Summary Report• Monthly Program Report
May 13, 2013
120
Skillsoft Perspectives
The Efficiency Statement
The Question The Answer
What is it? Brings together all the activity and cost elements necessary to judge how efficiently the outcomes were achieved
Who is the audience? Senior HR Leaders
Why do it? The efficiency statement helps ensure the learning function is well managed and reaching the desired audience
What reports does it feed
• Quarterly Summary Report• Monthly Operational Report
May 13, 2013
The Statements Compared
May 13, 2013Skillsoft Perspectives
Question Effectiveness Statement
Efficiency Statement Outcome Statement
What is it? • Focuses on how effectively or how well the business outcomes are achieved.
• Brings together all the activity and cost elements necessary to judge how efficiently the outcomes were achieved
• Depicts key goals / desired outcomes of the organization along with the expected impact of L&D on achieving those outcomes
Who is the audience?
• Senior L&D Leaders • Senior L&D Leaders • Senior L&D Leaders
Why do it? • The effectiveness statement provides insight into areas for continuous performance improvement of learning programs
• The efficiency statement helps ensure the learning function is well managed and reaching the desired audience
• The Outcome Statement depicts the value of the learning organization to the business
What reports does it feed?
• Quarterly L&D Summary Report
• Monthly L&D Program Report
• Quarterly L&D Summary Report
• Monthly L&D Operational Report
• Quarterly L&D Summary Report
• Monthly L&D Program Report
121
122
Skillsoft Perspectives
The Summary Report
The Question The Answer
What is it? This Summary Report contains the most significant measures from the business outcome,effectiveness and efficiency statements aggregated at the highest level. This report would be generated quarterly.
Who is the audience? Senior Organization Leaders & Senior HR Leaders
Why do it? The Summary Report provides a holistic view of the overall impact, effectiveness and efficiency of the learning organization
What statements feed it? • Outcome Statement• Effectiveness Statement• Efficiency Statement
May 13, 2013
123
Skillsoft Perspectives
The Program Report
The Question The Answer
What is it? This Program Report highlights theconnections between programs/ initiatives, the organizational goal, and the impacts on those goals.
Who is the audience? Senior HR Leaders and Managers
Why do it? The Program Report enables HR Leaders to manage programs and initiative
What statements feed it? • Outcome Statement• Effectiveness Statement• Efficiency Statement
May 13, 2013
124
Skillsoft Perspectives
The Operations Report
The Question The Answer
What is it? This Operations Report include efficiency details on the costs and activities of the HR function. This report shows actuals and performance relative to plan.
Who is the audience? Senior HR Leaders and Mangers
Why do it? The Operations Report enables HR Leaders to manage the efficiency of the HR operation
What statements feed it? Efficiency Statement
May 13, 2013
The Reports Compared
May 13, 2013Skillsoft Perspectives
Question L&D Program Report L&D Operations Report L&D Summary Report
What is it? • Highlights the connections between learning programs/ initiatives, the organizational goal and the impact of learning on those goals.
• Includes efficiency details on the costs and activities of the learning function.
• This report shows actuals and performance relative to plan.
• Contains the most significant measures from the outcome, effectiveness and efficiency statements aggregated at the highest level.
• This report should be generated quarterly.
Who is the audience?
• Senior L&D Leaders and their staff
• Senior L&D Leaders and their staff
• Senior Business Leaders & Senior L&D Leaders
Why do it? • Enables Learning Leaders to manage programs and initiative
• Enables Learning Leaders to manage the efficiency of the L&D operation
• Provides a holistic view of the overall impact, effectiveness and efficiency of the learning organization
What statements feed it?
• Outcome Statement• Learning Effectiveness
Statement• Learning Efficiency
Statement
• Learning Efficiency Statement
• Outcome Statement• Learning Effectiveness
Statement• Learning Efficiency
Statement
125
126Variations of L&D Statements
TypeStatement or Graph Level of Detail Sector
Quant, Qual, or Mixed/Proxies
Outcome
Statement High Private Quantitative
Statement High Private Qualitative
Statement High Private Mixed, Proxies
Statement Detailed Private Quantitative
Statement High Government Mixed, Proxies
Effectiveness
Statement High All Quantitative
Statement Detailed All Quantitative
Graph Detailed All Quantitative
EfficiencyStatement High All Quantitative
Statement Detailed All Quantitative
May 13, 2013Skillsoft Perspectives
127Variations of Reports: L&D
Type Level of Detail SectorQuant, Qual, or Mixed/Proxies
Summary High Private Quantitative (1)
Summary High Private Quantitative (2)
Summary High Private Qualitative
Summary High Government Mixed, Proxies
Program Simple All Quantitative
Program Medium All Quantitative
Program Multi Year All Quantitative
Program Detailed All Quantitative
Operations High All Quantitative
May 13, 2013Skillsoft Perspectives