GETTING STARTED USING LEARNING DATA TO IMPROVE DESIGN

#CLOwebinar

Sponsored By:

The presentation will begin at the top of the hour.

A dial in number will not be provided.

Listen to today’s webinar using your computer’s speakers or headphones.

Welcome to the webinar!

#CLOwebinar

Tools You Can Use Audio Control

–  A dial in number will not be provided.

–  Adjust the volume on your computer or headphones

#CLOwebinar

Questions and Handouts

You can submit questions by clicking on this icon here.

You can download a PDF of the slide deck by clicking here.

#CLOwebinar

Sponsored By:

1. May I receive a copy of the slides? YES! Click on the handouts list located on the right

portion of your screen.

2. May I review the webinar recording at a later date?

YES! You may log in again using today’s link to review the presentation on-demand.

3. Is this webinar HRCI certified?

YES! The HRCI certification code will be sent in the same follow up email as the recording

Frequently Asked Ques0ons

Using Data to Improve Learning Design Lori Niles-Hofmann @loriniles | www.lori.ca

Lori Niles-Hofmann

Senior learning strategist

20 years L&D experience (banking, consulting, marketing)

Author: Data-Driven Learning Design

Blog: www.lori.ca

@loriniles www.lori.ca

Diagnose vs. Autopsy


The Learning Autopsy?

▪ Completions ▪ Level 1-4 Evaluation

▪ ROI

▪ Too late

▪ Costly redesign

▪ Reputational risk

▪ Questionable Accuracy …the room was too cold, the food was not tasty


In the Classroom


Digital Distant Relationship


Paper to Online News

▪ “If it bleeds, it leads” ▪ Frontpage news cycle: 12-24 hours

▪ One metric: newspapers sold


Digital Changed News Media

▪ Real-time metrics on consumption ▪ Minute-by-minute pivoting

▪ Data revealed insights that challenged assumptions


Digital Changed News Media

▪ Real-time metrics on consumption ▪ Minute-by-minute pivoting

▪ Data revealed insights that challenged assumptions


Insights


Insights


Prediction vs. Reality: November 28th


Prediction vs. Reality: November 28th

Data reveals truths that could challenge what we

presume about our learners…

…it also explains the

Kardashians


Is Marketing a Bad Word?


Marketing vs. Learning

Both:

▪  Inform their audience with the intent of changing a behaviour

▪ Struggle with short attention spans and competition

▪ Face to face relationship is gone


Two Approaches

Marketing

• Data analysis for trends

• Content designed against insights

• Monitor and pivot

• Cyclical

Learning

• Performance consulting

•  Learning objectives

• Content design and development

• Evaluation

•  Linear

Downloads Drop-offs Time of Day

Likes

Shares

Comments

Trends

Mobile vs. Desktop

“Every drop-off, click, or share, is a learner shouting their likes and dislikes. These actions are the eye-rolls, smiles, and arms-crossed from the classroom, simply in digital format” Digital Body Language


What If…

…we considered the digital body language of our audiences to make design decisions?

…we used data like digital marketers do?

I Predict We will no longer measure learning by completions, but by engagement and

sentiment scores


Engagement vs. Completion

Company Response?

The manager had completed AHS food safety training with a mark of 96 per cent statement noted Webb had completed AHS food safety training with a mark of 96 per cent


Data-Driven Learning Design

Respond

Monitor

Discover Insights


Discover Insights

LMS IT Department Analytics

Most LMS track far more than most organisations explore. Consider: •  Time of day/day of

week most active •  Mobile vs. Laptop •  Search terms •  Incompletes

If your company has an intranet, inquire about some of these data points: •  Top searches •  Failed searches •  Pages most visited •  Time of day/day of

week most active

Simple internet searches will reveal loads of data on trends. Consider: •  YouTube or Vimeo

Analytics •  BuzzFeed •  Marketing blogs


Insights Strong preference for video Most active Fridays at noon

Most active Monday morning Strong preference for video

High mobile usage Text format preferred


Respond

Data Point Action

Top searches, Failed searches

•  Promote content aligned with search preferences

•  Determine if you need content to fill gap from failed searches

•  Use in performance consulting conversations

Pages most visited

•  Determine why the pages are most visited and what it tells you about what your learners value

•  Promote content on these pages


Respond

Data Point Action Time of day/day of week most active

•  Publish content at these key times for maximum exposure

Drop off rates •  Align length of content to be lower than attention threshold

Mobile vs. Laptop •  Use this data to make better decisions about media choices


Data-Driven Learning Design

Respond

Monitor

Discover Insights


Uncomfortable Truth?

The best performance consulting can be derailed by time, money, and

stakeholder opinion


5 Pieces of Data to Enhance Performance Consulting

▪  Intranet Search Terms

▪  Time of Day/Week Most Active

▪  Seat Time

▪  Downloads, Views, Likes, and Shares

▪  Mobile vs. Desktop


Story 1: Talking Heads

▪  Stakeholder wanted to be featured in seven minute video

▪  Similar videos: ~80% drop-off rates by 30 seconds

▪  Testimonial style videos by colleagues had significantly higher viewing lengths


Story 2: Content Gap

▪ SME and stakeholders want elearning solution on topic X

▪ Feed of intranet searches reveal topic X ranking #152

▪ Decision is made to pivot to build content against top 3 search terms and minor article on topic X


Cautionary Tale

▪ The average life expectancy in Canada is 81.24 years ▪ Most Canadians like maple syrup ▪ Therefore, maple syrup prevents people from living past 82


Only the Beginning

“It is not enough to make learning shorter, into micro-sized bites, to appease short

attention spans. The real magic is decoding digital body

language to captivate and engage learners, extending the impact and depth of learning”

Look at your data before designing Go beyond the LMS Be prepared to pivot Use data points to better manage SME and stakeholder conversations Digital Body Language


Final Thought

#CLOwebinar

Register for the next webinar!

Are You Ready to Embrace Informal Learning?

Wednesday, December 6, 2017

Webinars start at 2 p.m. Eastern / 11 a.m. Pacific

Register for all upcoming Chief Learning Officer Webinars at

clomedia.com/webinars

Recruiting & HR

GETTING STARTED USING LEARNING DATA TO IMPROVE DESIGN