Narrative analytics white paper

Narrative Analytics: From Data, To Insight, To Action

2 | Narrative Analytics | A Narrative Science Whitepaper

Big Data Excitement and FrustrationThe point of computers was always to make us smarter. A core mission of computer science is to get us the

information we need to make important decisions and make us smarter about how we make them. This goal of helping

us, and making us smarter, is the driver that sustains work in Artificial Intelligence and is particularly important in the

growing world of Big Data.

The rise of Big Data has been an interesting combination of excitement and frustration. Excitement because everyone

knows that there is tremendous value to be found in the mass of data that is flowing through the world around us.

Frustration because the enormous investment – hundreds of billions of dollars globally – has resulted in shockingly few

success stories.

This result is really not surprising. The push toward Big Data began with the realization that we had been both passively

and actively amassing data, and we wanted to do something with it. This, in turn led to a massive investment in areas of

large-scale data analytics and machine learning to mine the value that we knew was there. The expansion of Business

Intelligence and visualization tools has all been driven by the need to glean something from the data in which we find

ourselves immersed. And, most importantly, the work has not only proceeded in a “bottom up” approach, but has also

been focused on “data as data” rather than the role it should play in decision-making. People took their eye off the prize –

to get insight – and focused instead on the gathering and management of data as the end goal.

But there is a solution. We can turn the machines that hold onto all of our data into something useful and meaningful.

We just need to teach them how to talk to us in a way that we understand. We can transform them into systems that

make us smarter.

AI That works for youProgress with Big Data has always been reliant on a human interpreting the data as it’s displayed. All of our investment in

automation and high-speed performance at massive scale comes down to one guy in a chair looking at a screen, and we’re

relying on him to figure out what is going on and communicate it to everyone who needs the information. Recently, we

have elevated this role and invented a new type of analyst called a “data scientist.” Ironically, we need this role because we

have been given tools that are astoundingly difficult to understand, even as they are being cast as “easy to use.” The tools

themselves have made it nearly impossible for even the most data literate people to extract anything meaningful from the

data at hand.

We need something better, something with real power behind it that can empower every one of us. It is time to take an

approach based on business needs and address what organizations really want from data. Instead of having to go to a

machine, build queries, do the analysis and find the meaning hidden in the data, the machine should deliver the insight,

the meaning and the story to us. To put it simply, the machine should tell us the story that it finds in the data. Research

in Artificial Intelligence has already proven that computers can deliver on this promise; it’s now a matter of applying the

science universally.

To do this, we need a new approach and a new kind of analysis: analysis that is focused on business and communication

goals as the driver for the examination and analysis of data; an approach that looks at how to deliver meaning and insight

in a form that makes natural sense to us, as narratives. No spreadsheets, no charts, no struggle - just the story.


Narrative aNalytics: Driven by the Story, Not the Numbers

Everything we want from the data that surrounds us, the information and the insight we need to help make

us smarter and make better decisions, can be derived from Narrative Analytics, a method that leverages the

tremendous potential of artificial intelligence to automatically transform data into meaning, insight and stories.

In the world defined by Narrative Analytics, the machine thinks like you. It considers what information you

need and drives all of its thinking and calculation to get you that information. Once it does this, it transforms

the results into clear, concise prose that you can simply read. You don’t have to struggle with the machine for

help. It comes to you with insight. It is smart enough to help make you smarter.

The central idea behind Narrative Analytics is simple: We need to know what is happening in the world

around us. In particular, we want to know about those aspects of the world that are important and relevant

to us. Obtaining this knowledge requires more than just exposure to the data. We need clear and instructive

communication that is focused on our needs, our interests and the decisions we have to make on a daily basis.

And computers, through Artificial Intelligence, are there to do all of this for us.

If we want a machine to communicate, we need to teach it to not only extract meaning from the data it

manages, but also to derive relevant insight from it. Fortunately, both of these tasks require that the systems

we build understand how to analyze data in order to extract meaning and insight.

The important distinction with this type of analysis is that it must be completely driven by the needs of the

narrative. We need to start with communication goals like, “I want to know about my logistical problems.” “I

want to know how my sales team is doing.” Or, “I want to know how my portfolio is performing.” These goals

drive the analysis. Any analysis of the data or even data collection is simply a waste of time if it doesn’t result in

some sort of communication or reporting that someone needs.

This is one of the crucial differences between the Narrative Analytics approach and a traditional data analytics

view. With data analytics, the algorithm is the driver, and data scientists always want more algorithms. From a

Narrative Analytics point of view, the story is the driver. Certainly, there are algorithms, analysis and data. But

they are all instrumental to telling the story, writing the report and communicating the insight.


Food

Beer

Wine

Liquor

Total Sales

Meals Tax

Gift Certificates Sold

TOTAL RECEIPTS

LESS

Discounts

COmp Food & Bev

G.C. Redeemed

Other

subtotal

MON15-Dec

$2,000.00

$200.00

$300.00

$325.00

$2,825.00

$141.24

$100.00

$3,066.24

$20.00

$40.00

$25.00

-

$85.00

TUE16-Dec

$2,100.00

$220.00

$200.00

$310.00

$2,920.00

$146.00

-

$3,066.00

-

$70.00

-

-

$70.00

WED17-Dec

$2,100.00

$410.00

$340.00

$360.00

$3,210.00

$160.50

-

$3,370.50

$35.00

-

-

-

$35.00

THURS18-Dec

$2,220.00

$410.00

$340.00

$345.00

$3,315.00

$165.75

-

$3,480.75

-

$22.00

-

-

$22.00

FRI19-Dec

$2,640.00

$200.00

$350.00

$360.00

$3,550.00

$177.50

-

$3,727.50

$60.00

-

-

-

$60.00

TOTALS21-Dec

$2,010.00

$2,100.00

$2,340.00

$2,460.00

$22,860.00

$1,142.99

$150.00

$24,152.99

$115.00

$207.00

$125.00

-

$447.00

PERCENT21-Dec

10.3%

10.4%

10.2%

10.8%

100.0%

5.0%

0.7%

0.5%

0.9%

0.5%

0.0%

2.0%

Data vS. DecisioN-MAkINgThis focus on the story and its impact on communication goals is only the first part of the equation. The real

impact is realized by what gets produced: a narrative that gives voice to the important meaning and insight from

the data and presents it in natural language.

The differences between the current approaches to data analysis and its presentation and narrative analytics are

not subtle.

But when you are presented with a narrative that expresses the information that is truly

important to you and relevant to the decisions you need to make, all you have to do is read it.

Sales at the bar went up this week with a huge spike on Sunday. Overall dinner sales stayed on par with last week, but lunch sales dropped a bit. If these trends persist, it might make sense to pull a waiter off the lunch shift and get another bartender on Sunday.

It comes down to this: a simple choice between data and decision-making. You can either choose to spend

time figuring it all out, or choose to have it easily and quickly explained to you so you know what the data

means and can make decisions based upon the output.

Visualizations allow you to actually see these numbers in a form

that may be easier to deal with, but you still have to interpret those

visualizations to pull out the relevant components. Again, you are

presented with the data but are left to figure out what is meaningful.

A spreadsheet allows you to see all of the numbers and perform the

necessary calculations to figure out what you need to know. But the

job of doing this work is yours, and if you are not completely fluid with

the numbers, you will be lost.

Data iNput: What data do I need.Quarterly earnings numbers, expected values, actual values and any confidence weights

on the expected values


ThE ANAToMy oF Narrative aNalyticsThe starting point of Narrative Analytics is always the story. From the story, we establish a set of

communication goals that reflect what it is we want to say. These, in turn, define the analysis we need to

perform in order to get to the truth that supports the communication goals and the story.

Consider a simple example, such as a quarterly earnings report. The goal of this report is to provide

information about how a company is doing based upon its current and historical earnings. So, the content

of the story needs to include a combination of history and a comparison against expectations. These

communication goals then define the analysis that needs to happen and the data that needs to be used. The

process looks like this:

The analysis serves the story and is, in fact, defined by the story. If I want my communication to satisfy my goals

and be true, then I need certain information. To get to that information I am going to have to perform specific

analysis of the data. And to perform that analysis, I need to have my data organized in a way that makes that

analysis possible. This is the case for any communication goal you can imagine. If you want to speak the truth,

then you need to ground your content in the right data and analysis.

Data aNalysis: How I can figure it out.Conduct a times series analysis against quarterly results

iNformatioN NeeDs: What I need to know.Current earnings compared to last quarter’s

commuNicatioN goal: What I want to say.Are the company’s earnings improving or on the decline


After establishing what it is you want to say, the next question follows: “What do I need to know in order to say the things I want to say?” This step is extremely important in that it defines the fact base that any communication

is going to utilize. For example, if I want to say how well a company is doing, I need to know if its earnings are on the

rise and if the rate of change is going up or down. If I want to say a game was a thrashing, I need to know if the margin

on the score was above a particular threshold. If I want to say how a salesperson is doing, I need to know how his

performance has changed over time. At this stage, I am defining my information needs.

This transition from communication goals to information needs is crucial in defining the analysis that must be done

in order to actually write a document. Once we know that we want to understand how a company’s earnings have

changed over the past five years, we need to perform a time-series analysis of earnings over that time frame and use

the earnings data that will support that analysis. At this point, we have identified what analysis is required and exactly

what data we need to support it.

Of course, once the nature of the analysis has been determined, the data requirements are set. For example, in order

to do time series analysis around a particular metric, you need to have the historical data associated with that metric.

If you are comparing two objects, you need the data associated with the elements that are going to be compared. By

starting with communication goals and identifying the information, analysis and data needed to support those goals,

we end up with a complete set of requirements to drive the narrative.

While this may seem strikingly obvious, it hasn’t been addressed to date in the world of Big Data. Today’s “bottom-

up” approach ignores the notion that if you want to say something about the world, you need to have the data that

will support what you want to say. With Narrative Analytics, however, the linkage between story, analytics and data is

fundamental – you need the data, analysis, and information in order to say what you want to say.


NArrATIvE structureThere is tremendous power to be gained by viewing the world of data and analytics through the lens of communication goals. This lens focuses the type of computation that needs to be run, and it provides

a needed link between the data we are collecting and the messages that we are trying to communicate. It is even

more powerful when we think of these elements working together. Individual communication goals work together

to allow us to craft more complex narratives that tell complete stories.

A powerful way to look at these relationships is through the eyes of standard story types. In particular, there are

recurring story types that define related collections of communication goals that can be packaged together. These

packages of communication goals have parallel analytics that are equally repeatable. In the world of machines,

repeatability translates into scale. Because so many of the stories, reports and narratives that we use to

communicate with each other are the same, the types of processing that are required to support their generation

is the same as well. There are always differences, but the reality is that even these differences can be captured,

characterized and then turned into the parameters for systems that generate the stories we care about.

There is a natural, human desire to believe that the exact opposite of this is true, a desire to believe that everything we do is unique and absolutely and fundamentally different than anything that anyone else does. But when it comes to communicating things about the world, this idea of uniqueness is the enemy. In fact, our ability to draw the variety of information about the world

into clear and coherent categories is part of what makes us intelligent in the first place. The Narrative Analytics

approach provides us with the ability to see the world through a clear lens.

Think, for example, about a performance review report.

If we are putting together a performance review, no matter the object of focus, we need to define a set of metrics

against which we will evaluate that object. Those metrics may have components associated with them, or drivers.

And those drivers may themselves have further components as well.

If we want to say something about how this object is doing, we have to consider how those metrics and drivers

are changing over time. If we want to say how it compares to other things that are related to it, we have to run

a set of comparisons. And if we want to say something about how this object can get better, we have to know

what can change and what the opportunities for change are (such as a driver that has fallen off over time or is

underperforming in comparison to similar objects).


We can do exactly the same sort of mapping if we are looking at how a salesperson is doing.

Or how a company is performing.

Or how a student, an exercise plan, or an investment portfolio is doing. No matter what the object, the pattern of

the story you need to hear, the analysis that is required, and the nature of the data behind it remains the same.

Certainly there are differences between these examples. But they are also predictable and regular. While the

metrics are different, they are still metrics and can be treated with the same sort of analysis. While the time scale

for the analysis might change from case to case, the cycles of how often we want to look at the changes and the

overall time frame we are viewing can simply become parameters to the analytics. And while the definition of a

comparison cohort may be different, it still ends up just being another parameter.

The similarities far outweigh the differences, and it is the similarities that make it possible to create models of

what needs to be considered when any content of this sort is generated. In fact, it is the similarity and our ability

to model it that gives us a language for even talking about the ways in which the content is different. Once we

are talking about “metrics,” “drivers,” “benchmarks” and “cohorts,” we can begin to characterize things like how a

version of a performance review prioritizes one of these elements over another, or how a report is only focused

on the positive versus the negative drivers. The commonality gives us this language to explain the differences.

This pattern of standardization of communication goals and information needs around a type of story is found

everywhere. Communication, at least the sort we are considering here, is about insight based on truth. And truth

is derived from the data. So it is absolutely understandable that these patterns of standard communication define

patterns of standardized analysis.

This may seem very abstract, but now think about this in terms of a single metro area’s real estate market. The

top-level metric is home sales. The drivers are new, existing and short sales. The time series analysis is month-

over-month. The comparison is to other similar sized metro areas. The opportunities are defined by seeing

growth in one driver in a comparable metro area that can be improved upon in our target. So the abstract

becomes the specific:

In the Decatur, IL market, homes sales declined last month by nearly 6%. A decline in new homes sales (down 8% from last month) was the biggest driver but existing home sales fell as well. This decline was more pronounced than the fall in sales felt by nearby Bloomington which saw a 4% drop.

Dave Schmitt’s overall sales performance is up a bit this month. He has been closing smaller deals at a higher than expected rate and still has larger deals in the pipeline. He remains in the middle of the pack in the Southwest Region.

Zebra Technologies Corp.’s (ZBRA) fourth quarter profit is a continuation of the four consecutive quarters of earnings growth we’ve seen over the last year. This quarter’s results put the company in the top 10% of manufacturing firms in terms of earnings.


Narrative Analytics makes us smarter The mission is not to just provide an answer. That would be useful, but it doesn’t help you communicate with

others. What you need is the answer and the reasoning behind the answer. The rationale for thinking something is

true is what makes you smarter.

Technology that makes us smarter should always be the goal. Not just technology that is smart in and of

itself, but technology that is able to communicate with people in a way that amplifies our own abilities rather

than supplants them. Narrative Analytics solves this issue. By using the goals of communication as the driver,

Narrative Analytics opens the door to a world of communication in which the machine takes on the

task of explaining what it knows to us in a way that is both rigorous and natural. It tells us the story of the data

and the insight that it contains.

All you have to do is read.

Data & Analytics

Narrative analytics white paper