Analytics Novemberdecember 2012

8/11/2019 Analytics Novemberdecember 2012

1/50

H T T P : / / W W W . A N A L Y T I C S - M A G A Z I N E . O R G

ALSO INSIDE:

NOVEMBER/DEC EMBER 2012DRIVING BETTER BUSINESS DECISIONS

Image analytics:next really big data thing Distribution processing:math of uncertainty

Consulting & communication:achieving buy-in

DRILLING

with big dataDigital oil field helps oil & gas industryproduce cost-effective energy while addressingenvironmental concerns.

Executive EdMacys.com Kerem Tomaovercoming data, analychallenges
http://www.informs.org/http://www.informs.org/http://www.analytics-magazine.org/


2/50

The AIMMS PRO Platform

A New Revolution In Optimizatio

Ever thought about

...creating an optimization App Store

for business users?

...providing web access to your

optimization solutions?

...deploying in the cloud?

No? We have!

Check out our AIMMS PRO Platform at:

www.aimms.com/aimms-pro-platform

Norman Jerome, Aromatics Business Optimization Advisor - BP, plc.

"In the first three months since migrating to the AIMMS PRO platform,

we have seen our user base expand five-fold for one of our core

scheduling applications. And with so many new eyes reviewing this

application, the data are more reliable, and we are producing better

schedules than ever before."

Bellevue, WA, USA Haarlem, the Netherlands Singapore Shanghai, Chinawww.aimms.com [email protected]

AIMMS is a registered trademark of Paragon Decision Technology B.V. Other brands and their products are trademarks of their respective holder

www.aimms.com/aimms-pro-bp

1 | A N A L Y T I C S - M A G A Z I N E . O R G

Big Datas Big Daddy

I N S I D E S T O R Y

Can big data get any bigger?

The question reminds me of the

old joke about the bear in the woods.Of course big data is going to get

bigger. Today, and with apologies to

Sting, every breath you take, every

move you make, every bond you

break, every step you take seem-

ingly produces data. Multiply the

moves you make and the claims you

stake by the billions of other peo-

ple around the world and suddenly

youre talking really big data.

You might say Big Data has

many Big Daddies, all of whom are

prolic.

Wasnt it just a nanosecond ago

(in relative terms) that analysts every-

where were whining that if they only

had more data, they could solve all

manner of complex operational prob-

lems that were heretofore intractable?

Now those same analysts are drown-

ing in data and struggling to keep their

heads above the data deluge.

It turns out that irony really isa dish best served with cold, hard

facts. Just go easy on the side order.

Today, the analytics community

is basically scratching the surface in

terms of turning the deluge of data into

meaningful decision-making insight

on a widespread, corporate-world

scale. The sheer volume of availabledata is imposing enough, but then

the data has to be properly mined,

cleaned, analyzed and presented to

decision-makers or its going right

back on the scrap heap along with all

the other promising ideas that never

garnered C-level buy-in.

That, in a nutshell, is the theme

of several articles in this issue of

Analytics magazine. For example,

in his

cover story on the potential

of big data analytics and the digital

oil eld to revolutionize the oil and

gas industry, Adam Farris notes

that breaking into the oil and gas

industry is difcult for analysts be-

cause data scientists and petroleum

engineers not only dont speak the

same language, they dont appear

to be from the same planet. Yet the

potential for big data analytics to im-

prove energy production and safety

while protecting the environment isenormous.

Go gure.

PETER HORNER, EDITOR

[email protected]
http://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/http://www.aimms.com/http://www.aimms.com/aimms-pro-bphttp://www.analytics-magazine.org/mailto:[email protected]:[email protected]:[email protected]://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/aimms-pro-platformhttp://www.aimms.com/http://www.aimms.com/aimms-pro-bpmailto:[email protected]://www.analytics-magazine.org/


3/50

Discover the easiest, fastest, lowest cost and risk on

ramp to full-power data mining and predicve

analycs: Analyze your data in Microso Excel with

XLMiner soware, now from Frontline Systems.

You wontneed a stascs PhD, months of learning

me, or a huge budget for enterprise data mining

soware. But you will have the power of predicve

analycsand data visualizaonat your ngerps.

Sophiscated Data Mining Power.

XLMiner goes far beyond other stascs and forecasng

add-ins for Excel. It startswith mulple regression,

exponenal smoothing, and ARIMA models, but goes

further with regression trees, k-nearest neighbors, and

neural networks for predicon. It oers discriminant

analysis, logisc regression, k-nearest neighbors,

classicaon trees, nave Bayes and neural nets for

classifcaon, and associaon rules for anity analysis.

Praccal Data Cleaning and Exploraon.

XLMiner includes ulies for data sampling and

paroning, missing data handling, binning, and

transforming categorical data. You can apply princ

component analysis, k-means clustering, and

hierarchical clustering to simplify and cluster your

Help, Textbook and Instructor Support.

XLMiners extensive online Help introduces the da

mining methods and explains the opons in each di

With the popular textbook Data Mining for Busine

Intelligence built around XLMiner soware you

go from beginner to expert business analyst in a hi

demand eld. Instructors can take advantage of lo

cost classroom licenses, PowerPoint slides, examp

les, soluons to exercises, an online forum, and m

Download Your Free Trial Now.

Visit www.solver.com/xlminerto learn more, regis

and download a free trial or email or call us toda

Tel 775 831 0300 Fax 775 831 0314 info@solve

XLMINER: VISUAL DATA MININWorks with PowerPivot to Access 100 Million Rows in Ex

DRIVING BETTER BUSINESS DECISIONS

C O N T E N T S

FEATURES

IMAGES & VIDEOS: SOME REALLY BIG DATABy Fritz Venter and Andrew Stein

Sizing up the potential impact of prescriptive analytics driven by

proliferation of images and video.

HOW BIG DATA IS CHANGING OIL & GAS INDUSTRY

By Adam FarrisAdvent of the digital oil field helps produce cost-effective energy

while addressing safety and environmental concerns.

DISTRIBUTION PROCESSING ADDRESSES UNCERTAINTY

By Sam L. SavageNon-profit organization promotes standards for making rational,

auditable calculations based on probability distributions.

SOFT SKILLS: ART OF EFFECTIVE COMMUNICATION

By Gary CokinsHow to achieve corporate buy-in during the Twitter-influenced,

short-attention-span era.

SUCCESSFULLY OPERATIONALIZING ANALYTICS

By James TaylorA repeatable, efficient process for creating and effectively

deploying predictive analytic models into production.

SPORTS ANALYTICS: BASKETBALL GENOMICS

By William CadeEvaluation of performance: evolution of the official box score

reveals true on-court player values.

14

20

28

33

37

39


39

20

14

NOVEMBER/DECEMBER 2012

Brought to you by
http://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.solver.com/xlminermailto:[email protected]://www.solver.com/xlminerhttp://www.solver.com/xlminerhttp://www.analytics-magazine.org/http://www.informs.org/http://www.solver.com/xlminermailto:[email protected]://www.analytics-magazine.org/http://www.informs.org/http://www.analytics-magazine.org/


4/50

DRIVING BETTER BUSINESS DECISIONS Brought to you by


REGISTER FOR A FREE SUBSCRIPTION:

http://analytics.informs.org

INFORMS BOARD OF DIRECTORS

President Terry P. Harrison, Penn State University President-Elect Anne G. Robinson, Verizon Wireless

Past President Rina R. Schneur,Verizon Network & Technology

Secretary Brian Denton,University of Michigan

Treasurer Nicholas G. Hall, Ohio State University Vice President-Meetings William Bill Klimack, Chevron Vice President-Publications Linda Argote, Carnegie Mellon University Vice President- Sections and Societies Barrett Thomas, University of Iowa Vice President- Informati on Technology Bjarni Kristjansson, Maximal Software Vice President-Practice Activities Jack Levis, UPSVice President-International Activities Jionghua Judy Jin, Univ. of Michigan Vice President-Membership

and Professional Recognition Ozlem Ergun, Georgia Tech Vice President-Education Joel Sokol, Georgia Tech Vice President-Marketing,

Communications and Outreach E. Andrew Andy Boyd,University of Houston

Vice President-Chapters/Fora Olga Raskina, Con-way Freight

INFORMS OFFICES

www.informs.org Tel: 1-800-4INFORMS

Executive Director Melissa Moore Meetings Director Teresa V. Cryan Marketing Director Gary Bennett Communications Director Barry List

Headquarters INFORMS (Maryland) 7240 Parkway Drive, Suite 300

Hanover, MD 21076 USA Tel.: 443.757.3500 E-mail: [email protected]

ANALYTICS EDITORIAL AND ADVERTISING

Lionheart Publishing Inc., 506 Roswell Street, Suite 220, Marietta, GA 30060 USATel.: 770.431.0867 Fax: 770.432.6969

President & Advertising Sales John Llewellyn

[email protected] Tel.: 770.431.0867, ext.209 Editor Peter R. Horner [email protected] Tel.: 770.587.3172 Art Director Lindsay Sport [email protected] Tel.: 770.431.0867, ext.223 Advertising Sales Sharon Baker [email protected] Tel.: 813.852.9942

Analytics (ISSN 1938-1697) is published six times a year bythe Institute for Operations Research and the ManagementSciences (INFORMS). For a free subscription, register athttp://analytics.informs.org. Address other correspondence tothe editor, Peter Horner, [email protected]. Theopinions expressed in Analyticsare those of the authors, anddo not necessarily reect the opinions of INFORMS, its ofcers,Lionheart Publishing Inc. or the editorial staff of Analytics.Analytics copyright 2012 by the Institute for OperationsResearch and the Management Sciences. All rights reserved.

DEPARTMENTS

Inside Story

Executive Edge

Profit Center

Analyze This!

Forum

Conference Preview

Five-Minute Analyst

Thinking Analytically

1

4

7

9

11

42

44

47

42

44

The Excel Solvers Big Brother Makes Opmizaon

Easier, More Powerful and Far More Visual.

From its classic Solver Parameters dialog to its Ribbon,

Task Pane, charts to visualize the shape of funcons,

Constraint Wizard, model diagnosis, Help, and coverage

in leading management science textbooks, Risk Solver

Plaorm makes it easier to learn and use opmizaon.

Learn and Use Monte Carlo Simulaon, Decision

Analysis, and More with the Same Soware.

You can use its powerful opmizaon features alone

but Risk Solver Plaorm is also a full-power tool for

Monte Carlo simulaon and decision trees, with a Dis-

tribuon Wizard, 50 distribuons, 30 stascs and risk

measures, and a wide array of charts and graphs.

Use the Full Spectrum of Opmizaon Methods.

Fast linear, quadrac and mixed-integer programming is

just the starng point in Risk Solver Plaorm. SOCP,

nonlinear, non-smooth and global opmizaon are just

the next step. Easily incorporate uncertainty and s

with simulaon opmizaon, stochasc programm

and robust opmizaon all at your ngerps.

Go Beyond Single Opmizaon/Simulaon Solu

Dene parameters in seconds, run high speed mul

parameterized simulaons and opmizaons, cap

all the soluons, and produce instant reports, cha

and graphs, without programming. And you can do

even more in Excel VBA.

Free Trials and Support for Your Eorts at S olver.c

Download a free trial of Risk Solver Plaorm, and le

about the many other ways we can support your

learning and teaching eorts all at www.solver.c

TEACH AND LEARN OPTIMIZATION AND SIMULATIO

RISK SOLVER PLATFORM

http://www.informs.org/http://www.informs.org/http://www.analytics-magazine.org/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/mailto:[email protected]:[email protected]://www.solver.com/http://www.analytics-magazine.org/http://www.informs.org/http://www.analytics-magazine.org/


5/50

W W W . I N F O R M S . O R G

Its been more than a decade since the In-

ternet became a household shopping front.

We shop without leaving the sofa during a

commercial break due to the ease of a tablet

device. Our smartphone tells us how much anitem is on a competitive ecommerce site while

we are shopping in a retail store. If we like a

product we buy it instantly without waiting in a

checkout line.

One common theme behind all these ac-

tivities: we implicitly or explicitly create data

as we interact with these devices. We trans-

mit data to the cloud where it is stored. This

data (with our permission) then becomes part

of an analytic workow somewhere and comes

back to us with recommendations and/or of-

fers on what we should buy next, and the circle

of commerce continues.

Twenty years ago, 30MB of hard disk was

so immense that one didnt know what to do

with so much storage space. A gigabyte was

big data for an 8086 processor and DOS-

based Lotus 123 worksheets that were used.

The Internet did not exist, so the speed at

which data increased was contingent upon the

speed at which one could receive oppy disks

in the mail, 360KB at a time.

However, we still had the same workow

that we have today in relation to analytic ex-ercise. We sampled, ran descriptive statistics

and visualized the data. Based on our ndings,

we came up with a model or series of models

that best t the data, calibrated the model pa-

rameters based on simulations and completed

the version 0 of the analytics deliverable. As

we collected new data, we would revisit the

process and assess whether we needed a

new model or keep the existing one, making a

few parametric changes here and there. All the

data we had lled a spreadsheet back then.We could eyeball the data and see patterns

easily.

Similarly, when we sample data today, we

need efcient and fast visualization tools that

allow us to get to the nuggets quickly. Not

only is the data much larger, but the dimen-

sions over which the data is collected are nu-

merous. The belief that since we have more

data we do not need to sample is a awed

one. A critical assumption behind that thought

is that big data is accurately and comprehen-

sively capturing every known piece of informa-

tion there is to know about everything. Within

the modeling realm there is also the concept

of over-tting, data quality, etc., which still im-

plies sampling as a step in the analytic pro-

cess. However, a 1 percent sample of a 100TB

data is still large data.

RISING CUSTOMER EXPECTATIONS

As the time spans in which data is creat-

ed are compressed, customer expectations

of companies to provide information about

products and services such as availability, de-

livery, discounts in near real time, if not realtime, increase dramatically. To complicate

things even further, there is a new addition to

the data types that has added a twist to the

story: social media feeds. Semi- or un-struc-

tured data makes parsing, analyzing and in-

terpreting the data even more challenging, as

the data does not come in traditional columnar

setup. What is the value of a fans comment

on a businesss Facebook page? Who are the

social inuencers in a companys network of

fans and how can we use this information toreach to the right audience? How can a com-

pany understand which products are trendy or

what brands are in high demand from tweets?

After pre-processing and massaging the so-

cial data, these and similar questions can be

answered by using statistical tools and experi-

menting with ndings to see if any of those are

actionable.

Thanks to the cloud, we do not need to in-

vest a lot of money in hardware and software to

process all this data. Our ability of disseminat-

ing information quickly across different units

is constrained by the slowest link we maintain

in our network. If we are not comfortable with

moving and/or sharing a lot of data, we can

build our own cloud behind rewalls. Sophisti -

cated statistical and visualization software are

affordable as well. It can be only a matter of

4 | A N A L Y T I C S - M A G A Z I N E . O R G A N A L Y T I C S | N O V E M B E R / D E C E M B E R 2 012

Overcoming big datachallenges for analytics

BY KEREM TOMAK

As the time spans in

which data is created

are compressed,

customer expectations

of companies to provideinformation about products

and services increase

dramatically.

E X E C U T I V E E D G E
http://www.analytics-magazine.org/http://www.analytics-magazine.org/http://www.solver.com/


6/50


days before a company obtains more

than simple analytical capabilities. En-

terprise class operations still require

signicant investment, but even these

are relatively cheap.

These affordable technological ca-

pabilities enable the possibility of build-

ing a successful analytics function as if

the unit is a startup company within a

larger organization. This is one of the

many scenarios in which an analytics

team can be established. With buy-

in from senior management already

achieved and seed funding ready, the

main starting point is to hire an expe-

rienced analytics leader and empowerhim or her to build the roadmap to es-

tablish a proactive team.

ANALYTICS LEADERSHIP

Analytics leaders need to speak

the language of at least one quantita-

tive eld such as mathematics, statis-

tics, operations research or economics.

This is necessary to build a credible

leadership vertically and across the or-

ganization. Think of them as interpret-ers between the quantitative types and

execution teams. An efcient analytics

leader needs to understand the busi-

ness and trends, anticipate the chang-

es in requests for information and plan

ahead to build required capacity to

respond to the changes. Many analyt-

ics projects fail as either the informa-

tion is too overwhelming or the model

is too complex for a non-quantitative

end-user to comprehend and take an

action.

One of the key early steps is to

have a dedicated systems team that

is given the right funding and flex-

ibility to build the analytics systems

and support. Without a clear roadmap

toward scalable and robust systems

and processes, an analytics team is

limited in capabilities. Analytics lead-

ership needs to pass requirements to

the systems team or teams in orderto put the building blocks in place.

This requires a comprehensive un-

derstanding, exposure and hands-on

experience with data and analytics

systems and tools.

What does this flexibility enable

an analytics team to accomplish?

They can rapidly prototype automat-

ed, data-driven solutions in reporting,

product recommendations, personal-

ized offers and more. Being on thecutting edge of tools and techniques

enables the right data scientist to

have the freedom to invent. Business

units benefit from not only improved

internal processes to acquire the in-

formation they need much faster, but


The Opmizaon / Simulaon Library that Speaks the

Languages of Desktop and Web Compung.

No other callable library matches the exibility of

Solver SDK Plaorm from solving Excel models on

your server, to solving models created in your browser.

Build Your Model Faster in Excel or Visual Studio.

Use Risk Solver Plaorm or its subsets to build your

model in Excel. Use Microso Visual Studio to createyour model in C++, C# or VB.NET. Use C, R, MATLAB,

Java, PHP or JavaScript its your choice.

Use Opmizaon and Simulaon as a Web Service.

Run the Solver Server that comes with Solver SDK on

any PC or server, then solve your models over the

wire using our standard Web Service, SOAP and WSDL

protocols. Get real-me progress, or just disconnect

and retrieve the soluon later its your choice.

Solve Your Most Challenging Linear Mixed-Intege

Nonlinear and Stochasc Opmizaon Models.

Solver SDKs wide choice of built-in and plug-in So

Engines handle every type and size of opmizaon

problem, from Gurobi and Xpress-MP to Mosek an

KNITRO. Use the SDKs built-in simulaon opmiza

and stochasc programming algorithms, and supp

for mul-core CPUs.

Download a Free Trial atwww.solver.com.

Learn more and download a FREE trial version of R

Solver Plaorm, Solver SDK Plaorm, and our eigh

plug-in Solver Engines from Solver.com

THE MOST FLEXIBLE WAY TO DEPLOY YOUR MODE

SOLVER SDK PLATFORM

http://www.analytics-magazine.org/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/http://www.solver.com/mailto:[email protected]://www.solver.com/mailto:[email protected]://www.solver.com/http://www.analytics-magazine.org/


7/50

A membership in INFORMS will help!

visit http://join.informs.org to join online

New in 2013! Certification for Analytics Professionals

Online access to the latest in operations researchand advanced analytics techniques

Subscriptions to online and print INFORMS Publications

Networking Opportunities available at INFORMS Meetingsand Communities

Education programs around the world to enhance yourprofessional development and growth!

How will you stand out from the crowd?

2013INFORMS Renewa

available online

Renewing online is quand easy. Renew onlinto help us stay green.http://renew.informs


they also start to find novel ways to serve

their customers, to improve their product of-

ferings, and to understand where the bottle-

necks are within the organization, and the

list grows.

TESTING AND PRODUCTION OF PROTOTYPES

Finally, the path to testing and production

of working prototypes needs to be smooth and

supported by technology teams across differ-

ent business units. An analytics team needs to

be able to build dashboards and disseminate

the information through centralized systems

for everyone who needs that information to

use. They need to be able to test new algo-

rithms live or by using simulations to see what

needs to be tweaked and/or improved. Butmost importantly they need to work hand in

hand with agile technology teams to turn pro-

totypes into products that pass strict SLAs and

requirements to meet the performance criteria

of the production systems.

The road to taming big data passes through

people who are trained to handle the intrica-

cies of data, understand their business, ar-

ticulate what they see and, most importantly,

are enabled to feed their intellectual curiosity

by learning new tools and thinking outside the

box. Aligned with testing and delivery teams,

an analytics team with a keen focus on the

end-goal can be a major driver of a successful

business.

Kerem Tomak([email protected])is vice president ofMarketing Analytics at Macys.com. He is a member of INFORMS.


An analytics team

needs to be able to

build dashboards and

disseminate the information

through centralized

systems for everyone who

needs that information.

Subscribe to Analytics

Its fast, its easy and its FREE!Just visit: http://analytics.informs.org/
http://www.informs.org/Build-Your-Career/Analytics-Certificationhttp://www.informs.org/Apply-Operations-Research-and-Analytics/O.R.-Practitionershttp://www.informs.org/Apply-Operations-Research-and-Analytics/O.R.-Practitionershttp://www.informs.org/Sites/IPOLhttp://www.informs.org/Attend-a-Conferencehttp://www.informs.org/Attend-a-Conferencehttp://www.informs.org/Build-Your-Career/Professional-Educationhttp://www.informs.org/Build-Your-Career/Professional-Educationmailto:[email protected]://www.analytics-magazine.org/http://www.informs.org/http://www.informs.org/Build-Your-Career/Professional-Educationhttp://www.informs.org/Attend-a-Conferencehttp://www.informs.org/Sites/IPOLhttp://www.informs.org/Apply-Operations-Research-and-Analytics/O.R.-Practitionershttp://www.informs.org/Build-Your-Career/Analytics-Certificationhttp://www.analytics-magazine.org/mailto:[email protected]


8/50W W W . I N F O R M S . O R G

How does an organization move from

practicing little or no analytics to becoming

a world leader? The answer isnt simple. But

much can be gleaned from taking a look at

companies and industries that now employ

analytics at the highest levels. And one ofthe great success stories is that of the airline

industry.

Prior to 1978 the Civil Aeronautics Board

(CAB) regulated where, when and at what

price every airline could y. If an airline wanted

to offer a new ight, it had to le the appro-

priate paperwork then wait for a decision from

the CAB. Prices, which were identical across

carriers, were set by the CAB to reect the

airlines reported cost of service. The environ-

ment didnt encourage the industry to operate

efciently.

That situation changed with the Airline De-

regulation Act of 1978. Airlines were free to es-

tablish their own routes and schedules and to

set prices however they saw t. It was an era

of tremendous upheaval as airlines sought to

adapt to the competitive environment in order

to survive. Analytics proved to be a corner-

stone of the adaptation process.

Where exactly did analytics fit in? One

area was that of choosing the routes air-

craft would fly. If an airline serves 100 cities

and a typical route involves a plane visitingthree cities per day, there are roughly a mil-

lion different routes a single plane can be

assigned to. Of course, the actual problem

is far more complicated. All of the planes in

the fleet must be routed and scheduled so

that their arrival and departure times are co-

ordinated, thus allowing passengers to make

connections.

The problem of nding a single, reason-

able schedule is in itself a difcult task. But

to be competitive, airlines need to nd goodschedules schedules that ll ights with pas-

sengers. In the wake of deregulation, airlines

developed analytical models to predict pas-

senger demand, demand that was in turn fed

into large optimization models to generate the

most protable schedule.

Routing and scheduling are only part of

the operational problem. Pilots and flight at-

tendants must be assigned to staff flights.

The question for airlines is who to assign to

various flights. Simply finding an assignment

can be difficult since union contracts and

government regulations place restrictions on

what crews are allowed to do. A pilot, for ex-

ample, cant fly for 24 hours without mandat-

ed rest breaks. But among the many potential

crew assignments, some are more cost ef-

fective than others for example, those that

require fewer crews, reduce overnight stays

in hotels and other items. For large airlines,

crew costs run well into the billions of dollars

annually, and large optimization models are

routinely used to find crew assignments with

the lowest possible cost.One of the more interesting practices to

spring from the Airline Deregulation Act was

the practice of dynamic pricing. Airlines quick-

ly realized there were two primary classes of

flyers: business passengers, who were rela-

tively price insensitive, and leisure passen-

gers, who cared a lot about price. Airlines

were able to segregate these two groups by

introducing fare restrictions. A $200 ticket

might be available up to three weeks in ad-

vance, after which the price would go up to$300. Segregation of this type worked be-

cause business travelers frequently booked

only a few days ahead of departure while

leisure travelers were willing to book their

vacations further in advance to obtain lower

prices.

The practice worked well, and once Pan-

doras Box was open airlines rushed to take a

look inside. If raising the price three weeks be-

fore departure was successful, why not raise it

again to $600 with one week to go? If a plane

is nearly full four weeks before departure, why

wait another three weeks to raise the price to

$600? Why not do so immediately? Over time

the practice incrementally evolved to a point

where future demand was being forecast by

price point and the interaction between differ-

ent fares on routes using shared ight legs was


Embracing analytics

BY E. ANDREW BOYD

The practice worked well,

and once Pandoras Box

was open, airlines rushed

to take a look inside.

P R O F I T C E N T E R
http://www.informs.org/http://www.analytics-magazine.org/http://www.informs.org/http://www.analytics-magazine.org/http://www.sas.com/269


9/508 | A N A L Y T I C S - M A G A Z I N E . O R G

being accounted for. Dynamic pricing in

the airline industry (revenue manage-

ment in industry jargon) is one of the

most advanced applications of analyt-

ics in use today.

The rise of advanced analytics in

the airline industry can be attributable

to many factors, but two stand out in

particular. One was Robert Crandall,

the CEO of American Airlines from

1985 to 1998, who believed in the

power of analytics. Crandall was no

lover of mathematics, but he was no-

toriously competitive and believed an-

alytics could be used as a competitive

weapon. Under his leadership Ameri-can embraced analytics and became

the most feared and revered airline of

the 1980s and 1990s, employing hun-

dreds of analytics professionals who

had their hands involved in every as-

pect of running the airline.

Americans innovations caught the

attention of other carriers who realized

the value of analytics. And this was the

second factor leading to wide-scale

adoption of analytics: airlines needed

it to remain competitive. The practice

of analytics had become necessary to

stay in business.

Most industries havent undergone

the analytics conversion experienced

by the airlines. While its true that de-

regulation helped serve as a catalyst

for the airline industry, earth-shaking

events arent required to embrace ana-

lytics. All thats needed is recognizing

the competitive advantage it provides

and nurturing a sustained effort to im-

prove over time. American Airlines

started with an analytics group of eight

people doing what they could in an or-

ganization devoid of analytics. It tooktime to grow in size and sophistication,

but American was ahead of its competi-

tors. And in a period that saw the de-

mise of dozens of established airlines,

American survived and thrived. Its one

of the great analytics success stories,

and one we have much to learn from.

Andrew Boyd, senior INFORMS member andINFORMS VP of Marketing, Communications and

Outreach, has been an executive and chief scientist atan analytics rm for many years. He can be reached [email protected].

P R O F I T C E N T E R

A leading marketing companys scoring models used to take4.5 hours to process. Now, with high-performance analyticsfrom SAS,theyre scored in 60 seconds.

Get the relevant insights you need to make decisions in anever-shrinking window of opportunity and capitalize on

the complexity of big data to differentiate and innovate.

269What would you dowith an extra 269 minutes?

High-Performance Computing

Grid Computing

In-Database Analytics

In-Memory Analytics

Big Data

high-performanceanalytics A realgame changer.

sas.com/269 for a free white paper

EachSAScustomersexperienceisunique.Actualresultsvarydependingonthe customersindividualconditions.SASdoesnot guaranteeresults,andnothinghereinshouldbeconstruedasconstitutingan additionalwarranty.SASand allotherSAS InstituteInc.productor servicenamesareregisteredtradofSAS InstituteInc.in theUSAand othercountries.2012 SASInstituteInc.Allrightsreserved.S91566US.0412

Help Promote Analyt ics

Its fast and its easy! Visit: http://analytics.informs.org/button.html
http://www.analytics-magazine.org/mailto:[email protected]://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://www.sas.com/269http://analytics.informs.org/button.htmlhttp://www.sas.com/http://www.sas.com/269mailto:[email protected]://analytics.informs.org/button.htmlhttp://www.analytics-magazine.org/


10/50


Not long after nishing graduate school,

I found myself working at what used to be

known as an operations research consulting

rm (today, this company would be called an

analytics services provider or some such),

working full-time on a project for a large client.I still have a lot of scars from that project.

The core of the model that we were build-

ing was coded and implemented on a main-

frame computer. This meant that I often had

to struggle with writing a few crucial and con-

fusing lines of Job Control Language (JCL)

something that I never did quite master

and every time one of my compute jobs

crashed, I would receive a late-night phone

call at home.

Also, upon joining the project team, I had

inherited a largely undocumented SAS pro-

gram from a departing colleague, a mess of

spaghetti code that contained the guts of the

model that we were implementing. I spent

countless hours trying to sort it out and clean it

up without causing our nightly production runs

to crash (see late-night phone call above).

THE BIGGEST PROBLEM

By far the biggest problem, however, was

the lack of clarity about the projects purpose.

The project was sponsored and funded by the

clients IT organization. The actual business

groups who were expected to use the modelswere not at all clear on what the value proposi-

tion was for them, and we could see that our

project deliverables were being shoved down

their throats.

Not surprisingly, the business users we

worked with were motivated to nd problems

with what we were doing and they often

did. Some instability in the networking infra-

structure often prevented us from delivering

updated results, which generated one set of

complaints. Even when everything worked onschedule, our results (based on a daily snap-

shot of the systems state) would inevitably be

out of synch with the latest data that was avail-

able, which in turn produced a whole other set

of complaints. At core, there was a fundamen-

tal disconnect between the business users

(who believed they had asked IT for a tactical

reporting tool) and the IT organization (who

believed that we had been asked to deliver a

more sophisticated decision support system).

Meanwhile, we were pushed for political

reasons to get the model into production as

soon as possible, while the business users

kept nding reasons not too sign off on the de-

liverables and refused to use our solution at all

until it had been formally accepted. As such, a

huge amount of effort was spent designing the

perfect user interface, right down to the choice

of colors to be used to represent differen t kinds

of outputs, while the models core logic and

functionality was never seriously examined by

the clients who would ultimately have to use it.

PROJECTS VALUE PROPOSITIONThough nearly 20 years have passed since

then, this project came to mind again the other

day when an old friend of mine (lets call him

Doug) told me a familiar tale about one of

his recent projects. From the beginning, our

understanding was that the purpose of the

project was to build an alpha version, Doug

explained, something to demonstrate the

potential of the application while giving us a

chance to establish data connectivity, get a

bunch of technologies talking to each other,and use a sample of the operational data to

show that the optimization could actually pro-

vide signicant savings.

So far, this made great sense to me. In fact,

of late I have had conversations with many

people in the analytics eld about the value of

rapid prototyping for engaging potential stake-

holders, for demonstrating business value, and

for ensuring that the team really understands

the problem domain.

So what went wrong for Doug?

Turns out his project was also being led by

the IT organization, and that those folks did

not have any sense at all about the projects

value proposition. In addition, not long after

Doug and his team had begun working, some-

one somewhere in the IT chain of command

made an ad hoc decision to roll out the alpha


Even tragic projects canhave happy endings

BY VIJAY MEHROTRA

The business users we

worked with were

motivated to find problems

with what we were doing

and they often did.

A N A L Y Z E T H I S !
http://www.analytics-magazine.org/http://www.analytics-magazine.org/


11/50

W W W . I N F O R M S . O R G1 0 | A N A L Y T I C S - M A G A Z I N E . O R G A N A L Y T I C S | N O V E M B E R / D E C E M B E R 2 012

version of the software to a group of busi-

ness users around the county. Within the

clients IT group, this was interpreted as

a decision to treat this alpha version as a

production system and as such to hold

the consultants feet to the re as if tak-

ing delivery on commercial enterprise

software.

Doug and his team were perplexed

and distracted from what they thought

their focus should be. A great deal of time

was spent on minute details associated

with the graphical user interface, includ-

ing long discussions about the layout and

coloring of various output values, even

though there was no established accep-tance criteria (since the GUI had not been

viewed as a signicant part of the original

projects scope). Meanwhile, despite re-

peated attempts by Doug, no one on the

client side was willing to even look at the

results of the optimization until the entire

user interface design was signed off and

fully functional.

In fact, with the possible exception of

the original executive sponsor (who was

extremely busy and far removed from

the reality of the project), it seemed to

Doug that no one really understand how

the systems pieces (including the GUI,

a conguration rules engine, the optimi-

zation model, a relational database, and

java code for developing and deploying

the application) t together, or why the

project was being done in the rst place.

ABANDON SHIP

After more than a year on my project ,

I had become fed up. The partners in my

rm had made a great deal of money

from my billable hours on this project, but

I had come to understand that this was

clearly just part of the standard profes-

sional services business model. Howev-

er, I had nothing to show for my efforts

but lost sleep, a bunch of unhappy people

within the client organization, and a deep

sense that I was wasting their time and

my own. I left the project, and the rm,soon thereafter.

For many years, I felt quite smug about

this decision to abandon ship. Indeed, I

have since been told by several people

that I respect and trust that the willing-

ness to put your job on the line for what

you believe every single day should be

a core value for successful project lead-

ership. In any case, I was young, single

and free of debt, and walking out of my

employers ofces on that nal day, I felt

that I had very little to lose by leaving it all

behind.

After talk ing to Doug, however, Im

not so sure. Far older and wiser now

than I was then, Doug has worked

through this challenging project calmly

despite the many frustrations, commu-

nicating his concerns to his own man-

agement and doing his best to educate

people throughout the client organiza-

tion all the way along. Though the ini-

tial project was ultimately cancelled, the

clients executive sponsor has recently

re-engaged with Dougs company, rec-

ognizing her own part in the projects

failure and still believing in the poten-

tial business value that the optimization

might be able to provide.

While his initial project had appeared

to be a tragedy, or at least a black come-

dy, Dougs story may yet turn out to have

a happy ending. In any case, I plan to

stay tuned, and I hope to learn something

along the way.

Vijay Mehrotra([email protected]) is an

associate professor in the Department of Analytics andTechnology at the University of San Franciscos Schoolof Management. He is also an experienced analytics

consultant and entrepreneur, an angel investor in severalsuccessful analytics companies and a longtime memberof INFORMS.

A N A L Y Z E T H I S !

prizecall for

nominations

The Institute for Operations Research and the ManagementSciences annually awards the INFORMS Prize for effectiveintegration of Operations Research/Management Science(OR/MS) and advanced analytics into organizational decision

making. The award is given to an organization that hasrepeatedly applied the principles of OR/MS and advancedanalytics in pioneering, varied, novel, and lasting ways.

2013DEADLINE FOR APPLICATIONS IS DECEMBER 1, 2012

2013 COMMITTEE CHAIR:

Stefan KarischJeppesen

voice:+1 303.328.6389 e-mail: [email protected]

Tell us why the title should be yours.

log onto: www.informs.org/informsprize for more information

Which organization has the best O.R. department inthe world?
http://www.informs.org/http://www.analytics-magazine.org/mailto:[email protected]://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/informsprizehttp://www.informs.org/mailto:[email protected]://www.analytics-magazine.org/


12/50


13/50


was spent in discussions with senior man-

agement across various groups including

QA & Testing, Engineering, Development

& Engineering, Product and Release Man-

agement, among others. These discus-

sions are the rst opportunity to establish

priorities and set realistic expectations. In

addition to developing a set of key ques-

tions and project objectives, these discus-

sions provide valuable historic context on

the problem and can identify sensitive top-

ics that a consultant has to tread very care-

fully around.

For this project, the learning agenda

was distilled down to three key questions.

1. What is the likelihood that a customerwill encounter a bug as a function of

time in usage?

2. How do characteristics of the bug

impact that likelihood?

3. Which bug types are the most likely

to be encountered by a customer?

DATA COLLECTION & AGGREGATION

Collecting and aggregating all the

needed data can be one of the most chal-

lenging and time-consuming aspects of

an analytics initiative. The data needed for

this effort was spread across myriad data-

bases requiring many different resources

to fully source. Resources worked on this

stage for over two months, often requiring

multiple data extractions.

The denition of data should be broad-

ened to include information not contained

within a database. Resources intimate

with the software and how the customer

uses it can provide a wealth of knowledge

that can often be translated into quantita-

tive variables providing additional dimen-

sions to the analysis.

QA, ANALYSIS & DATA CLEANSINGOnce all the data has been extract-

ed, its important to plan for a proper

amount of time and effort to validate

and clean the data. This step is often

underestimated in analytics projects but

is one of the most critical as misleading

results can be produced if this work is

not done thoroughly. Almost all data will

have issues that need to be resolved.

Errors, incorrect values, unusual ob-

servations, extreme outliers and data

inconsistencies are quite common; ad-

dressing these issues will benefit both

the project at hand as well as other

applications that these data are being

used for. For this particular project, the

technical teams uncovered a host of

problems with a few of the databases

revealing that an uncomfortable level of

inaccurate data was being used to cre-

ate various reports distributed across

the business. A separate project was

kicked off to fix these issues and im-

prove the accuracy of these reports.

Validating and cleaning data gener-

ates some very rich conversations among

stakeholders and technical teams. This

stage is also helpful to set expectations

with stakeholders when the gaps and

limitations of the data can be more clearly

shown. These discussions help the stat-

istician better connect how the business

views the problem to how the available

data can be used to produce actionable

business metrics. Valuable insight into

the nature of the data is gleaned as the

statistician is able to examine the vari-

ability and correlation structure identify-

ing issues that may impact the statistical

modeling and analysis techniques to be

used.

F O R U M

Limited print copies are available to purchase for $45.00http://tutorials.pubs.informs.org

NOW AVAILABLE

FREE

!

2012T

utORia

lsavailable

ONLIN

E

forINF

ORMS

Annua

lMeeting

attende

es!

New Directions in Informatics,Optimization, Logistics, and ProductionPitu Mirchandani, volume editorJ. Cole Smith, series editor

INFORMS 2012 edition of the TutORials in Operations Researchseries is available online to registrants of the 2012 INFORMSAnnual Meeting. It will be made available online to all 2013INFORMS members on January 1, 2013.

To access the 2012 TutORialslog in at:

www.informs.org/tutorialsonlineand enter your INFORMS username and password.Request a no-obligation

INFORMS Member Benefits Packet

For more information, visit:http://www.informs.org/Membership
http://www.informs.org/http://www.analytics-magazine.org/http://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/tutorialsonlinehttp://www.informs.org/http://www.informs.org/http://www.analytics-magazine.org/


14/50

W W W . I N F O R M S . O R G13 | A N A L Y T I C S - M A G A Z I N E . O R G A N A L Y T I C S | N O V E M B E R / D E C E M B E R 2 012

STATISTICAL MODELING

Once the data has been adequately

cleaned and prepared, the statistical mod-

eling work begins. By this point, enough

analysis work and data examination should

have been done so the statistician has a

very clear idea of the technique and ap-

proach that will be used. The goal here is

to reduce the data to a mathematical ex-

pression with a component that provides

a description of the overall structure in the

data and a component that accounts for

the variability and uncertainty around that

structure. Its important to remember that

the goal of statistical modeling is to build

as simple of a model as possible that ad-equately describes the key features in the

data allowing the hypotheses/questions of

interest to be addressed without adding un-

necessary complexity.

A great quote that most statisticians

keep top of mind during this process to

help strike this balance is from one of the

pioneers of the science, George Box: All

models are wrong, but some are useful.

For time-to-event analyses, the mod-

eling technique needs to account for

the censored nature of the data. Many

statistical model forms that are common

in time-to-event analyses can handle

censored data and a variety of statistical

software that contains these techniques.

The author used the Minitab Statistical

Software, which is a software package

common among reliability engineers.

RESULTS

In most analytics projects, the more

advanced statistical analyses and models

are not shared beyond the core techni-

cal team doing the analysis work. These

models and analyses need to be trans-

lated into a variety of summary statistics

and graphical displays that communicatethe features in the data and are easy

to share across a broad range of audi-

ences. For this project, a technical report

containing a variety of graphical displays

and data tables was produced. Figure 1

is an example of one of the graphical dis-

plays produced in this project, and one

thats commonly used in time-to-event

analyses.

The graph displays the likelihood that a

customer will encounter a bug as a func-

tion of time (Note: the probability values are

not shown to protect the condentiality of

the work). This approach shows the rate at

which that likelihood increases over time.

The likelihood for ve different bug types is

displayed (A, B, C, D and E), allowing for a

comparison across the bug types. For ex-

ample, bug type E has the greatest chance

of being found by a customer while bug

type A and B have the least chance.

Management can use graphical dis-

plays such as these to help determine

the time in usage at which certain bug

types would have a likelihood of being

encountered beyond desired. In this proj-

ect, a certain level of likelihood was de-

cided upon by senior management and

displayed on the graph (shown by the

grey horizontal line). As can be seen, bug

types A and B dont reach that likelihood

until almost three years in usage, indicat-

ing that xing these bugs can be of lower

priority. Bug types D and E, on the oth-

er hand, reach that likelihood within the

rst few months of usage indicating that

these bugs have a high chance of being

encountered and should be top priority to

x before too many customer encounter

them.

Kevin Potcner([email protected]) isa director at Exsilon D ata & Statistical Solutions. Astatistician, Potcner has provided analytics consulting andtraining for a variety of industries including automotive,

biotech, medical device, pharmaceutical, nancialservices, software, e-commerce and retail. He holds amasters degree in applied statistics from the RochesterInstitute of Technology.

F O R U M

Figure 1: Graph displaying likelihood that a customer will encounter a bug as a function of time.

Joi n th e An alyti cs Sect ion of INF OR MS

For more information, visit: http://www.informs.org/Community/Analytics/Membership
http://www.informs.org/http://www.analytics-magazine.org/mailto:[email protected]://www.informs.org/community/analytics/membershipmailto:[email protected]://www.informs.org/community/analytics/membershiphttp://www.informs.org/http://www.analytics-magazine.org/


15/50


Sizing up the potential impact of prescriptive analytics driven by proliferation of images and video.

The human brain simultane-

ously processes millions of

images, movement, sound

and other esoteric informa-

tion from multiple sources. The brain is

exceptionally efcient and effective in its

capacity to prescribe and direct a courseof action and eclipses any computing

power available today. Smartphones now

record and share images, audios and vid-

eos at an incredibly increasing rate, forc-

ing our brains to process more.

Technology is catching up to the

brain. Googles image recognition in

Self-taught Software is working to

replicate the brains capacity to learn

through experience. In parallel, pre-

scriptive analytics is becoming far more

intelligent and capable than predictiveanalytics. Like the brain, prescriptive

analytics learns and adapts as it pro-

cesses images, videos, audios, text

and numbers to prescribe a course of

action.

THE FUTURE IS NOW

Google is working on simulating the

human brains ability to compute, evalu-

ate and choose a course of action using

massive neural networks.

The image and video analytics science

has scaled with advances in machine vi-sion, multi-lingual speech recognition and

rules-based decision engines. Intense in-

terest exists in prescriptive analytics driv-

en by real-time streams of rich image and

video content. Consumers with mobile

devices drive an explosion of location-

tracked image and video data. Lowering

costs have democratized cloud-based

high-performance computing. Andrew

McAfee and Erik Brynjolfsson in Har-

vard Business Review in October 2012

called this Big Data: The ManagementRevolution.

Image analytics is seen as a po-

tential solution to social, political, eco-

nomic and industry issues. Thirty years

of Intels Gordon E. Moores law and

Images & videos: really big data

BY FRITZ VENTER (LEFT) AND ANDREW STEIN

T

T H E N E X T B I G T H I N G

http://www.informs.org/http://www.technologyreview.com/fromthelabs/428910/self-taught-software/http://www.technologyreview.com/fromthelabs/428910/self-taught-software/http://www.technologyreview.com/fromthelabs/428910/self-taught-software/http://www.nytimes.com/2012/06/26/technology/in-a-big-network-of-computers-evidence-of-machine-learning.html?_r=0http://www.nytimes.com/2012/06/26/technology/in-a-big-network-of-computers-evidence-of-machine-learning.html?_r=0http://www.nytimes.com/2012/06/26/technology/in-a-big-network-of-computers-evidence-of-machine-learning.html?_r=0http://www.nytimes.com/2012/06/26/technology/in-a-big-network-of-computers-evidence-of-machine-learning.html?_r=0http://hbr.org/2012/10/big-data-the-management-revolution/ar/1http://hbr.org/2012/10/big-data-the-management-revolution/ar/1http://hbr.org/2012/10/big-data-the-management-revolution/ar/1http://hbr.org/2012/10/big-data-the-management-revolution/ar/1http://en.wikipedia.org/wiki/Moore%27s_lawhttp://en.wikipedia.org/wiki/Moore%27s_lawhttp://en.wikipedia.org/wiki/Moore%27s_lawhttp://en.wikipedia.org/wiki/Moore%27s_lawhttp://www.analytics-magazine.org/http://en.wikipedia.org/wiki/Moore%27s_lawhttp://hbr.org/2012/10/big-data-the-management-revolution/ar/1http://hbr.org/2012/10/big-data-the-management-revolution/ar/1http://www.nytimes.com/2012/06/26/technology/in-a-big-network-of-computers-evidence-of-machine-learning.html?_r=0http://www.nytimes.com/2012/06/26/technology/in-a-big-network-of-computers-evidence-of-machine-learning.html?_r=0http://www.technologyreview.com/fromthelabs/428910/self-taught-software/http://www.informs.org/http://www.analytics-magazine.org/


16/50


Harvard Business Schools Clayton

Christensens disruptive innovation

have created the current experience-

driven generation that is fully aware

of technologys potential to solve is-

sues plaguing these global domains.

On the consumption side, mobile

consumption of video is growing dra-

matically. Bandwidth is no longer a con-

cern. Prescriptive analytics is poised to

deliver relevant video to viewers be-

yond Netix algorithm for DVDs to rent

based on viewing interests.

IMAGE ANALYTICS: TECHNOLOGY

PROCESS

Image analyticsis the automatic al-

gorithmic extraction and logical analy-

sis of information found in image data

using digital image processing tech-

niques. The use of bar codes and QR

codes are simple examples, but in-

teresting examples are as complex

as facial recognition and position and

movement analysis.

Today, images and image sequenc-

es (videos) make up about 80 percent

I MAG E ANA L Y T I C S

nnnnnnnnnnnnnnnn

nnnnnnnnnnnnnnnn

nnnnnnnnnnnnnnnn

nnnnnnnnnnnnnnnn

nnnnnnnnnnnnnnnn

nnnnnnnnnnnnnnnn

nnnnnnnnnnnnnnnn

nnnnnnnnnnnnnnnn

Decision Sciences Institute

advancing the science and practice of decision making

The Decision Sciences Institute is a nonprofit

professional organization of researchers, managers,educators, and students interested in decision-

making techniques and processes in private and

public organizations. The Institute is an international

organization with over 3,500 members in 32 coun-

tries. The annual meetings and regional conferences

attract over 3,500 participants a year.

The Decision Sciences Institute

Is Committed to . . .

Research.A focus on the integration of research

in the art and science of managerial decision making

across traditional functional academic disciplines; an

international forum for presentation and discussion

of research.

Teaching.A forum for presentation and discussion

of innovative teaching; recognition of teaching

excellence and curriculum innovation.

Practice.An exchange of ideas between leading

professional practitioners and educators.

Benefits Members Receive:

Annual meeting. Members receive discounted fees to

attend, and the meeting draws over 1,500 professionals

together to share current thoughts on theoretical and

applied issues.

Decision Sciencesis a highly respected journal among

scholars and is subscribed to by over 1,000 libraries.

It seeks and publishes high quality, theoretical and

empirical articles.

Decision Sciences Journal of Innovative Education is

a high quality, peer-reviewed scholarly journal whose

mission is to publish significant research.

Decision Line, the Institutes news publication, is

published five times annually and includes feature

columns, as well as information on members, regions,

annual meetings, and placement activities.

Job Placement Servicesare offered throughout the

year, with position and applicant listings available via

the Internet. Also at the conference, special facilities

aid in position search activities.

www.DecisionSciences.org

Decision Sciences Institute43rd ANNUAL MEETING

NOVEMBER 17-20, 2012

Join us.

Log on to our website and complete

the membership application to begin

enjoying your benefits.

Regular membership: $160/year

Student membership: $25/year

Figure 1: Fast-growing consumption of mobile video.
http://www.analytics-magazine.org/http://hbr.org/web/tools/2008/12/disruptive-innovation-model-explainedhttp://hbr.org/web/tools/2008/12/disruptive-innovation-model-explainedhttp://www.businessinsider.com/bii-report-why-mobile-video-is-set-to-explode-2012-10#ixzz28p9HrrUQhttp://www.businessinsider.com/bii-report-why-mobile-video-is-set-to-explode-2012-10#ixzz28p9HrrUQhttp://www.businessinsider.com/bii-report-why-mobile-video-is-set-to-explode-2012-10#ixzz28p9HrrUQhttp://www.businessinsider.com/bii-report-why-mobile-video-is-set-to-explode-2012-10#ixzz28p9HrrUQhttp://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.decisionsciences.org/http://www.businessinsider.com/bii-report-why-mobile-video-is-set-to-explode-2012-10#ixzz28p9HrrUQhttp://www.businessinsider.com/bii-report-why-mobile-video-is-set-to-explode-2012-10#ixzz28p9HrrUQhttp://hbr.org/web/tools/2008/12/disruptive-innovation-model-explainedhttp://www.decisionsciences.org/http://www.analytics-magazine.org/


17/50


of all corporate and public unstructured

big data. As growth of unstructured data

increases, analytical systems must as-

similate and interpret images and videos

as well as they interpret structured data

such as text and numbers.

An image is a set of signals sensed

by the human eye and processed by

the visual cortex in the brain creat-

ing a vivid experience of a scene that

is instantly associated with concepts

and objects previously perceived and

recorded in ones memory. To a com-

puter, images are either a raster image

or a vector image. Simply put, raster

images are a sequence of pixels with

discreet numerical values for color; vec-

tor images are a set of color-annotated

polygons. To perform analytics on im-

ages or videos, the geometric encod-

ing must be transformed into constructs

depicting physical features, objects and

movement represented by the image

or video. These constructs can then be

logically analyzed by a computer.

The process of transforming big data

(including image data) into higher-level

constructs that can be analyzed is or-

ganized in progressive steps that each

adds value to the original information in

a value chain (see Figure 2) a conceptdeveloped by Harvard professor Michael

Porter. Prescriptive analytics leverages

the emergence of big data and computa-

tional and scientic advances in the elds

of statistics, mathematics, operations

research, business rules and machine

learning.

Prescriptive analytics is essentially

this chain of transformations whereby

structured and unstructured big data is

processed through intermediate repre-

sentations to create a set of prescrip-

tions (suggested future actions). These

actions are essentially changes (over a

future time frame) to variables that in-

fluence metrics of interest to an enter-

prise, government or another institution.

These variables influence target metrics

over a specified time frame. The struc-

ture of the relationship between a met-

ric and the variables that influence it is

a called a predictive model. A predictive

model represents detected patterns,

time series and relationships among

sets of variables and metrics. Predictive

models of key metrics can project future

time series of metrics from forecasted

influencing variables.

The first step in the prescriptive

analytics process transforms the ini-

tial unstructured and structured data

sources into analytically prepared data.

Although there are paral lels with stan-dard data-warehousing/ETL, this step

is different from that approachin that it

contends with the complexities of pre-

processing of unstructured data, as well

as structured data including databases,

narrative text files, images, videos and

sound.

For more details on the image analyt-

ics technology process,click here.

DEFENSE AND SECURITY DRIVING

DEMAND

The need to analyze data and pro-

actively prescribe actions is pervasive

in nearly every vibrant growth industry,

government and institutional sector.

This has created a vacuum, or demand,

I M A G E A N A L Y T I C S

Figure 2: Value chain of transformations.

The need to analyze

data and proactively

prescribe actions is

pervasive in nearly

every vibrant growth

industry, government and

institutional sector.
http://www.informs.org/http://www.analytics-magazine.org/http://tdwi.org/blogs/philip-russom/2011/07/analytic-data-prep-is-not-etl-for-data-warehousing.aspxhttp://tdwi.org/blogs/philip-russom/2011/07/analytic-data-prep-is-not-etl-for-data-warehousing.aspxhttp://analytics-magazine.org/november-december-2011/694-images-a-videos-really-big-data#sidebarhttp://analytics-magazine.org/november-december-2011/694-images-a-videos-really-big-data#sidebarhttp://analytics-magazine.org/november-december-2011/694-images-a-videos-really-big-data#sidebarhttp://tdwi.org/blogs/philip-russom/2011/07/analytic-data-prep-is-not-etl-for-data-warehousing.aspxhttp://www.informs.org/http://www.analytics-magazine.org/http://meetings.informs.org/analytics2013


18/50

1 7 | A N A L Y T I C S - M A G A Z I N E . O R G

for prescriptive analytics systems. De-

fense and security, as well as health-

care, are particularly good examples

of industries that are driving demand

for such systems.

The defense industry has pushed the

envelope for image processing, and it is

reected in the storage that is being pro-

cured by government. GovWin Consult-

ing reports that Defense agencies are

the largest spenders on a per-agency ba-

sis at the federal level for electronic data

storage. The Army, Navy and Air Force,

along with the Department of Defense,

account for 58.4 percent of all federal

spending for storage. GovWin indicates

that the drivers for this spend are big

data and full motion video.

The proliferation of captured data of

interest to defense and security comes

from four clear sources.

1. Predator drones gathering

intelligence via video and image

reconnaissance at reduced

risk as they seek out hostile

scenarios.

2. In-place surveillance cameras

increasingly prevalent in public

places, managed by federal, state

and local governments.

3. Stationary commercial and

institutional surveillance mounted


For submission information:

http://meetings.informs.org/analytics2013

Real-world use of descriptive, PREDICTIVE and prescriptive analy

Focus on BIG DATA, marketing analytics, forecasting

Most rigorous and REAL-WORLD analytics conference offered

Administration of Certified Analytics Professional exam

Not just what to do but HOW TO DO IT

Only INFORMS can provide an analytics and O.R. conference backed u

the best minds in industry and academia. Hand-picked speakers tak

through case studies on how analytics can maximize the value of you

data, driving better business decisions and impacting the bottom line

Present a Talk or Poster and SaveSubmission deadline: December 15, 2012

Present your work at the conference known for real-world analytics

and save 35-40% on registration. Case studies, best practice examples,

and academic research with a practitioner orientation are all welcome.

Open call for presentations, with special focus on:

Big data analytics

Supply chain management & logistics

Marketing analytics

Forecasting & risk management

Plus all other areas and topics within the business analytics and O.R. arena.

Thanks to our Sponsor

April 7-9, 2013Grand HyattSan AntonioSan Antonio, TX

Learn from the BEST about High-Impact ANALYTICS & O.R. APPLICATION

Predator drones gather intelligence via video and image reconnaissance.
http://www.analytics-magazine.org/http://govwin.com/bernaspi_blog/big-data-cloud-and-video/713070http://govwin.com/bernaspi_blog/big-data-cloud-and-video/713070http://govwin.com/bernaspi_blog/big-data-cloud-and-video/713070http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://govwin.com/bernaspi_blog/big-data-cloud-and-video/713070http://govwin.com/bernaspi_blog/big-data-cloud-and-video/713070http://www.informs.org/http://meetings.informs.org/analytics2013http://meetings.informs.org/analytics2013http://www.analytics-magazine.org/


19/50


in public places of business, the

workplace, hospitals and schools.

4. Consumer-created image and video

shared on YouTube, Facebook,

Twitter, blogs and other online social

media sharing/publishing sites.

While the demand drives proliferation,

it also presents a conict between safety

and privacy. People value surveillance

as a resource when a child is taken or

a loved one goes missing. On the other

hand, people see it as an invasion of pri-

vacy during everyday activities. Likewise,

people value sharing their personal pho-

tos with family and friends, but they areconcerned that their images and videos

may be anonymously processed and an-

alyzed to identify criminal activity. Where

is the ethical line of too much drawn?

And, do younger generations have the

same privacy-loss perspective?

Major cities around the world, from

London to Las Vegas, have cameras in-

stalled so densely that its nearly impos-

sible to move about the city without being

recorded. Keeping up with the installa-

tion statistics is almost impossible. The

availability of easy-to-deploy, consumer-

installed cameras is ubiquitous. This rate

of adoption for security video capture

makes an accurate assessment of how

much video is being recorded difcult.

We just know it is BIG.

Is all this surveillance coupled with the

potential of video/image analytics help-

ing? Research published in the Journal

of Experimental Social Psychology sug-

gests that increased surveillance only

increases our propensity to be Good Sa-

maritans, not reduce crime. Eric Jaffe

calls this the reverse the bystander ef-

fect in his recent article. In the end, sur-

veillance and image analytics does giveprovide data that can help ofcials pur-

sue criminal activity and pursue justice,

albeit ex post facto.

How does cost drive the demand for

video and image analytics? People expect

the nations defense and security effort

to be cost-effective. This means that the

country will move to a smaller but more

educated ghting force and at the same

time increase the use of remote sensing,

observation and monitoring tools. Simply

put, this means more image and video

capture or surveillance everywhere.

HEALTHCARE A PERFECT DOMAIN

The complexity of healthcare makes it

a perfect domain to explore the potential

for prescriptive analytics and imaging.

Healthcare has been a pioneer in captur-

ing rich imaging information and built da-

tabases to develop a variety of statistical

medical norms. The next step is to usethis image analytics to provide real-time

insight to healthcare providers during di-

agnosis and treatment.

The advances in medical science

come fast, and physicians have a dif-

ficult time keeping up with new proce-

dures, treatments and pharmacology

while they care for patients. Whether a

routine office visit, serious dise ase or an

emergency, prescriptive analytics inte-

grated in medical workflow promises to

improve the standard of care and speed

of diagnosis, treatment and recovery.

Its happening now. InScience Busi-

ness,Alan Kotok wrote about University

of Michigan researchers who adapted

computed tomography image analytics to


Parametric response mapping lung images.

Research suggests that

increased surveillance only

increases our propensity

to be Good Samaritans,

not reduce crime. Eric Jaffe

calls this the reverse the

bystander effect.

Help Promote Analyt ics

Its fast and its easy! Visit:http://analytics.informs.org/button.html
http://www.informs.org/http://www.analytics-magazine.org/http://en.wikipedia.org/wiki/Mass_surveillancehttp://en.wikipedia.org/wiki/Mass_surveillancehttp://en.wikipedia.org/wiki/Mass_surveillancehttp://en.wikipedia.org/wiki/Mass_surveillancehttp://www.theatlanticcities.com/arts-and-lifestyle/2012/06/could-surveillance-cameras-make-us-better-samaritans/2363/http://www.theatlanticcities.com/arts-and-lifestyle/2012/06/could-surveillance-cameras-make-us-better-samaritans/2363/http://www.theatlanticcities.com/arts-and-lifestyle/2012/06/could-surveillance-cameras-make-us-better-samaritans/2363/http://www.theatlanticcities.com/arts-and-lifestyle/2012/06/could-surveillance-cameras-make-us-better-samaritans/2363/http://sciencebusiness.technewslit.com/?p=11538http://sciencebusiness.technewslit.com/?p=11538http://sciencebusiness.technewslit.com/?p=11538http://sciencebusiness.technewslit.com/?p=11538http://analytics.informs.org/button.htmlhttp://sciencebusiness.technewslit.com/?p=11538http://sciencebusiness.technewslit.com/?p=11538http://www.theatlanticcities.com/arts-and-lifestyle/2012/06/could-surveillance-cameras-make-us-better-samaritans/2363/http://www.theatlanticcities.com/arts-and-lifestyle/2012/06/could-surveillance-cameras-make-us-better-samaritans/2363/http://en.wikipedia.org/wiki/Mass_surveillancehttp://www.informs.org/http://analytics.informs.org/button.htmlhttp://www.analytics-magazine.org/


20/50


diagnose chronic obstructive pulmonary

disease (COPD).

Advan ced medical decision-support

systems (MDDS) link massive knowl-

edge bases to multiple clinical data-

bases. These in turn are linked to a

pa

Documents

Analytics Novemberdecember 2012