The Data is the Model.Exploratory Data Analysis

We are in the fortune telling business.

Sensor Data ModelNow Future

The economics of fortune telling

accuracy

cost

Data Cost

Model Cost Data used to be expensive, and as a result, we ended up using complicated models that uses

relatively small data.

accuracy

cost

Data CostIs decreasing fast

Model Cost

As the data cost decreases fast, we need to strike a new balance between

model and big data.

I think most of you are as clueless as I am

Traditional M

odels are

Too complicated!

Traditional model cannot handle BIG DATA.

We used less than 10% of the data we have,

What about the rest 90%?

We need something different.

And the good news is, it is much simpler!

Exploratory Data Analysis

Let the data talk to you.

Don’t think,

Look!

Assess observed

assumptions

R=0.875

Collect more data

See if the assumption still holds