5

Moving and Staging Data

Embed Size (px)

Citation preview

Page 1: Moving and Staging Data
Page 2: Moving and Staging Data

As more data moves to the Cloud, the issues become accessing data

that is in different silos and also managing the gross amount of data.

Some of those hosted applications have huge amounts of data. Think

about Twitter, Facebook or LinkedIn. They have huge amounts of

data. If an application is trying to access data in those applications

and the only alternative is to warehouse all that data, it becomes an

enormous challenge. Even with big data databases and technology, to

think about moving mass amounts of Facebook data into data storage

and then keeping it up-to-date is daunting. All you really want is

current records, current data to answer a specific question today.

Page 3: Moving and Staging Data

The real challenges are accessing not only data in multiple sources,

but also huge amounts of data. If you have to move all that data or

warehouse that data, it is a big problem. The better solution is to

access the data you need, when you need it, and not have to move all

the data into some massive warehouse.

A real example of this kind of problem is as follows. I have marketing

data in Eloqua, I have sales force automation data in Salesforce and I

have finance data in Zuora. What I would like to do is get analysis on

my last three campaigns, and all those applications are relevant.

Page 4: Moving and Staging Data

Eloqua has the marketing information, Salesforce has my information

about opportunities, what closed, what is still open what state they are

in, and Zuora has the financial information about what those accounts

are actually worth to the company. If I want to get an answer using

traditional integration products, I move all my Eloqua, Salesforce and

Zuora data into a warehouse, join it, and then ask questions of that

warehouse. If all I want is information on the current campaign, it

makes more sense to go and get current information from those three

applications, bring only that limited data across, analyze it in real time

and then get the answers I need today, to my current question.

Page 5: Moving and Staging Data

The other issue with warehousing the data is when you are building a

warehouse, you really need to have a preconception of the kinds of

questions you are going to ask, and obviously, of what date you going

to ask that question. That is not how the world works. The controller

goes into the CFO’s office and the CFO always asks a new question

like, “So what about this?” If that data was not warehoused, the

correct data was not there or it was not structured the way you could

actually get an answer to that question, then you are starting all over

again and rebuilding that warehouse. You are going back to IT and

getting them to load the data in a different way to answer today’s

question.