Big data anuj

Anuj Pandey

12115021

WHAT IS BIG DATA?

... HOW BIG IS BIG?

“BIG DATA IS THE FRONTIER OF A FIRM'S

ABILITY TO STORE, PROCESS, AND

ACCESS (SPA) ALL THE DATA IT NEEDS TO

OPERATE EFFECTIVELY, MAKE DECISIONS,

REDUCE RISKS, AND SERVE CUSTOMERS.”

-- Forrester

“BIG DATA IS THE FRONTIER OF A FIRM'S

ABILITY TO STORE, PROCESS, AND

ACCESS (SPA) ALL THE DATA IT NEEDS TO

OPERATE EFFECTIVELY, MAKE DECISIONS,

REDUCE RISKS, AND SERVE CUSTOMERS.”

“BIG DATA IN GENERAL IS DEFINED AS HIGH

VOLUME, VELOCITY AND VARIETY

INFORMATION ASSETS THAT DEMAND

COST-EFFECTIVE, INNOVATIVE FORMS OF

INFORMATION PROCESSING FOR

ENHANCED INSIGHT AND DECISION

MAKING.”

-- Gartner

“BIG DATA IN GENERAL IS DEFINED AS HIGH

VOLUME, VELOCITY AND VARIETY

INFORMATION ASSETS THAT DEMAND

COST-EFFECTIVE, INNOVATIVE FORMS OF

INFORMATION PROCESSING FOR

ENHANCED INSIGHT AND DECISION

MAKING.”

-- Gartner

“BIG DATA IS DATA THAT EXCEEDS THE

PROCESSING CAPACITY OF

CONVENTIONAL DATABASE SYSTEMS. THE

DATA IS TOO BIG, MOVES TOO FAST, OR

DOESN'T FIT THE STRICTURES OF YOUR

DATABASE ARCHITECTURES. TO GAIN

VALUE FROM THIS DATA, YOU MUST

CHOOSE AN ALTERNATIVE WAY TO

PROCESS IT.”

-- O’Reilly

“BIG DATA IS DATA THAT EXCEEDS THE

PROCESSING CAPACITY OF

CONVENTIONAL DATABASE SYSTEMS. THE

DATA IS TOO BIG, MOVES TOO FAST, OR

DOESN'T FIT THE STRICTURES OF YOUR

DATABASE ARCHITECTURES. TO GAIN

VALUE FROM THIS DATA, YOU MUST

CHOOSE AN ALTERNATIVE WAY TO

PROCESS IT.”

“BIG DATA IS THE DATA CHARACTERIZED BY

3 ATTRIBUTES: VOLUME, VARIETY AND

VELOCITY.”

-- IBM


3 ATTRIBUTES: VOLUME, VARIETY AND

VELOCITY.”


4 KEY ATTRIBUTES: VOLUME, VARIETY,

VELOCITY AND

VALUE.”

-- Oracle


4 KEY ATTRIBUTES: VOLUME, VARIETY,

VELOCITY AND

VALUE.”

LET’S LOOK AT

BIG DATA IN A DIFFERENT WAY.

WHAT WAS YOUR

FIRST COMPUTER?

WHAT WAS ITS

“BIG DATA” LIMIT?

LET’S TRY AGAIN…

Byte : one grain of rice


Kilobyte : cup of rice

Kilobyte



Megabyte : 8 bags of rice

Megabyte




Gigabyte : 3 Semi trucks

Gigabyte





Terabyte : 2 Container Ships

Terabyte






Petabyte : Blankets Manhattan

Petabyte







Exabyte : Blankets west coast states

Exabyte








Zettabyte : Fills the Pacific OceanZettabyte








Zettabyte : Fills the Pacific Ocean

Yottabyte : A EARTH SIZE RICE BALL! Yottabyte









Yottabyte : A EARTH SIZE RICE BALL!

Our 1st Cmptr










Hobbyist

Desktop










Hobbyist

Desktop

Internet










Hobbyist

Desktop

Internet

Big Data










Big Data sources

Big Data sources

Large and growing files

(Big data files)

Mobile Devices

Microphones

Readers/Scanners

Science facilities

Programs/ Software

Social Media

Cameras

Activity DataAlmost every activity generate data.

Simple activities like listening to music,

reading a eBook, even our smart phone(how

we use), web browser(what we search) and

credit card company collects data on where

you shop and your shop collects data on what

you buy.

Conversation DataOur conversations are now digitally

recorded. It all started with emails but

nowadays most of our conversations

even that on Facebook n Twitter leave

a digital trail. Even many of our phone

conversations are now digitally

recorded.

Photo and Video Image

DataWe upload and share 100s of

thousands of photos n videos on You-

tube and other social media sites every

second. The increasing amounts of

CCTV cameras also take video images

and generate data.

Sensor DataWe are increasingly surrounded by

sensors that collect and share data.

Take your smart phone, it contains a

GPS and accelerometer that track

exactly where you are and the speed

and direction at which you are travelling.

The Internet of Things DataWe now have smart TVs, smart watches, smart

fridges, and smart alarms. The Internet of

Things connects these devices so that e.g. the

traffic sensors on the road send data to your

alarm clock which will wake you up earlier than

planned because the blocked road means you

have to leave earlier.

Types of tools used in

Big-Data

• Where processing is hosted?– Distributed Servers / Cloud (e.g. Amazon EC2)

• Where data is stored?– Distributed Storage (e.g. Amazon S3)

• What is the programming model?– Distributed Processing (e.g. MapReduce)

• What operations are performed on data?– Analytic / Semantic Processing

Latest technology such as cloud-Computing and Distributed System together with latest software and analysis approaches allows us to leverage all types of data and gain insight and add values.

TURNING BIG DATA INTO VALUE

Turning Big Data into Value:

The

‘Datafication’ of

our World;

• Activities

• Conversatio

ns

• Words

• Voice

• Social Media

• Browser logs

• Photos

• Videos

• Sensors

• Etc.

Analysing

Big Data:

Text analytics

Sentiment

analysis

Face

recognition

Voice

analytics

Movement

analytics

Etc.

Volume

Velocity

Variety

Veracity

Value

Three Characteristics of Big Data V3s

Volume

• Data quantity

Velocity

• Data Speed

Variety

• Data Types

1st Character of Big DataVolume

•Every minute we send 204 million emails, generate 1.8 million

facebook likes,278 thousand tweets, and upload 200,000

photos to fb.

•90% of data in world was created in past two years.

2nd Character of Big DataVelocity

• high-frequency stock trading algorithms reflect market

changes within microseconds.

• infrastructure and sensors generate massive log data in

real-time.

• on-line gaming systems support millions of concurrent

users, each producing multiple inputs per second.

3rd Character of Big DataVariety

• Big Data isn't just numbers, dates, and strings.

Big Data is also 3D data, audio and video, and

unstructured text, including log files and social

media.

• Big Data analysis includes different types of

data.

USES OF BIG DATA

(EXAMPLES)

Example 1

Better understand and target customers:

To better understand and target customers,

companies expand their traditional data sets

with social media data, browser, text analytics

or sensor data to get a more complete picture

of their customers. Using big data, Telecom

companies can now better predict customer

churn.

Example 2

Understand and Optimize Business

Processes:

Big data is also increasingly used to optimize

business processes. Retailers are able to

optimize their stock based on predictive

models generated from social media data, web

search trends and weather forecasts.

Example 3

Improving Health:

The computing power of big data analytics enables

us to find new cures and better understand and

predict disease patterns and links between

lifestyles and diseases. Big data analytics also

allow us to monitor and predict epidemics and

disease outbreaks.

Example 4

Improving Security and Law Enforcement:

Security services use big data analytics to foil

terrorist plots and detect cyber attacks. Police

forces use big data tools to catch criminals

and even predict criminal activity.

But the applications of Big

Data are endless!

BECAUSE…

THE SIZE

MATTERs!!!

THANKS

Engineering

Big data anuj