Big Data and Security Aspects

7/28/2019 Big Data and Security Aspects

1/47

Geo Intelligence

13-14 Jun

New


2/47

Do lafzon ki hai DATA ki kahani...............

Ek hai ZERO....duja ha

2


3/47

Big Spatial Data

Security

WELCOME

3


4/47

BIG SPATIAL DATA has been with us for

various formsbut pretty invisible!!

4


5/47

5Anc

Riv

Engto tanapreyie

669


6/47

Basic Intro

Concepts

Perce

C

the 15 min route to THANK YO

6


7/47

An English professor wrote the words :

A Woman without her man is

nothingOn the chalk board and asked his students to punctuate

A Woman,without her man,is

A Woman: Without her, man is not

7


8/47

A greater scope of Geo Int info

New kinds of Geo data and analysis

Real time Geo information

Data influx from new technologies

Non traditional forms of Geo data

Large volumes of Geo data

The latest buzzword

Social media data

0 2 4 6 8 10 1

Series 1

DEFINING BIG SPATIAL DATA

8

How we understandit ?


9/47

Spatial data sets exceeding capacity ofcurrent computing systems

.to manage, processthe data with reasonab

due to Volume, Velocity, Variety andVeracity


BIG SPATIAL DATA9

0


10/47

10

DATA is Exploding in

Volume Velocity VARIETY

While decreasing in

Veracity

11


11/47


BIG SPATIAL DATAFindingactionable infoin Massivevolumes of bothstructured andunstructuredgeo data that isso large andcomplexthat itsdifficult to process

with traditionaldatabase andsoftwaretechniques

V

Veloc

VA

VERACITY

Data inMotion

Da

11


12/47

90% of data inthe world wascreated in thelast 2 years

U.S. drone aircraftsent back 24 years

worth of videofootage in 2009

GigabTerabyPetabExaby

13


13/47

* Estimated revenue FY 2013

growth ofgeospatial data is outpacingboth software and services and is setto become a major contributor to the

overall growth of the industry

13

14


14/47

100% security is a mytNo

B

14

Increasing attacksurface

15


15/47

The technology isready.

But arewe r?

15

16


16/47

16DISASTER RELIEF

FINANCIAL

FRAUD DETECTION

CALL CENTER REQUESTS

DISEASE SURVEILLANCE

INSURANCE

RETAIL

TELECOMMUN

UTILITIES

ECO-ROUTING

17


17/47

The otherof the

sidestory

17

18


18/47

Security challenges before we adBig spatial data

18

19


19/47

Distributed programming fram

Ek

19

20Utilise parallilism in computation & storage to process massiv


20/47

Distributed programming framework

Input file

Map IntermediateCombining Shuffle Output File

LocalReduce

Reduce

Mapper performscomputation& outputs akey/value pairs

20

Reducer combinesthe valuesbelonging to eachdistict key andoutputs the result

Utilise parallilism in computation & storage to process massivdata

MAP21


21/47

MAP REDUCE

FRAMEWORK

Splits the input data-set intoindependent chunks which areprocessed

in a completely parallel manner

Aggregate results from

performs a summary

Schedules and re-runs tasks

Splits the input

Moves map outputs to reduce inputs

Receive the results

Distributed programming framewo

21


22/47

So challenge is not storage but it is I/O sp

One Machine

4 i/o ChannelsEach channel : 100 MB/s

10 Machin

4 i/o ChanEach channel :

Read 1 TB

45 Min 4.5 Mi

23


23/47

Untrusted Mappers

Securing the dapresence of an

mapper

Distributed programming framework

23

24


24/47

NO SQL ISSUES

TWO

25


25/47

First off : the name

NoSQL is not NEVER SQL

NoSQL is not No To SQL

26


26/47

NoSQL

Is simply

Not Only SQL!!!!!

27


27/47

Redis

NoSQL DB are stillevolving with

respect to securityinfrastructure

28


28/47

Data storage & transaction

29


29/47

STORAGE TIERS

- Multi-tiered storage

- Necessitated by sc

- Different categor- Different types of

Data storage & transaction logs 30


30/47

Lower tier measecurity, loose

controls

Keeping track of datalocation

Data storage & transaction logs 31


31/47

INPUTVALIDATION/FILTE

32


32/47

How can we trust data ?

Validating data when sourceof input data is not reliable?

Filtering malicious data @BYOD

Input validation/filtering 33


33/47

REAL TIMEMONITORING

34


34/47

Humongous number ofalerts!!!!

False positives

Filtering malicious data @BYOD

REAL TIME MONITORING 35


35/47

Secure communication

36


36/47

End to end security ?

Data encryption : attribute based encryptmade richer

Secure communication 37


37/47

Granular audits

38


38/47

New attacks will keephappeningand to find

out we need detailedaudit logs

Missed true positiv

Granular audits 39


39/47

PRIVACY ISSUES

40


40/47

EG : How a retailer wasable to identify that ateenager was pregnantbefore her father knew

PRIVACY ISSUES

In the world of big data,privacy invasion is a busine

41


41/47

And...

WeAlso Have cloud with us?

42


42/47

At 1.4% in 2011-12Cloud was a very small

percentage of the total IT spend

43


43/47

Pace of Big Spatial Data adoption has been

Sluggish

44


44/47

There is unlikely to bea day soon in near

future when we have a

FINDTERRORIST

BUTTON

45


45/47

We have mostlybeen reactive tilldate..

46USE KERBEROS FOR NODE AUTHENTICATION(BUT WE KNOW ITS A PAIN TO SET UP)


46/47

(BUT WE KNOW ITS A PAIN TO SET UP)

STRINGENT POLICIES

STANDARD TO INTRA COUNTRY LAWS

EXHAUSTIVE LOGS

SECURE COMMUNICATION

STRINGENT POLICIES

47


47/47

Documents

Big Data and Security Aspects