25
1 © Copyright 2012 EMC Corporation. All rights reserved. Cloud meets Big Data Roger Samdal Sr. Technology Consultant EMC Børge Wilhelmsen Business Developer EMC ATEA

Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

  • Upload
    others

  • View
    41

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

1 © Copyright 2012 EMC Corporation. All rights reserved.

Cloud meets Big Data

Roger Samdal Sr. Technology Consultant EMC

Børge Wilhelmsen Business Developer EMC ATEA

Page 2: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

2 © Copyright 2012 EMC Corporation. All rights reserved.

Source: 2010 IDC Digital Universe Study

IN THIS DECADE THE DIGITAL UNIVERSE

WILL GROW 44X FROM 0.9 ZETTABYTES TO 35.2 ZETTABYTES

Page 3: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

3 © Copyright 2012 EMC Corporation. All rights reserved.

90% OF THE DIGITAL UNIVERSE IS

UNSTRUCTURED

Source: 2011 IDC Digital Universe Study

Page 4: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

4 © Copyright 2012 EMC Corporation. All rights reserved.

Big Data is Changing Enterprise Storage

2009 2010 2011 2012 2013 2014

0

10

20

30

40

50

60

70

80

90

Source: IDC

File Based: 60.7% CAGR Block Based: 21.8% CAGR

By 2012, 80% of all storage capacity sold will be for file-based data

Big

Data

Sources

Page 5: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

5 © Copyright 2012 EMC Corporation. All rights reserved.

500TB

50PB 1PB 1

0P

B

100TB

HOW BIG ?

Page 6: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

6 © Copyright 2010 EMC Corporation. All rights reserved.

IT is changing (again)

Cloud Computing

Minicomputer

Mainframe

PC/

Microprocessor

Networked/ Distributed Computing

Page 7: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

7 © Copyright 2012 EMC Corporation. All rights reserved.

2003 AND BEFORE

2005

2007

2009

ENTERPRISE STORAGE

INFORMATION LIFECYCLE

MANAGEMENT

INFORMATION & VIRTUAL

INFRASTRUCTURE

JOURNEY TO THE CLOUD

2012

CLOUD &

BIG DATA

The EMC journey

Page 8: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

8 © Copyright 2010 EMC Corporation. All rights reserved.

Compute & Storage Convergence

Enterprise Applications Big Data Applications vM

otio

n

Fully Automated Storage Tiering (FAST)

Compute Tier

Storage Tier

Cloud Infrastructure

Page 9: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

9 © Copyright 2010 EMC Corporation. All rights reserved.

Back Office:

SAP:

(Revenue accounting, production accounting, plant operations, etc.)

Oracle E-Business Suite, JD Edwards, MS Dynamics, Lawson, others

Upstream MidStream

Technical Computing

Traditional Data Center IT

IT in the Oil & Gas Lifecycle

Downstream Midstream Upstream

Refine Distribute Retail Explore Develop Produce Transport Trade

Downstream

Traditional Computing Requirements

Page 10: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

10 © Copyright 2010 EMC Corporation. All rights reserved.

EMC in the Oil & Gas Lifecycle

Downstream Midstream Upstream

Refine Distribute Retail Explore Develop Produce Transport Trade

Page 11: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

11 © Copyright 2010 EMC Corporation. All rights reserved.

Upstream Trends

• Increasing survey sizes – New sensor technology

• Growing importance of pre-stack

in interpretation

• Mixing old 3D with new 3D (4D)

• Predictive Analytics

• On-demand virtualized private

cloud – Processing and interpretation from

the same infrastructure

– Global collaboration

Page 13: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

13 © Copyright 2011 EMC Corporation. All rights reserved.

Flexible

Ultra Scale

High Performance

Cloud meets Big Data

Page 14: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

TRADISJONELL ARKITEKTUR

16 TB LUN 16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN

Page 15: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

16 TB LUN 16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN 16 TB LUN

16 TB LUN

VÅR LØSNING – EMC ISILON

ETT filsystem, ETT volum

Opp til 15+ Petabyte >80% utnyttbar lagring

Høy, forutsigbar YTELSE ENKEL administrasjon og skalering

Page 16: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

CIFS NFS iSCSI

X-Noder NL-Noder S-Noder

Automatisk Sikring og Optimalisering

Enkel Administrasjon

ISILON

• Automatisk Tiering • Lastbalansering • Flash optimalisert • Detaljerte Analyser • Lokal- og Fjernsikring

Page 17: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

OPERATIVSYSTEMET

ISILON IQ NODER

INFINIBAND NETTVERK

BYGGEKLOSSER

Page 18: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

SKALERER PÅ 60 SEKUNDER

Page 19: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

NODE NETWORK CPU MEMORY DISK

NODE DISK NETWORK CPU MEMORY

NODE DISK NETWORK CPU MEMORY

NODE DISK NETWORK CPU MEMORY

NODE DISK NETWORK CPU MEMORY

NODE DISK NETWORK CPU MEMORY FORUTSIGBARHET, LAV RISIKO

DAGENS BEHOV

FORUTSIGBAR SKALERING

Page 20: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

TOM

TOM

TOM

TOM

TOM

FULL

FULL

FULL

FULL

BALANSERT

BALANSERT

BALANSERT

BALANSERT

BALANSERT

AUTOMATISK BALANSERING

Page 21: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

FJERNSIKRING AV DATA

Asynkron replikering over LAN eller WAN

Kun nye/ modifiserte blokker blir

replikert

Sync jobbene skeduleres i

GUI

STAVANGER OSLO

Page 22: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

Infi

nib

and

Co

re N

etw

ork

DATASET

DATASET DATASET

DATASET M

B/s

A

rch

ive

I/

Op

s

SMARTPOOLS - AUTOTIERING

Subgrupper av hardware innenfor samme cluster

Atributter dikterer initiell plassering

Automatisk flytting av data

Eliminerer migrering

Page 23: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

INSIGHTIQ – DETALJERT ANLYSE

Page 24: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

24 © Copyright 2010 EMC Corporation. All rights reserved.

VNX UNIFIED

VMAX

ISILON

DATA

DOMAIN

GREEN- PLUM

CENTERA

ATMOS

Page 25: Cloud meets Big Data - IDGweb.idg.no/app/web/online/Event/Energyworld/2012/Presentasjoner/E… · 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16 tb lun 16

25 © Copyright 2010 EMC Corporation. All rights reserved.