1

BIG DATA MANAGEMENT

ZEBA KHANAMSenior Research Fellow, DLIS, Aligarh Muslim University

2

Outline

Introduction Characteristic of big data Data centers Focus about Hadoop technology Conclusion

3

Introduction

“Big Data is the next big thing in computing and generates value from very large datasets, but cannot be analyzed with traditional computing techniques”.

Managing and analyzing data of the customers have been a great benefit for the industries, but also a great challenge for them. Till the time they were having a handful of customers things were quite simple, but as soon as the number increased, complications in managing information raised along with.

Things were not confined to industries only, problems kept arising in medical science, research and development field, 3D simulations and designing etc. The large data from medical science gets deleted every week due to insufficient storage management. To face these challenges the term Big Data came into the scene. ‘Technically speaking, the process of handling big data encompasses collection, storage, transportation and exploitation.

( who are the generators of Big Data ?) Some Real Facts: New York stock exchange generates a 1 TB / day. Google processes 700PB/month Facebook hosts 10 billion photos taking 1PB of storage

4

Characteristics of Big Data

Big Data is often characterized using 4V’s

5

Data centers Such a big data management definitely requires a big big big data center for the

storage purpose. The idea that cloud computing means data isn’t stored on computer hardware isn’t accurate. Your data may not be on your local machine, but it has to be housed on physical drives somewhere -- in a data center

Some of the words about Data Centers were given by the experts are given below:-

Data centers are the brains of the Internet. The engine of the internet, it is a giant building with a lot of power, a lot of

cooling and a lot of computers. With its row upon row upon row upon row of machines, all working together

to provide the services that make all functions.

6

Management of Data Centers Through

Open source software Design for large amount of structured, unstructured data Implemented on the racks of commodity servers as a adopt cluster. It is designed to parallelize data processing across computing nodes to speed

computation and hide latency. Hadoop distributed file system

Motivation (Store the data on multiple machines) Architecture and Concepts (Run on commodity hardware) Inside (Software is intelligent enough to handle hardware failure) User Interface (Replicate the data)

Hadoop distributed file system service includes Namenodes Datanodes

7

Conclusion

Increase information explosion in every seconds indicates both Big Data challenge and Big Data opportunity . And the biggest problem rely is the storage of such data. The storage of your laptop could fit in your hand, but here they need something bigger. This is where the concept of data centers comes into picture.

The data centers work and compile all that information and send it back to the open internet again and all of this happens in milliseconds.

Some people consider the internet a cloud as it is floating around in the sky, but it’s not, it’s a real physical thing, internet is a real physical building interconnected with miles and miles of fibers and all of these buildings can talk to each other and share data back and forth.

These buildings are going to increase day after the day, so we should get ready to face several new upcoming technologies emerging in the near future.

8

Thank you