Upload
alluxio-inc
View
443
Download
0
Embed Size (px)
Citation preview
Alluxio (formerly Tachyon):Accessing Data Anywhere with Unified Namespace
Jiri Simsa
June 15, 2016 @ Alluxio Meetup (hosted by Intel)
About Me
• Software Engineer @ Alluxio, Inc.
• PMC Member and Maintainer of Alluxio Open Source Project
• Ph.D. from Carnegie Mellon University (Parallel Data Lab)
• Worked at Google before joining Alluxio
• Twitter: @jsimsa, Github: jsimsa
2
Outline
• Motivation
• Unified Namespace
• Use Cases
• Demo
3
Big Data Ecosystem
4
Big Data Ecosystem
5
Big Data Ecosystem
6
Alluxio Benefits
• Future-proofing your applications–applications can communicate with different storage systems, both existing and new, using the same namespace and interface–seamless integration between applications and new storage systems enables faster innovation
• Enabling new workloads–one-time effort to enable an application to access many different types of storage systems and a storage system to be accessed by many different types of applications
7
Outline
• Motivation
• Unified Namespace
• Use Cases
• Demo
8
Unified Namespace
an abstraction that makes it possible for
applications to access different storage
systems through the same interface
9
Transparent Naming
•Operations over persisted Alluxio objects
mapped transparently to underlying storage
•Alluxio paths are preserved in storage layer
Alluxio Storage System (HDFS, S3, …)
alluxio://host:port/
Data Users
Reports Sales Alice Bob
hdfs://host:port/
Data Users
Reports Sales Alice Bob
10
Multiple Storage Systems
•Unified namespace for multiple data sources
•Sharing of data across storage systems
•API for on-the-fly mounting / unmounting
AlluxioStorage System A
alluxio://host:port/
Data Users
Alice Bob
hdfs://host:port/
Users
Alice Bob
Storage System B
s3://host/bucket
Reports SalesReports Sales
11
Outline
• Motivation
• Unified Namespace
• Use Cases
• Demo
12
Multiple Storage / Compute
13
Changing Storage Backend
14
Changing Storage Backend
15
Outline
• Motivation
• Unified Namespace
• Use Cases
• Demo
16
Resources
• Alluxio Project: http://www.alluxio.org
• Development: https://github.com/Alluxio/alluxio
• Meet Friends: http://www.meetup.com/Alluxio
• Alluxio, Inc.: http://www.alluxio.com
• Contact us: [email protected]
Backup Slides
18
Architecture Overview
19