Interactive Parallel Data Visualization and Exploration

March 27, 2013

Mihai Budiu

Moises GoldszmidtJean-Philippe Martin

Mark ManasseAlex Andoni

Gordon PlotkinQingzhuo Luo (intern 2012)

Cluster-Based Visualization

Client GUI

Server workers

RMI

Partial renderings

Multi-coredata

extraction,transformation

rendering

Renderingoverlay

Data

Data size

Client GUI

Server workers

RMI

Partial renderings

Multi-coredata

extraction,transformation

rendering

Renderingoverlay

O(1)

O(1)

O(n)

~1MPx = O(1)

Composable Interfaces

IViewIView

IView IView

IViewIView

IView

LocalView LocalView

Server 1

ParallelView

ProxyView ProxyView

ParallelViewIView

IView

LocalView LocalView

Server 2

ParallelViewIView

Client

LocalViewIView

LocalView

Cluster-levelparallelism

Multi-core-levelparallelism

Computing “Small” Results

IDistributionIDistribution

IDistribution

LocalDistr LocalDistr

ServerParallelDistr

ProxyDistr

ParallelDistrIDistribution

IDistribution

Client

Plot

Plot

Plot PlotImage Image

Image

Image

Image

overlay

overlay

Result sizeindependent of the data size

Plot

Computing “Big” Results

IDistributionIDistribution

IDistribution

LocalDistr LocalDistr

ServerParallelDistr

ProxyDistr

ParallelDistrIDistribution

IDistribution

Client

Filter

Filter

Filter Filter

Filter

IDistributionIDistribution

IDistribution

LocalDistr LocalDistr

ParallelDistr

ProxyDistr

ParallelDistrIDistribution

IDistribution

Data nevermoves

result

A Language of Linear Transformations

• f : A → B, f(a + b) = f(a) + f(b)• Various + operations:– DB Tables, with union– Distributions, with union– Histograms, with pointwise addition– Transparent images with overlays– Timeseries, with union– Multisets, with pointwise addition– Top K values, with mergesort

Software Stack

Domain-specific App

Object transport

RMI

ILocal

IView IDistribution IHistogram IScatter

Plotting

IParallel IProxy

ITimeSeries IColorMap

Built-in structs (Distribution, View, Histogram, ScatterPlot, TimeSeries, ColorMap)

IPlot

Domain-specificData-structures

client side server side

X

Running on 700 cores