Pradeep K. Dubey, [email protected] Applications Pradeep K Dubey for Bob Liang Applications Research Lab Microprocessor Technology Labs Corporate

Pradeep K. Dubey, [email protected]

ApplicationsApplications

Pradeep K Dubey for Bob LiangPradeep K Dubey for Bob Liang

Applications Research LabApplications Research Lab

Microprocessor Technology LabsMicroprocessor Technology Labs

Corporate Technology GroupCorporate Technology Group

December 8, 2005December 8, 2005

2Pradeep K. Dubey, [email protected]

Application Research LabApplication Research Lab

Carole Dulong Pradeep Dubey

Tim Mattson Gary Bradski Horst Haussecker

Jim Hurley

Bob Liang


What is a killer app?What is a killer app?

““A reasonable man adapts himself to his environment. An A reasonable man adapts himself to his environment. An unreasonable man persists in attempting to adapt his unreasonable man persists in attempting to adapt his environment to suit himself …environment to suit himself …

… … Therefore, all progress depends on the unreasonable Therefore, all progress depends on the unreasonable man.”man.” -- -- George Bernard ShawGeorge Bernard Shaw

Replace “man” with “application”, and you get one definition of a Replace “man” with “application”, and you get one definition of a killer killer appapp, namely , namely that unreasonable application which succeeds in leaving that unreasonable application which succeeds in leaving its mark on the surrounding architectureits mark on the surrounding architecture. .

All architectural progress depends on such unreasonable apps!All architectural progress depends on such unreasonable apps!All architectural progress depends on such unreasonable apps!All architectural progress depends on such unreasonable apps!


Mining

Indexing

Recognition

Synthesis

Streaming

Graphics

Large dataset miningSemantic Web/Grid MiningStreaming Data Mining

Distributed Data MiningContent-based Retrieval

Collaborative FiltersMultidimensional IndexingDimensionality Reduction

Dynamic OntologiesEfficient access to large, unstructured, sparse datasets

Stream Processing

Multimodal event/object Recognition

Statistical ComputingMachine Learning

Clustering / ClassificationModel-based:

Bayesian network/Markov ModelNeural network / Probability networks

LP/IP/QP/Stochastic Optimization

Photo-real SynthesisReal-world animation

Ray tracingGlobal Illumination

Behavioral SynthesisPhysical simulation

KinematicsEmotion synthesis

Audio synthesisVideo/Image synthesisDocument synthesis

Data-data everywhere, not a bit of sense!Data-data everywhere, not a bit of sense!


Scene complexity: moderateLocal processing dominated

Media Evolution

Graphics Evolution

Mining Evolution

Scene complexity: largeGlobal processing dominated

Scene complexity: real-worldPhysical simulation dominated

Modality-specific streaming

Modality-aware transformation

Multimodal recognition

Dataset: static/structuredResponse: offline

Dataset: dynamic, multimodalResponse: real-time

Dataset: massive+streamingResponse: interactive

Upcoming Transition

Next Transition

Evolving towards model-based computingEvolving towards model-based computing

Workload convergence: multimodal recognition and synthesis over complex datasetsWorkload convergence: multimodal recognition and synthesis over complex datasets


What is a tumor? Is there a tumor here? What if the tumor progresses?

It is all about dealing efficiently with complex multimodal datasetsIt is all about dealing efficiently with complex multimodal datasets

Recognition Mining Synthesis

Images courtesy: http://splweb.bwh.harvard.edu:8000/pages/images_movies.html


Visual InputStreams

Find an existing model instance

What is …? What if …?Is it …?


Create a newmodel instance

What Killer app – grep, ctrl-c, ctrl-v?What Killer app – grep, ctrl-c, ctrl-v?

Model

Most RMS apps are about enabling interactive (real-time) RMS Loop or iRMSMost RMS apps are about enabling interactive (real-time) RMS Loop or iRMS

SynthesizedVisuals

Learning &Modeling

Graphics Rendering + Physical Simulation

ComputerVision

RealityAugmentation


iRMS Loop IllustrationiRMS Loop Illustration

Facial Muscle Activations:Compact motion representation,well suited for modeling and synthesis

Video Input Feature TrackingAnalytically Correct, Muscle-

Activated Human Head ModelPhysics-Based Deformable Tissue

(Finite Element Method)

User Interaction:Modified Muscle Activations

Video Output

User Interaction:Modified Physical Model

Source: E. Sifakis, I. Neverov and R. Fedkiw, “Automatic Determination of Facial Muscle Activations from Sparse Motion Capture Marker Data”, ACM SIGGRAPH, 2005 (to appear)


Virtual Reality, Games, Simulations …Virtual Reality, Games, Simulations …


Computer Vision – OpenCV 1M downloads!

Computer Vision – OpenCV 1M downloads!

Video Camera Input Desert Road identified

Parallel Body Tracker

Adding vision input asNatural Interface to Physics and Rendering

Visual Programming – rendering, physics, vision inputVisual Programming – rendering, physics, vision input


Visual ComputingVisual Computing

Closing the loop between computer vision, physical simulation, and photo-realistic rendering, and building an interactive system.

Closing the loop between computer vision, physical simulation, and photo-realistic rendering, and building an interactive system.

TrackingTracking Physics SimulationPhysics SimulationMulti-camera inputMulti-camera input RenderingRendering

Applications:• Games• Virtual Reality• Surgery and health care• Virtual dressing room• Movies and special effects• . . .

Applications:• Games• Virtual Reality• Surgery and health care• Virtual dressing room• Movies and special effects• . . .


Digital Libraries in the 90’sDigital Libraries in the 90’s Data Base extenders for media data managementData Base extenders for media data management

Server basedServer based

CBIRCBIR IBM QBICIBM QBIC Virage, etc.Virage, etc.

Good for Ad professionalGood for Ad professional Similarity for fade, wipe, etcSimilarity for fade, wipe, etc

Consumers wantConsumers want ““just find it”just find it” Natural user interfaceNatural user interface


DemosDemos


New Killer App?: New Killer App?:

There isn’t one There isn’t one -- Same old one: -- Same old one: grep, ctrl-c, ctrl-v

It’s a parallel world!It’s a parallel world!

Shall we look on the other side of theShall we look on the other side of the serial death valley serial death valley??

It’s an analog and non-linear world!It’s an analog and non-linear world!

Computers have digitized and linearized, but … Computers have digitized and linearized, but …

- Real-world problems are still largely non-linear and analog Real-world problems are still largely non-linear and analog

Almost infinite appetite for computational power, if … Almost infinite appetite for computational power, if …

- You reach a certain threshold needed for simulated interactions in real-You reach a certain threshold needed for simulated interactions in real-time.time.

SummarySummary


Thank You!Thank You!


Back-upBack-up


Tomorrow



Create a model instance

RMS: Recognition Mining SynthesisRMS: Recognition Mining Synthesis

Model-basedmultimodalrecognition

Find a modelinstance

Model

Real-time analytics ondynamic, unstructured,

multimodal datasets

Photo-realism andphysics-based

animation

TodayModel-less Real-time streaming and

transactions on static – structured datasets

Very limited realism


Visual InputStreams

Find an existing model instance



Create a newmodel instance

Emerging Workload Focus: iRMSEmerging Workload Focus: iRMS

Model

Most RMS apps are about enabling interactive (real-time) RMS Loop or iRMSMost RMS apps are about enabling interactive (real-time) RMS Loop or iRMS

SynthesizedVisuals

Learning &Modeling

Graphics Rendering + Physical Simulation

ComputerVision

RealityAugmentation


“Cognitive SQL”? “Cognitive SQL”?

SQL Today:Table

A B C D . . .Name Age Grade Salary

1 Dave 37 7 $82,000 . . .2 . . .3

. . .

USER:

“Give me all employees who make between $80K and $87K”SQL

API Dave, 37, 7, $82,000, …

. . .

USER:

“Find the variable that most describes compensation”

To

tal

Co

mp

ens

ati

on

Tenure with Company

IPO

Cognitive SQL :A B C D . . .

Name Age Grade Salary1 Dave 37 7 $82,000 . . .2 . . .3

. . .

SpectralClustering

Probabilistic ModelsInference

Clustering,Feature Selection Histograms

DiscriminativeClassifiers/Trees

ML

API

SQL

API


SummarySummary Design parallel algorithms with parallel computing mindset Design parallel algorithms with parallel computing mindset

from the beginning, not parellelizing serial algorithms. Even from the beginning, not parellelizing serial algorithms. Even “inherently” parallel applications such as Ray tracing and “inherently” parallel applications such as Ray tracing and computer vision requires workcomputer vision requires work

Potential killer app - To satisfy consumer’s requirement of Potential killer app - To satisfy consumer’s requirement of “Just Find it” with natural user interface “Just Find it” with natural user interface

Examples of (iRMS) Interactive Recognition-Mining-Examples of (iRMS) Interactive Recognition-Mining-Synthesis – the essence is the timely delivery of the Synthesis – the essence is the timely delivery of the knowledgeknowledge

Machine learning techniques will play an important role in Machine learning techniques will play an important role in help us extract useful knowledge from the massive amount help us extract useful knowledge from the massive amount of digital datasetof digital dataset

Explore the parallel programming patterns for each domain. Explore the parallel programming patterns for each domain. before we have a book “Parallel computing for dummies” . before we have a book “Parallel computing for dummies” .


the algorithms can be re-formulated as a sum over the data points and the sum can be broken up over one to many threads

AlgorithmAlgorithm Have summation form?Have summation form?

1.1. Linear regressionLinear regression YesYes

2. 2. Locally weighted linear regression Locally weighted linear regression YesYes

3. 3. Logistic regressionLogistic regression YesYes

4. 4. Gaussian discriminant analysisGaussian discriminant analysis YesYes

5. 5. Naïve BayesNaïve Bayes YesYes

6.6. SVM (without kernel)SVM (without kernel) YesYes

7.7. K-means clusteringK-means clustering YesYes

8.8. EM for mixture distributionsEM for mixture distributions YesYes

9.9. Neural networks Neural networks YesYes

10.10. PCA (Principal components analysis)PCA (Principal components analysis) YesYes

11.11. ICA (Independent components analysis)ICA (Independent components analysis) YesYes

12.12. Policy search (PEGASUS) Policy search (PEGASUS) YesYes

13.13. Boosting Boosting UnknownUnknown

14. 14. SVM (with kernel)SVM (with kernel) UnknownUnknown

15.15. Gaussian process regressionGaussian process regression UnknownUnknown

Machine Learning on Multi-CoreMachine Learning on Multi-Core


More powerful computer to help us discover new knowledge?More powerful computer to help us discover new knowledge?

Computer has been used to help Researchers discover new knowledge

“Pure Mathematics” - 4 color problem

We have computational geometry, computational chemistry, etc.

Will we have computational history?


ApplicationsApplications

“New” Innovative Applications?

“There is nothing new under the sun”

This talk: multi-core processing power and “new” techniques to make

some “old” applications work

Innovative Applications -> Afternoon Panel

Documents

Pradeep K. Dubey, [email protected] Applications Pradeep K Dubey for Bob Liang Applications Research Lab Microprocessor Technology Labs Corporate