“Fifty Years of Supercomputing: From Colliding Black Holes to Dynamic Microbiomes

to the Exascale”

Director’s Distinguished Lecture National Center for Supercomputing Applications

University of Illinois at Urbana-ChampaignSeptember 16, 2016

Dr. Larry SmarrDirector, California Institute for Telecommunications and Information Technology

Harry E. Gruber Professor, Dept. of Computer Science and Engineering

Jacobs School of Engineering, UCSDhttp://lsmarr.calit2.net

1

Abstract

For the last thirty years, NCSA has played a critical role in bringing computational science and scientific visualization to the national user community. I will embed those three decades in the 50 year period 1975 to 2025, beginning with my solving Einstein's equations for colliding black holes on the megaFLOPs CDC 6600 and ending with the Exascale supercomputer. This 50 years spans a period in which we will have seen a one trillion-fold increase in supercomputer speed. Today we see the rise of data science and a new generation of architectures that balance traditional HPC with data analytics and pattern recognition. I will illustrate that with my current research into the microbial ecology dynamics within our human body. Finally, I will describe a number of trends in the decade to come, in which brain-inspired computing will become more prevalent in both software and hardware.

Personal Reflections on a Half Century of Supercomputing

• This Lecture Will Not Attempt to Cover in Detail the History of:– Supercomputers– NCSA– NSF’s HPC Program– Black Hole Collisions– Microbiomes– Scientific Visualization– Cyberinfrastructure– March to the Exascale– Machine Learning and Artificial Intelligence

• Rather I Will Discuss Significant Trends I Have Been Involved With

Forty Years of Computing Gravitational Waves From Colliding Black Holes – One Billion Times Increase in Supercomputer Speed!

1977

L. Smarr and K. EppleyGravitational Radiation Computed

from an Axisymmetric Black Hole Collision 40 Years

2016

LIGO ConsortiumSpiral Black Hole Collision

MegaFLOPS PetaFLOPSHolst, et al. Bull. Amer. Math. Soc 53, 513-554 (1916)

Documenting The Unmet Supercomputing Needsof A Broad Range of Disciplines Led to the NCSA Proposal to NSF

1982 1983

1985

NCSA Numerical Astrophysics GroupUsed NCSA Supercomputers to Explain Cosmic Phenomena

Mike Norman, Charles Evans, Roger Ove, John Hawley, Dean Sumi, Rob Wolff, Larry Smarr

Gas Accretion Onto a Black HoleCreates “Exhaust Channels”

Cosmic JetsEmerge from

Galactic CentersCollision of Neutron Stars

Exponential Increases in Supercomputer Speed and Visualization Technology Drive Understanding and Applications

Source: Donna Cox, Robert Patterson, Bob Wilhelmson, NCSA

1987

2005

Showed Thunderstorms Arise from Solving Physics Equations

Vastly Higher Resolution Uncovers Birth of Tornadoes

From Scientific Visualization of Supercomputing Science to Movie Special Effects

http://access.ncsa.uiuc.edu/http://movies.warnerbros.com/twister

www.jurassicpark.com; www.jamescameron.orgwww.cinemenium.com/perfectstorm/

NCSA 1987

1993

1996

2000

Computer GraphicsFrom NCSA to ILM

1991

Stefen Fangmeier  

The Electronic Visualization Lab’s CAVE Virtual Reality System:Fully Immersive Science and Fantasy Worlds

CAVE conceived in 1991 by Tom DeFanti and Dan Sandin (EVL co-directors) and implemented by Carolina Cruz-Neira (Ph.D. student)

Crayoland

Colliding Galaxies QUAKE II

The CAVE

• EVL Invents ‘91• Debuts SIGGRAPH ’92• National Access NCSA

‘93

Industry Has Been a Close Collaborator in NCSA Advances:Example - Caterpillar

Real Time Linked Virtual Reality and Audio-Video Between NCSA, Peoria, Houston, and Germany

www.sv.vt.edu/future/vt-cave/apps/CatDistVR/DVR.html1996

Distributed Virtual Reality for Global-Scale Collaborative Prototyping

Visual Supercomputing Goal:Make Pattern Recognition Analysis as Powerful as Simulation

Colliding Galaxies (Smithsonian IMAX)-Donna Cox, Bob Patterson, NCSA-From “Cosmic Voyage”-Nominated for Academy Award 1997

• Virtual Director in CAVE• 1000 Hour SDSC Supercomputer Run to Generate Data• Tens of Thousands of Hours of NCSA SGI Time to Render Data• Cross-Country Transfer to IMAX Film of Massive Amounts of Data

Sponsored by Motorola for the Smithsonian

NCSA’s National Supercomputing LeadershipWith Blue Waters

March 28, 2012November 5, 2008

NCSA Blue Waters Continues NCSA’s Exponential Growth in Supercomputer Power

NCSA Blue Waters

2012

10,000,000

1,000,000

Smarr 2000 Slide

Twenty-Five Years from Bleeding Edge Research to Consumer Mass Market

• 1990 Leading Edge University Research Center-NCSA– Supercomputer GigaFLOPS Cray Y-MP ($15M)– Megabit/s NSFnet Backbone

• 2015 Global Market for Smartphones– Gigaherz Smartphones ($150)– Megabit/s Wireless Cellular Internet

But, NSF Blue Waters Petascale Supercomputer Is Over 1 Million Times Faster than Cray Y-MP!

Enormous Growth in ParallelismProcessors: Y-MP 4, Blue Waters 400,000

www.ncsa.uiuc.edu/BlueWaters/system.html

Big Data Analytics is a Key Componentof The Future of Supercomputing

Global Scientific Instruments Will Produce Ultralarge Datasets Continuously Requiring Dedicated Optic Fiber and Supercomputers

Large Synoptic Survey Telescope

www.lsst.org/sites/default/files/documents/DM%20Introduction%20-%20Kantor.pdf

Tracks ~40B Objects,Creates 10M Alerts/Night

Within 1 Minute of Observing

2x40Gb/s

There are 100 billion stars in the Andromeda galaxy…

…and 100 billion galaxies in theknown universe.

It’s a microbial world…

…there are 100 million times as many bacteria on Earth as stars in the universe. Microbiology is the ultimate Big Data science!

Most of Life’s Evolutionary Time Was in the Microbial World

You Are Here

Source: Carl Woese, et al

Tree of Life Derived from 16S rRNA Sequences

Carl Woese Mentored Me

on the MicrobiomeWhile I Was at UIUC

Your Body Has 10 Times As Many Microbe Cells As DNA-Bearing Human Cells

Your Microbiome is Your “Near-Body” Environment

and its CellsContain 100x as Many DNA Genes

As Your Human Cells

Inclusion of the “Dark Matter” of the BodyWill Radically Alter Medicine

As a Model for the Precision Medicine Initiative, I Have Tracked My Internal Biomarkers To Understand My Body’s Dynamics

My Quarterly Blood DrawCalit2 64 Megapixel VROOM

Longitudinal Time Series RevealedOscillatory Behavior in an Immune Variable Which is Antibacterial

Normal Range<7.3 µg/mL

124x Upper Limit for Healthy

Lactoferrin is a Protein Shed from Neutrophils -An Antibacterial that Sequesters Iron

TypicalLactoferrin Value

for Active

Inflammatory Bowel Disease

(IBD)

To Map Out the Dynamics of Autoimmune Microbiome Ecology Couples Next Generation Genome Sequencers to Big Data Supercomputers

Source: Weizhong Li, UCSD

Our Team Used 25 CPU-yearsto Compute

Comparative Gut MicrobiomesStarting From

2.7 Trillion DNA Bases of My Samples

and Healthy and IBD Subjects

Illumina HiSeq 2000 at JCVI

SDSC Gordon Data Supercomputer

The Supercomputer Converts Tens of Billions of DNA Fragments Into Relative Abundance of Hundreds of Microbial Species

Average Over 250 Healthy PeopleFrom NIH Human Microbiome Project

Note Log Scale

Clostridium difficile

Calit2’s Qualcomm Institute Has Developed Interactive Scalable Visualization for Biological Networks

20,000 Samples60,000 OTUs

18 Million Edges

Runs Native on QI’s 64Million Pixels Tiled Wall

Software Written by Philip Weber, Calit2’s QI

Visually Analyzing Our Supercomputing Data Reveals Vast Changes in the Human Gut Microbiome in Health and Disease

IlealCrohn’s

Healthy

UlcerativeColitis

www.sandia.gov/~smartin/presentations/OpenOrd.pdf

Source: Philip Weber,

QI, UCSD

President Obama Announces National Microbiome InitiativeMay 13, 2016

Complexity of Computing First Gut Microbiome DynamicsVersus First Dynamics of Colliding Black Holes

• My 1975 PhD Dissertation– Solving Einstein’s Equations of General Relativity for Colliding Black Holes and Grav Waves

– CDC 6600 Megaflop/s– Hundred Hours of Computing

• Rob Knight & Smarr Gut Microbiome Map Using 800,000 Core-Hours on SDSC’s Comet– Mapping From Illumina Sequencing to Taxonomy and Gene Abundance Dynamics

– Comet Petaflop/s – Comet Core is 40,000x CDC6600 Speed

– ~Million Core-Hours– 10,000x Supercomputer Time

• Gut Microbiome Takes ~ ½ Billion Times the Compute Power of Early Solutions of Dynamic General Relativity

The Future Foundation of Medicine is an Exponential Scaling-Up of the Number of Deeply Quantified Humans

Source: @EricTopolTwitter 9/27/2014

President Obama Has Committed The United StatesTo Building an Exascale (1000 PetaFLOPs) Computing System by 2025

• “Accelerating delivery of a capable exascale computing system” • “Increasing coherence between the technology base used for

modeling & simulation and that used for data analytic computing”

Is It Time to Radically Expand Our Computer Architectures?

NCSA 1988

Supercomputer Architectures Remain von NeumannShared Memory CPU Plus SIMD Co-Processor

NCSA 2016

The Future of Supercomputing Will Blend Traditional HPC and Data Analytics Integrating Non-von Neumann Architectures

“High Performance Computing Will Evolve Towards a Hybrid Model,

Integrating Emerging Non-von Neumann Architectures, with Huge Potential in Pattern Recognition,

Streaming Data Analysis, and Unpredictable New Applications.”

Horst Simon, Deputy Director, U.S. Department of Energy’s

Lawrence Berkeley National Laboratory

Brain-Inspired ProcessorsAre Accelerating the non-von Neumann Architecture Era

“On the drawing board are collections of 64, 256, 1024, and 4096 chips.

‘It’s only limited by money, not imagination,’ Modha says.”Source: Dr. Dharmendra ModhaFounding Director, IBM Cognitive Computing Group

August 8, 2014

Calit2’s Qualcomm Institute Has Established a Pattern Recognition Lab For Machine Learning on non-von Neumann Processors

“On the drawing board are collections of 64, 256, 1024, and 4096 chips.

‘It’s only limited by money, not imagination,’ Modha says.”Source: Dr. Dharmendra Modha

Founding Director, IBM Cognitive Computing Group

August 8, 2014

UCSD ECE Professor Ken Kreutz-Delgado Brings the IBM TrueNorth Chip

to Start Calit2’s Qualcomm Institute Pattern Recognition Laboratory

September 16, 2015

New Brain-Inspired Non-von Neumann Processors Are Emerging:KnuEdge Has Provided Processor to Calit2’s PRL

www.tomshardware.com/news/knuedge-announces-knuverse-and-knupath,31981.html

www.calit2.net/newsroom/release.php?id=2704

“KnuEdge and Calit2 have worked together since the early days of

the KnuEdge LambdaFabric processor, when key

personnel and technology from UC San Diego

provided the genesis for the first processor design.”

www.calit2.net/newsroom/release.php?id=2726

June 6, 2016

The Rise of Brain-Inspired Computers – Driver 1:Left & Right Brain Computing: Arithmetic vs. Pattern Recognition

Adapted from D-Wave

The Rise of Brain-Inspired Computers – Driver 2: Realtime Simulation of Human Brain Possible With Exascale Supercomputer

Horst Simon, Deputy Director, Lawrence Berkeley National Laboratory’s

National Energy Research Scientific Computing Center

Fastest Supercomputer

Trend LineTianhe-2

• Exascale Power Consumption of >20–30 MW

• The Human Brain Runs on 20 W• Our Brain is a Million Times More Power Efficient!

Horst Simon, Deputy Director, Lawrence Berkeley National Laboratory’s

National Energy Research Scientific Computing Center

The Rise of Brain-Inspired Computers – Driver 3:The Exascale Power Conundrum

The Rise of Brain-Inspired Computers – Driver 4: Reverse Engineering of the Brain Is Accelerating

www.whitehouse.gov/infographics/brain-initiative

Interactively Exploring Microscope Images of Brains:40Gbps From NCMIR to Calit2 64Mpixel Wall

Data and Image Source: Mark Ellisman, NCMIR, UCSD

Artificial Intelligence (AI) is Advancing at a Amazing Pace:Deep Learning Algorithms Working on Massive Datasets

Training on 30M Moves, Then Playing Against Itself

Less Than 2 Years!

Google Released Its AI Software as Open SourceAccelerating Development

https://exponential.singularityu.org/medicine/big-data-machine-learning-with-jeremy-howard/

From Programming Computers Step by Step To Achieve a Goal

To Showing the Computer Some Examples of

What You Want It to Achieve and Then Letting the Computer

Figure It Out On Its Own--Jeremy Howard, Singularity Univ.

2015

November 9, 2015

Corporate Cloud ProvidersAre Already Using Non von Neumann Accelerators

www.microsoft.com/en-us/research/project/project-catapult/

https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html

Microsoft Installs FPGAs into Bing Servers and 432 into TAAC for Academic Access

Deep Learning Will Provide Artificial Intelligence Personalized Assistants to Coach Us to Wellness

Where Medicine Coaching is Now

Where Wellness Coaching is Going

January 10, 2014

Can a Planetary Supercomputer with Artificial IntelligenceTransform Our Sickcare System to Healthcare?

Using this data, the planetary computer will be able to build a computational model of your body

and compare your sensor stream with millions of others. Besides providing early detection of internal changes

that could lead to disease, cloud-powered voice-recognition wellness coaches

could provide continual personalized support on lifestyle choices, potentially staving off disease

and making health care affordable for everyone.

ESSAYAn Evolution Toward a Programmable UniverseBy LARRY SMARRPublished: December 5, 2011

In Spite of the Enormous Promise,A Global Debate is Underway About the Dangers of Superintelligence

"Those disposed to dismiss an 'AI takeover' as science fiction may think again after reading this original and well-argued book." —Martin Rees, Past President, Royal Society

If our own extinction is a likely, or even

possible, outcome of our technological

development, shouldn't we proceed with great

caution? – Bill Joy

Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks. – Steven Hawking

Things Are About to Get Very Interesting…

Source: Hans Moravecwww.transhumanist.com/volume1/power_075.jpg

Smarr Slide from 2001