MSR AI

MSR AI

Principles of more general artificial intelligence

Combine competencies into symphonies of intelligence

Wedges of competency Integrative intelligence

Designs for mix of Initiatives

Machine learning & inference

Human

cognition

Machine

intelligence

Trustworthiness & safety

Fairness & accuracy

Transparency & explanation

Explorations of human-AI relationship

Qualification problem

All preconditions?

Ramification problem

All effects of action?

Knowing that you do not know is the best.

Not knowing that you do not know is an illness.

- Laozi, 500-600 BCE

Toyama & H. 2000

Models of inference reliability

back. subtract color basedmotion decay

Target inference

Inferred reliability

Toyama & H. 2000

lights out

jolted camera

facing away

distraction

Unmodeled situations

Inference reliability Robust portfolios

back. subtract color based motion decay Joint inference

Fang, et al., 2015

Learn about abilities & failures

Performance

Successes & failures

p( fail | E, t)

Confidence

Image

H1

H2

H3

W1

W2

W3

W4

Input s

H3

Caption:

a man holding a tennis

racquet on a tennis court

H1

H2

H3

W1

W2

W3

Input t1

H3

W4

Deep learning about deep learning performance

Reliable predictions of performance: Known unknowns

Grappling with Open-World Complexity

Grappling with Open-World Complexity

Reliable predictions of performance: Known unknowns

Challenge of unknown unknowns

Expanded real-world testing

Algorithmic portfolios

Failsafe designs

People + machines

M

training data

M

real-world concepts

x= (𝑓1, … , 𝑓𝑘)

wrong label

high confidence

Conceptual incompleteness

cats

dogs

How to define & search regions of data space?

How to trade exploration and exploitation? Lakkaraju, Kamar, Caruana, H, 2017.

Identifying classifier blindspots

M

training data

cats

dogs

M

training data

x= (𝑓1, … , 𝑓𝑘)

wrong label

high confidence

Partition

space by

attributes

White CatsWhite Dogs Brown DogsBrown Cats Lakkaraju, Kamar, Caruana, H, 2017.

Identifying classifier blindspots

Transfer learning

Learn from rich simulations

Learn generative models

Hospital A Hospital B

Hospital C

Site-specific data

Observations, definitions

Patients, prevalencies

Covariate dependencies

Transfer learning opportunity

Hospital A

Hospital CHospital C

Hospital BHospital A

Hospital C

Hospital B

J. Wiens, J. Guttag, H, 2015.

A: Community hosp: 10k pts/yr

B: Acute care & teaching: 15k/yr

C: Major teaching & research: 40k/yr

M. Gabel, R. Caruana, M. Philipose, O. Dekel

Less data with better features

ImageNet 1000, 1M photos

Cut off top layer

M. Gabel, R. Caruana, M. Philipose, O. Dekel

Less data with better features

ImageNet 1000, 1M photos

Cut off top layer

Raspberry Pi

Camera

Battery

Trillions of sessions in complex scenarios

Learn & evaluate core competencies

Learn to optimize action plans

Mapping

Planning

Next actions

Map

Plans

Stereo

algorithmCNN

Hybrid

Deep Stereo

Depth Image

D. Dey, S. Sinha, S. Shah, A. Kapoor

Mapping

Planning

Next actions

Map

Plans

Stereo

algorithmCNN

Hybrid

Deep Stereo

Depth Image

D. Dey, S. Sinha, S. Shah, A. Kapoor

Learn expressive generative models

Generalize from minimal training sets

Harness physics

Multilevel variational autoencoder

Learn disentangled representations

Groups of observations latent models

Learning generative models

Vary IDVary style

D. Buchacourt, R. Tomioka, S. Nowozin, 2017

Smooth control over learned latent space

Inject physics to disentangle & generalize

Same?

Kulkarni, Whitney, Kohli & Tenenbaum, 2015

Illumination Nod Shake

Inject physics to disentangle & generalize

Kulkarni, Whitney, Kohli & Tenenbaum, 2015

Potential biases in data, performance

Best practices & processes

Industry and beyond

Bernard Parker: rated high risk Dylan Fugett: rated low risk.

March 2017

A. Howard, C. Zhang, H., 2017

Machine learning “contact lens” for children

Scientific foundations

Engineering practices

AI, people, and society

Much to do