UNIFORM CONVERGENCE MAY BE UNABLE TO EXPLAIN GENERALIZATION IN DEEP LEARNINGVAISHNAVH NAGARAJAN1 ZICO KOLTER1,2

1Computer Science Department, Carnegie Mellon University 2Bosch Center for Artificial Intelligence, Pittsburgh

THE HIGH LEVEL MESSAGE

THE GENERALIZATION PUZZLE & UNIFORM CONVERGENCE (U.C)

PROGRESS SO FAR

Our 1st finding: GENERALIZATION BOUNDS ↑ WITH TRAINING SET SIZE

Key element in proof: TIGHTEST UNIFORM CONVERGENCE

CONCLUSIONS AND FUTURE WORK

REFERENCES

PROOF INTUITION

Our 2nd finding: PROVABLE FAILURE OF UNIFORM CONVERGENCE

FAILURE OF U.C. IN A HYPERSPHERE EXAMPLE