Upload
netis
View
42
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Learning and Testing Submodular Functions. Columbia University. + Work in progress with Rocco Servedio. With Sofya Raskhodnikova (SODA’13). Grigory Yaroslavtsev. October 26, 2012. Submodularity. Discrete analog of convexity/concavity, law of diminishing returns - PowerPoint PPT Presentation
Citation preview
Learning and Testing Submodular Functions
Grigory Yaroslavtsev
Columbia UniversityOctober 26, 2012
With Sofya Raskhodnikova (SODA’13)
+ Work in progress with Rocco Servedio
Submodularity• Discrete analog of convexity/concavity, law of
diminishing returns• Applications: optimization, algorithmic game
theory
Let • Discrete derivative:
• Submodular function:
Exact learning• Q: Reconstruct a submodular with poly(|X|) queries (for all
arguments)?• A: Only-approximation (multiplicative) possible [Goemans, Harvey,
Iwata, Mirrokni, SODA’09]
• Q: O-fraction of points (PAC-style learning with membership queries under
uniform distribution)?
• A: Almost as hard [Balcan, Harvey, STOC’11].
Pr𝑟𝑎𝑛𝑑𝑜𝑚𝑛𝑒𝑠𝑠 𝑜𝑓 𝑨 [ Pr
𝑺∼𝑈 (2𝑋)[ 𝑨 (𝑺 )= 𝒇 (𝑺 ) ]≥1−𝜖]≥ 12
Approximate learning• PMAC-learning (Multiplicative), with poly(|X|) queries :
(over arbitrary distributions [BH’11])
• PAAC-learning (Additive)
• Running time: [Gupta, Hardt, Roth, Ullman, STOC’11]
• Running time: poly [Cheraghchi, Klivans, Kothari, Lee, SODA’12]
Pr𝑟𝑎𝑛𝑑𝑜𝑚𝑛𝑒𝑠𝑠 𝑜𝑓 𝑨 [ Pr
𝑺∼𝑈 (2𝑋)[¿ 𝒇 (𝑺 )− 𝑨 (𝑺 )∨≤ 𝜷 ]≥1−𝜖 ]≥ 12
Learning
Goemans,Harvey,Iwata,Mirrokni
Balcan,Harvey
Gupta,Hardt,Roth,Ullman
Cheraghchi,Klivans,Kothari,Lee
Our result with Sofya
Learning -approximationEverywhere
PMAC Multiplicative
PAACAdditive
PAC
(bounded integral range )
Time Poly(|X|) Poly(|X|)
Extrafeatures
Under arbitrary distribution
Tolerant queries
SQ- queries,Agnostic
Agnostic
• For all algorithms
Learning: Bigger picture
}XOS = Fractionally subadditive
Subadditive
Submodular
Gross substitutes
OXS
[Badanidiyuru, Dobzinski, Fu, Kleinberg, Nisan, Roughgarden,SODA’12]
Additive (linear)
Value demand
Other positive results:• Learning valuation functions [Balcan,
Constantin, Iwata, Wang, COLT’12]• PMAC-learning (sketching) valuation functions
[BDFKNR’12]• PMAC learning Lipschitz submodular functions
[BH’10] (concentration around average via Talagrand)
Discrete convexity• Monotone convex
• Convex
1 2 3 … <=R … … … … … … … … n02468
1 2 3 … <=R … … … … >= n-R
… … … n02468
Discrete submodularity
• Monotone submodular
𝑋
∅|𝑺|≤𝑹
• Submodular
𝑋
∅|𝑺|≤𝑹
|𝑺|≥|𝑿|−𝑹
• Case study: = 1 (Boolean submodular functions )Monotone submodular = (monomial)Submodular = (2-term CNF)
Discrete monotone submodularity
≥𝒎𝒂𝒙 ( 𝒇 (𝑺𝟏 ) , 𝒇 (𝑺𝟐))
|𝑺|≤𝑹
• Monotone submodular
Discrete monotone submodularity• Theorem: for monotone submodular f• (by monotonicity)
|𝑺|≤𝑹
𝑇
Discrete monotone submodularity
• S smallest subset of such that • we have => Restriction of on is monotone increasing =>
|𝑺|≤𝑹
𝑇
𝑆 ′
𝑆 ′
Representation by a formula
• Theorem: for monotone submodular f
• Notation switch: ,
• (Monotone)
(no negations)
• (Monotone) submodular can be represented as a (monotone) pseudo-Boolean 2R-DNF with constants
Discrete submodularity• Submodular can be represented as a
pseudo-Boolean 2R-DNF with constants • Hint [Lovasz] (Submodular monotonization): Given submodular define
Then is monotone and submodular.
𝑋
∅|𝑺|≤𝑹
|𝑺|≥|𝑿|−𝑹
Learning pB-formulas and k-DNF• = class of pB-DNF of width with • i-slice defined as
• If its i-slices are -DNF and:
• PAC-learning
(
Pr𝑟𝑎𝑛𝑑 (𝑨) [ Pr
𝑺∼𝑈( {0,1 }𝑛)[ 𝑨 (𝑺 )= 𝒇 (𝑺 ) ]≥1−𝜖]≥ 12
iff
Learning pB-formulas and k-DNF• Learn every i-slice on fraction of arguments • Learning -DNF () (let Fourier sparsity = )
– Kushilevitz-Mansour (Goldreich-Levin): queries/time. – ``Attribute efficient learning’’: queries– Lower bound: () queries to learn a random -junta ( -DNF) up to
constant precision.• Optimizations:
– Slightly better than KM/GL by looking at the Fourier spectrum of (see SODA paper: switching lemma => bound)
– Do all R iterations of KM/GL in parallel by reusing queries
Property testing• Let C be the class of submodular • How to (approximately) test, whether a given is in C?• Property tester: (Randomized) algorithm for distinguishing:
1. -far):
• Key idea: -DNFs have small representations:– [Gopalan, Meka,Reingold CCC’12] (using quasi-sunflowers [Rossman’10]), -DNF formula F there exists:-DNF formula F’ of size such that
Testing by implicit learning• Good approximation by juntas => efficient property
testing [surveys: Ron; Servedio]
– -approximation by -junta– Good dependence on :
• [Blais, Onak] sunflowers for submodular functions
– Query complexity: (independent of n)– Running time: exponential in (we think can be reduced it to
O(– We have lower bound for testing -DNF (reduction from Gap
Set Intersection: distinguishing a random -junta vs + O(log )-junta requires queries)
Previous work on testing submodularity
[Parnas, Ron, Rubinfeld ‘03, Seshadhri, Vondrak, ICS’11]:
• U. • Lower bound: Special case: coverage functions [Chakrabarty, Huang, ICALP’12].
Gap in query complexity
Thanks!