Fourier Analysis of Stochastic Sampling For Assessing Bias and Variance in Integration

Kartic Subr, Jan Kautz University College London

great sampling papers

Spectral analysis of sampling must be

IMPORTANT!

BUT WHY?

numerical integration, you must try

assessing quality: eg. rendering

Shiny ball, out of focusShiny ball in motion

…pixel

multi-dim integral

variance and bias

High variance High bias

bias and variance

High variance High bias

predict as a function of sampling strategy and

integrand

variance-bias trade-off

High variance High bias

analysis is non-trivial

Abstracting away the application…

0

numerical integration implies sampling

0sampled integrand

(N samples)

numerical integration implies sampling

0sampled integrand

the sampling function

integrand

sampling functionsampled integrand

multiply

sampling func. decides integration quality

integrandsampled function

multiplysampling function

strategies to improve estimators

1. modify weights

eg. quadrature rules

strategies to improve estimators

1. modify weights

eg. importance sampling

2. modify locations

eg. quadrature rules

abstract away strategy: use Fourier domain

1. modify weights 2. modify locations

eg. quadrature rulesanalyse sampling function in Fourier domain

abstract away strategy: use Fourier domain

1. modify weights

a. Distribution eg. importance sampling)

2. modify locations

eg. quadrature rules

sampling function in the Fourier domain

frequency

amplitude (sampling spectrum)

phase (sampling spectrum)

stochastic sampling & instances of spectra

Sampler (Strategy 1)

Fouriertransform

draw

Instances of sampling functions Instances of sampling spectra

assessing estimators using sampling spectra

Sampler (Strategy 1)

Sampler(Strategy 2)

Instances of sampling functions Instances of sampling spectra

Which strategy is better? Metric?

accuracy (bias) and precision (variance)

estimated value (bins)

freq

uenc

yreference

Estimator 2

Estimator 1

Estimator 2 has lower bias but higher variance

overview

related work

signal processing

assessing sampling patterns

spectral analysis of integration

Monte Carlo sampling

Monte Carlo rendering

stochastic jitter: undesirable but unavoidable

signal processing

Jitter [Balakrishnan1962] Point processes [Bartlett 1964] Impulse processes [Leneman 1966]

Shot noise [Bremaud et al. 2003]

we assess based on estimator bias and variance

assessing sampling patterns

Point statistics [Ripley 1977] Frequency analysis [Dippe&Wold 85, Cook 86, Mitchell 91] Discrepancy [Shirley 91]

Statistical hypotheses [Subr&Arvo 2007]

Others [Wei&Wang 11,Oztireli&Gross 12]

recent and most relevant

spectral analysis of integration

numerical integration schemes [Luchini 1994; Durand 2011] errors in visibility integration [Ramamoorthi et al. 12]

recent and most relevant

spectral analysis of integration

numerical integration schemes [Luchini 1994; Durand 2011] errors in visibility integration [Ramamoorthi et al. 12]

1. we derive estimator bias and variance in closed form2. we consider sampling spectrum’s phase

Intuition(now)

Formalism(paper)

sampling function = sum of Dirac deltas

+

+

+

Review: in the Fourier domain …

primal Fourier

Dirac deltaFourier transform

Frequency

Real

Imaginary

Complex plane

amplitudephase

Review: in the Fourier domain …

primal Fourier

Dirac deltaFourier transform

Frequency

Real

Imaginary

Complex plane

Real

Imaginary

Complex plane

amplitude spectrum is not flat

=

+

+

+

primal Fourier

=

+

+

+

Fourier transform

sample contributions at a given frequency

Real

Imaginary

Complex plane

5

1 2 3 4 5

At a given frequency

3

2

4

1

sampling function

the sampling spectrum at a given frequency

sampling spectrum

Complex plane

53

2

4

1

centroid

given frequency

the sampling spectrum at a given frequency

sampling spectrum instances

expected centroid centroid variancegiven frequency

expected sampling spectrum and variance

expected amplitude of sampling spectrum variance of sampling spectrum

frequency

DC

intuition: sampling spectrum’s phase is key

• without it, expected amplitude = 1!– for unweighted samples, regardless of distribution

• cannot expect to know integrand’s phase– amplitude + phase implies we know integrand!

Theoretical results

Result 1: estimator bias

bias

reference

inner product

frequency variable

S f

sampling spectrum integrand’s spectrum

Implications

1. S non zero only at 0 freq. (pure DC) => unbiased estimator

2. <S> complementary to f keeps bias low

3. What about phase?

expanded expression for bias

bias

expanded expression for bias

reference

bias

phase

amplitude

Sf fS

omitting phase for conservative bias prediction

reference

bias

phase

amplitude

Sf fS

new measure: ampl of expected sampling spectrum

ours periodogram

Result 2: estimator variance

variance

frequency variableinner product

S || f ||2

sampling spectrum integrand’s power spectrum

the equations say …

• Keep energy low at frequencies in sampling spectrum– Where integrand has high energy

case study: Gaussian jittered sampling

1D Gaussian jitter

samples

jitter using iidGaussian distributed 1D random variables

1D Gaussian jitter in the Fourier domain

real

Imaginary Complex planeFourier transformed samples at an arbitraryfrequency

Jitter in position manifests as phase jitter

centroid

derived Gaussian jitter properties

• any starting configuration

• does not introduce bias

• variance-bias tradeoff

Testing integration using Gaussian jitter

random points

binary function p/w constant function p/w linear function

bias-variance trade-off using Gaussian jitter

bias

varia

nce

Gaussian jitter

random

grid

Poisson disklow-discrepancy Box jitter

Gaussian jitter converges rapidly

Log-number of primary estimates

log-

varia

nce

Gaussian jitter

Random: Slope = -1O(1/N)

Poisson disklow-discrepancyBox jitter

Conclusion: Studied sampling spectrum

sampling spectrum

integrand spectrum

integrand

sampling function

Conclusion: bias

sampling spectrum

integrand spectrum

integrand

sampling function

bias depends on E( ) .

Conclusion: variance

sampling spectrum

integrand spectrum

integrand

sampling function

bias depends on E( ) .

variance is V( ) .2

Acknowledgements

Take-home messages

53

2

4

1

relative phase is key Ideal sampling spectrum

No energy in sampling spectrumat frequencies where integrand has high energy

Questions?

http://www.wordle.net/show/wrdl/6890169/FMCSIG13

Sorry, what? Handling finite domain?

• Integrand = integrand * box

conclusion