Embed Size (px)
Citation preview
Importance Sampling
What is Importance Sampling ?
AsimulationtechniqueUsedwhenweareinterestedinrareeventsExamples:
BitErrorRateonachannel,Failureprobabilityofareliablesystem
1
We saw some of it already
Q:We simulate R=10000samples andfind nobiterror.What can we sayaboutthebiterror rate?
A:with confidence0.95,BER<3.710
2
3
What is the Problem ?
Assumeyou can simulate asystemYouwant toevaluate theprobability ofarareeventWe want tosay morethan ananswer like : ∈ 0, 3.6910
i.e.we want agoodrelativeaccuracy on
Assumeproba ofrareevent is 10 :howmany simulationruns doyou needtoobtain anestimate of with 10%relativeaccuracy ?
4
What is the Problem ?
Assumeproba ofrareevent is 10 :howmany simulationruns doyou needtoobtain anestimate of with 10%relativeaccuracy ?
5
What is the Problem ?
Assumeproba ofrareevent is 10 :howmany simulationruns doyou needtoobtain anestimate of with 10%relativeaccuracy ?
6
replicationsevents
Confidence interval 1.96 1
Relative accuracy = .
1.96 1.96
Relative accuracy = 10% ⇔ 1.96 0.1 ⇔ ..
The Goal of Importance Sampling
Obtainsmall probability with goodaccuracy…while keeping small
Intheprevious example,thedirectapproach requires 4. 10 runs toestimate 10 with 10%accuracy
We can domuch better with ImportanceSampling
7
The Idea of Importance Sampling
8
The Idea of Importance Sampling (cont’d)Ifwe simulate ,howdowe estimate ?
Ifwe simulate insteadofX,we cannot use
But:
Showthis !
9
Importance Sampling Monte Carlo
10
Example: Bit Error Rate (BER)
11
12
13
, , … ,∼ 0,
discrete, on , , … , ,
, ,Estimate ⋯
1 ⋯
, , … ,on ∞, ∞ discrete, on , , … , ,
Estimate
14
, , … ,∼ 0,
discrete, on , , … , ,
, ,Estimate ⋯
1 ⋯
, , … ,on ∞, ∞ discrete, on , , … , ,
Estimate
, , ,,
,
15
16
, , … ,∼ 0,
12
on ∞, ∞
17
, , … ,∼ 0,
12
on ∞, ∞
121212
12
∼ ,
18
, , … ,∼ 0,
discrete, on , , … , ,Estimate ⋯
1 ⋯
, , … ,on ∞, ∞ discrete, on , , … , ,
, , ,
, … ,
⋯
…
Estimate
Importance Sampling Monte Carlo
Wedothis forseveral valuesof andfindthesame estimate6.4510
What is different ?
19
Importance Sampling Monte Carlo
Wedothis forseveral valuesof andfindthesameestimate 6.4510What is different ?Hopefully ,thenumber ofruns
20
Choosing an Importance Sampling Distribution
Whatisagoodimportancesamplingdistribution?Onethatminimizesthenumberofruns
Thiscanbequantifiedwiththevarianceoftheimportancesamplingestimator
21
22
Thesmallestvarianceisfor
23
R (proportional to variance)
Choosing an Importance Sampling Distribution (1)
Ruleofthumb:Theeventsofinterest,undertheimportancesamplingdistributionshouldbe
notrare
notcertain
24
Choosing an Importance Sampling Distribution (2)
Theoptimalimportancesampling distributionis theonethat minimizes
Isthis thesame asminimizing thevarianceoftheimportancesamplingestimator ?
25
26
27
A Generic Algorithm
Ideas:empiricallyfindimportancesamplingdistributionsuchthatAverageoccurrenceofeventofinterestiscloseto0.5Minimizes
CanbecomputedbyMonteCarlowithsmallnumberofruns
Thealgorithmdoesnotsayhowtodooneimportantthing:whichone?
28
Conclusion
Ifyouhavetosimulaterareevents,importancesamplingisprobablyapplicabletoyourcaseandwillprovidesiginificantspeedup
Agenericalgorithmcanbeusedtofindagoodsamplingdistribution
29
