Introduction

Voting: decision-making strategyMajority voteWeighted majority votingNaive Bayes combination

JustinTalbot,ErinWirchVoting

Majority Voting

Three Main Types of Majority Voting

[1]

JustinTalbot,ErinWirchVoting

Ensemble of Classifiers Increases Accuracy

The accuracy of the ensemble is

Pmaj =L�

m=�L/2�+1

�Lm

�pm(1 − p)L−m assuming that the

probability of success p is the same for all classifiers

[1]JustinTalbot,ErinWirchVoting

Majority Voting Perormance

[1]

JustinTalbot,ErinWirchVoting

Patterns of Success and Failure

Upper bounds and lower bounds can be established on theperformance of the classifiersThe upper bound is pmaj =

� Ll+1

�α assumingα the probability

of each combination of classifiersThe lower bound is pmaj =

pL−1l+1 where p is the overall

accuracy of an individual classifier

JustinTalbot,ErinWirchVoting

Weighted Voting

Classifiers are not all equalWeigh more accurate classifiers

JustinTalbot,ErinWirchVoting

Examples

Where weighted voting worksWhere weighted voting does not work

JustinTalbot,ErinWirchVoting

Maximizing Accuracy

bi ∝ log pi1−pi

Note: Minimum error not guaranteed

JustinTalbot,ErinWirchVoting

Key Points

Weights give more accurate classifiers more powerWorks well in some cases, poorly in othersMinimum error not guaranteed

JustinTalbot,ErinWirchVoting

Conclusion

Voting: decision-making strategyMajority voteWeighted majority votingNaive Bayes combination

JustinTalbot,ErinWirchVoting

The End

Questions?

JustinTalbot,ErinWirchVoting

References

Ludmila I. Kuncheva.Combining Pattern Classifiers: Methods and Algorithms.Wiley-Interscience, July 2004.

JustinTalbot,ErinWirchVoting