Embed Size (px)
Citation preview
SciPy and Real-time Big Data for Site Optimization
Pyleus
Message Processor
Bolt
Pyleus
EventWorker
Bolt
PyleusSciPy
Optimizer
Bolt
Pyleus
UpdateMessenger
Bolt
SciPy
BayesianBandit
Spout
Spout
ApplicationState
Visitors to Bankrate.com
Impressions and Clicks
Improve User Experience
For more info, contact:[email protected]
Which Variation to
show
Bankrate.com Data Science and Engineering Team
Example: Pick better story headlines
Objective for Site Optimization:Enable fast and cost-efficient ways of testing new designs to improve user experience
Algorithmically decide which of two headlines to show user to maximize click-thru-rate (CTR)
Computation Framework with Kafka-Storm
Simulation Results
With more data, algorithm becomes more confident of estimated CTR for each variation
Bayesian Multi-Armed Bandit algorithm on Storm Topology decides how often to show each variation by analyzing impressions and clicks
Iteration: 100
W1: 56.37%W2: 42.63%
Iteration: 1000
W1: 9.82%W2: 90.18%
Iteration: 2000 behavior reversalW1: 64.06%W2: 39.94%
Iteration: 3000
W1: 94.84%W2: 5.16%
Iteration: 4000
W1: 97.12%W2: 2.88%
