Sequential Line Search for Efficient Visual Design Optimization by Crowds

Takeo IgarashiIssei SatoYuki Koyama Daisuke Sakamoto

Motivation

Parameter Tweaking Based on Preference

Design Exploration Optimization

x

⇤= argmax

x2XGoodness(x)

Optimization

x

⇤= argmax

x2XGoodness(x)

Human-in-the-Loop Optimization

Crowdsourced Human Computation

011001011101

Alexander J. Quinn and Benjamin B. Bederson. 2011. Human computation: a survey and taxonomy of a growing field. In Proc. CHI '11. 1403-1412.

Related Work on Crowdsourced Human Computation

Contributions

Contributions

Microtask Design

Microtask Design

😁

Technical Challenges

Technical Background: (Standard) Bayesian Optimization

(Standard) Bayesian Optimization実験計画

New Technique: Bayesian Optimization Based on Line Search

Our Method

comparative な答えしか使えない

How to Define Slider Spaces

S

How to Define Slider Spacesx

+= argmax

x2{xi}µ(x)

x

EI= argmax

x2XEI(x)

Web Interface for Crowdsourcing

Applications #1 Photo Color Enhancement (6D)

Evaluation: Crowdsourced Voting

Applications #2 Material Appearance (3D / 7D)

Comparative Evaluation

Comparative Evaluation

Experiment #1: Synthetic Setting

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0 5 10 15 20

0.000.050.100.150.200.250.300.350.400.45

0 5 10 15 20

Optimizing a 2D function

Optimizing a 6D function

0.000.100.200.300.400.500.600.700.800.901.00

0 5 10 15 20

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0 5 10 15 20

Optimizing a 20D function

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

0 5 10 15 20

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

0 5 10 15 20

Experiment #2: Crowdsourcing Setting

Experiment #2: Crowdsourcing Setting

0

5

10

15

20

25

30

35

0 2 4 6 8 10 12 14 16 18 20 0

5

10

15

20

25

30

35

0 2 4 6 8 10 12 14 16 18 20

Summary

Summary

Concept:

Strategy:

Technique:

Applications:

Limitation & Future Work

Sequential Line Search for Efficient Visual Design Optimization by Crowds

Takeo IgarashiIssei SatoYuki Koyama Daisuke Sakamoto

Optimization with Different Initial Conditions

Task Burden (Completion Time)

0

10

20

30

40

50

60

70

80

90

SSM 2GC 4GC

Task

Com

plet

ion

Tim

e [s

]

Advantages of Involving Many Crowds

Assumptions on Design Domains

Assumptions on Crowds

Difficult Cases

SIGGRAPH

SIGGRAPH

SIGGRAPH