Introduction to Online Algorithms - tfocs.iiitb.ac.in

Introduction to Online Algorithms

Arindam KhanIndian Institute of Science,

Bengaluru

Optimization problems.

Ø Optimization: Find the best solution from a set of feasible solutions.Ø Ubiquitous in life as we try to minimize cost or maximize profit.Ø Where to buy grocery? How to score more in exam?

Which classes to take? How can we get more likes?...

“If you don't optimize for the consumer on the internet, you're dead.” Dara Khosrowshahi, Uber CEO

Optimization can be difficult!

q Intractability: computational-hardness such as NP-hardness.v We don’t know how to quickly find the exact solution of the problem.v Can we color a graph with minimum colors? Can we find an optimal

schedule of jobs? Can we find the longest path in a graph?

q Uncertainty: may not have complete data.v We may have to take decisions without knowing the future.v Should I marry this person? Should I buy the stock?

Algorithms to rule!

There are many paradigms of algorithms.Ø Online Algorithms.Ø Approximation Algorithms.Ø Parameterized Algorithms.Ø Dynamic Algorithms.Ø Distributed and Parallel Algorithms.Ø Randomized AlgorithmsØ Streaming Algorithms.Ø Quantum Algorithms.

Offline Algorithms: We have access to the complete input at the beginning.

Online Algorithms: We don’t have access to the complete input at the beginning.

Online Algorithms: Efficient Optimization under Uncertainty

Ø There is an underlying (complete) input. ØDon’t have access to the full input at the beginning.Ø Part of input arrives sequentially over time (arrival

order).ØDecisions for each part must be taken without the

knowledge of future input.ØDecisions are immediate and irrevocable.Ø The performance quality of online algorithms is

measured by competitive ratio, the worst-case ratio of the cost of the decision maker and the cost of the optimal offline algorithm.

Online Algorithms: Optimization under Uncertainty


• Problem 0. Taxy-dispatching of Ola. (k-server/k-taxy)

• There are k-taxies. • Ride requests are coming online.• Goal: minimize the total travel

distance of taxies to serve a series of pickup requests!• Input arrival order plays a role in

the competitive ratio.


• Problem 0. Taxy-dispatching of Ola. (k-server/k-taxy)

• There are k-taxies. • Ride requests are coming online.• Goal: minimize the total travel

distance of taxies to serve a series of pickup requests!• Input arrival order plays a role in

the competitive ratio.

Online Algorithms

7/16/21 9

• Why study online algorithms? • Most practical problems are online in nature and we need fast

solutions for them. • Provides mathematical rigor to study and analyze heuristics. • Interplays with different areas such approximation algorithms,

online learning, convex optimization, information theory, etc.• Gives a metric for difficulty of different discrete optimization

problems. • It’s cool!

Online Algorithms: competitive ratio

7/16/21 10

Ø The performance quality of online algorithms is measured by competitive ratio.

• For a minimization problem, an algorithm A is α-competitive (α>1) if OPT(I) ≤ A(I) ≤ α OPT(I)+ c for all input instances I, where c is a constant.• If OPT(I) ≤ A(I) ≤ α OPT(I), then A is α-strictly competitive. • For a maximization problem, an algorithm A is α-competitive (α<1)

if A(I) ≤ OPT(I) ≤ α A(I)+ c for all input instances I.• We generally look for polynomial time algorithms.

Online Algorithm: Types of Adversaries

• Adversary knows the algorithm!• Oblivious Adversary: must construct the input sequence in advance, and

pays optimal offline solution. (weak adversary)• Adaptive adversary: chooses action based on the previous actions taken by

the algorithm.-- Adaptive online: serves the current request online and then chooses the next action based on the online algorithm’s action so far, pays based on its actions. (medium adversary)-- Adaptive offline: chooses the next request based on online algorithm’s action thus far, pays the optimal offline cost to service the resulting request sequence. (strong adversary)• We will only focus on oblivious adversaries.

Problem 1: Ski rental

• You want to go skiing for 𝑥 number of days • In the online setting, 𝑥 is unknown! • Options:• You can buy the equipment

(for a one-time cost of 𝐵).• or rent each day for a cost of 1 per day.• Goal: Minimize the total cost over 𝑥 days.

Ski rental: Buy or Rent?

• buy: one-time cost of 𝐵, or, rent: each day for a cost of 1 per day.• If we know 𝑥, what is the best solution? • Buy at the beginning if 𝑥 ≥ 𝐵, otherwise, rent every day.

• Competitive ratio (C.R.) of an algorithm that buys at day 1? • In the worst case, you go skiing once; so C. R. = !

"= 𝐵, bad when 𝐵 → ∞.

• C.R. of an algorithm that always rent? Worst-case: we go skiing 𝑛 days. C. R. = 𝑛/𝐵 , Arbitrarily bad when 𝑛 → ∞.

• HW: Find 𝑦 for optimal strategy: rent for 𝑦 days and then buy.

Ski rental: Buy or Rent?

• Online strategy break-even: • Rent for the first (𝐵 − 1) days and buy in the next day. • Competitive ratio of Break-even algorithm? • ≤ #$" %#

#≈ 2 .

• NO deterministic online algorithm can achieve C.R. < 2.• Randomization helps!

Ski rental

• Online “optimal” strategy: • Say 𝑝& : Probability that we rent 𝑖 days and buy on (𝑖 + 1)𝑡ℎ day.• We need: (𝑝'+𝑝" +⋯+ 𝑝#$") =1;

• Choose 𝑝& =(#⋅ 1 − "

#

#$"$&, 𝑤ℎ𝑒𝑟𝑒 𝑐 = "

"$ "$!"" .

• Competitive ratio: c ≤ ))$"

≈ 1.58• No randomized algorithm can beat this factor!

Ski rental

Problem 2: Bin Packing

Bin Packing Problem:

• Given : 𝑛 items with sizes 𝑠", 𝑠*, … 𝑠+, s.t. 𝑠& ∈ (0,1]• Goal: Pack all items into min number of unit bins.

• Example: items {0.8, 0.6, 0.3, 0.2, 0.1} can be packed in 2 unit bins: {0.8, 0.2} and {0.6, 0.3, 0.1}.• NP Hardness from Partition- Cannot distinguish in poly time if need 2 or 3 bins

7/16/21 18

Online Bin Packing

• 𝑛 items with sizes in (0,1]• Items arrive one by one. • We don’t know item sizes till they arrive.• We have to pack immediately and we can not repack.• We need to pack all items using minimum number of bins.• In fact online algorithms can’t achieve < 1.69 competitive ratio

even if we allow exponential time and O(1) space.

Bin Packing

7/16/21 20

• Greedy Algorithm (Next Fit): • Maintain one open bin at any given time.• Place a newly arrived item in the open bin if there is space there;

Otherwise close the bin and open a new bin.

1

0.2, 0.9, 0.3, 0.8, 0.4.

7/16/21 21



1

0.2, 0.9, 0.3, 0.8, 0.4.

Bin Packing

7/16/21 22



0.2, 0.9, 0.3, 0.8, 0.4.

Bin Packing

7/16/21 23



0.2, 0.9, 0.3, 0.8, 0.4.

Bin Packing

7/16/21 24



0.2, 0.9, 0.3, 0.8, 0.4.

Bin Packing

7/16/21 25



0.2, 0.9, 0.3, 0.8, 0.4.

Bin Packing

7/16/21 26



0.2, 0.9, 0.3, 0.8, 0.4.

Bin Packing

7/16/21 27



0.2, 0.9, 0.3, 0.8, 0.4.

Bin Packing: Next Fit

OPT=3

NF=5

7/16/21 28

• How good is Next-Fit (NF)? – O(n) time, O(1) memory.• 𝑂𝑃𝑇 ≥ Sum of sizes of all items. • Observation: In the final packing, total size of items in each two

consecutive bins is larger than 1.• ,-

*= ∑ &."

,-/* 1 < ∑ &.",-/*{size of items in (2𝑖 − 1)𝑡ℎ bin and (2𝑖)𝑡ℎ bin}

≤ Sum of sizes of all items.• 𝑂𝑃𝑇 ≥ ,-

*⇒𝑂𝑃𝑇 ≤ 𝑁𝐹 ≤ 2 𝑂𝑃𝑇 ⇒ 𝑁𝐹 is 2-approximation.

0.2, 0.9, 0.3, 0.8, 0.4.

Bin Packing: NF Analysis

OPT=3

NF=5

7/16/21 29

• Can we do better?

0.2, 0.9, 0.3, 0.8, 0.4.

Bin Packing: First Fit

OPT=3

FF=3

• First Fit: -- O(n) time, O(n) memory.• Consider all bins and place an item in the first bin where there is space;

Otherwise open a new bin and place it there. • FF packs items in at most ⌊1.7 𝑂𝑃𝑇⌋ bins

– analysis is based on weight functions.

Harmonic Algorithm [Lee and Lee, JACM’85]

• Classify items into 𝑀 “harmonic” classes. If item size is between ( "

0%", "0], it is class k. for 1 ≤ 𝑘 < 𝑀.

if size is in (0,1/𝑀] it is in class 𝑀.• Keep 𝑀 open bins, each for one class.• ALGO: When items of class k comes, if there is space put in the bin of

its class. Or open a new bin of its class.• Gives 1.69103-competitive algorithm. The best we can do if all we are

allowed to keep only O(1) number of bins open. • HW: Check performance for 𝑛/3 items of size ½+ 𝜖, 𝑛/3 items of

size 1/3 + 𝜖, 𝑛/3 items of size 1/6 − 2𝜖.

Problem 3: Online Matching


• Ad-allocation problem in Google. (Online matching)

• Users (buyers) and advertisers (sellers) form a bipartite graph. • User appears and interested

advertisers form probable edges.• Each user can see only one ad,

each ad can be shown to one user.• Goal: to maximize matches!




advertisers form probable edges.• Each user can see only one ads,





























• Two factors impacting performance ratio of the algorithm:• Input (offline graph),• Arrival order of vertices

[Algorithm on same offline graph can have different solutions depending on the arrival order of vertices].



• Greedy Algorithm achieves 2-approx!• Pf idea: size of maximum matching is

at most 2* size of maximal matching.• No deterministic algorithm can

achieve <2 competitive ratio.



• Randomization helps!• Ranking [Karp-Vazirani-Vazirani ’90]• Pick a random permutation of left side

vertices. This induces a ranking.• When a new vertex arrives, match it to

highest ranked available neighbor.

1

2

3





the highest ranked available neighbor.
















• C.R. ≈ ))$"

≈ 1.58.Best C.R. one can achieve.


• What is the worst case?• Adversary can choose a worst-case input.• Adversary can also choose a adversarial-order arrival on the input.• For some problems this adversarial-order arrival is too pessimistic.• Considering random-order arrival seems reasonable.

Problem 4: Secretary problem

• There are 𝑛 (rankable) candidates and the candidates arrive one by one. • Once a candidate arrives, then only we see its value. • Decision maker can either select or reject.• If she selects, she marries the selected candidate.• If she rejects, she get to see more candidates, but the decision is irrevocable • Goal: Select the best match!

5 7 10 6

Secretary Problem

5 7 10 6

Secretary Problem

Adversarial-order arrival is hopeless – can behave arbitrarily bad!

• There are 𝑛 (rankable) candidates and the candidates arrive one by one. • Once a candidate arrives, then only we see its value. • Decision maker can either select or reject.• If she selects, she marries the selected candidate.• If she rejects, she get to see more candidates, but the decision is irrevocable. • Goal: Select the best match!

5 7 10 6

Secretary Problem

Random-order arrival is a standard assumption.

• There are 𝑛 (rankable) candidates and the candidates arrive one by one. • Once a candidate arrives, then only we see its value. • Decision maker can either select or reject.• If she selects, she marries the selected candidate.• If she rejects, she get to see more candidates, but the decision is irrevocable.• Goal: Select the best match!

• Random-order arrival: • There is a set of 𝑛 (rankable) candidates 𝐶.• Choose a random permutation π of 𝑛 candidates out of 𝑛! permutations.• The suitors arrive one by one sequentially (in rounds) according to π. • Goal: Maximize the expected value of the accepted candidate.

5 7 10 6

Secretary Problem

Secretary Problem

• Suitor arrive in a random permutation π.• Goal: Maximize the expected value of the accepted candidate.

• Optimal strategy: (when 𝑛 → ∞)ü Reject first 𝑛/𝑒 (randomly-arrived) candidates (say this set is 𝑆). ü Let 𝑝 be the value of the best candidate in 𝑆.ü Consider the following candidates, select the first candidate with value ≥ 𝑝.

• Gives 1/𝑒 competitive ratio & it is tight.

5 7 10 6

Secretary Problem

• Proof: ℙ[𝑐∗ is selected] = ∑&ℙ[𝑐∗comes in 𝑖’th round and we accept in 𝑖’th round].• ℙ[𝑐∗ comes in 𝑖’th round] = 1/𝑛.• ℙ[ We accept in 𝑖’th round]• = ℙ [(𝑖 > 𝑛/𝑒) and (no items in +

)+ 1, …, 𝑖 − 1 had value≥ 𝑝)]

• = ℙ [The best among previous (𝑖 − 1) candidates appear in 𝑆]• = (𝑛/𝑒)/(𝑖 − 1).• Hence, ℙ[𝑐∗ is selected] = ∑&.#$%",…,+

"+⋅ +)(&$")

≥ ")ln( +

+/)) ≈ "

).

Thm: Let 𝑐∗ be the best candidate in 𝐶, then ℙ[𝑐∗ is selected] ≥ "#.

1 2!"+1…

𝑛𝑒 𝑛

𝑝

𝑖𝑖 − 1

𝑐∗

Interplay of techniques and tools

Approximation AlgorithmsDynamic Programming,Local Search,Linear Programming,Rounding of Data,Spectral methods,

Online Algorithms

Doubling,Potential methods, Work function, Fenchel duality,

Online Learning

Convex Optimization

Mirror Descent

RegularizationGreedy algorithms,Dual Fitting, Primal-Dual Methods,

Tools from math: Probability, Linear Algebra, Calculus/Analysis, Abstract Algebra, Convex Optimization, Graph Theory…

Some interesting related problems• Online load balancing and scheduling• Online network routing• Online graph coloring• Paging, metrical task systems, and k-server• Online learning• Online TSP and Steiner tree• Online Facility Location problem• Online Set Cover problem.

Other Related Models• Online Learning.• Online convex optimization.• Dynamic Algorithms.• Streaming Algorithms.• Recourse. • Random-order Arrival.• IID model.• With advice.

More references• 1. Books:

- Online Computation and Competitive Analysis: Allan Borodin and Ran El Yaniv (An old but comprehensive book for pre-2000 algorithms).- Online Algorithms: The State of the Art (Important algorithms from pre-1998).• 2. Surveys/book chapters:

- Bookchaper by Susanne Albers (from 2003).- Primal Dual Survey by Buchbinder and Naor (See the videos and slides ) • 3. Workshops:

- Algorithms and Uncertainty: Simons Institute for the Theory of Computing- Online Algorithms and Learning Workshop- Workshop on modern online algorithms.

https://www.amazon.com/Online-Computation-Compet-Analysis-Borodin/dp/0521619467/ref=sr_1_fkmrnull_1?keywords=online%20algorithms%20borodin&qid=1552676099&s=gateway&sr=8-1-fkmrnull

https://www.amazon.in/Online-Algorithms-Lecture-Computer-Science/dp/3540649174/ref=sr_1_2?ie=UTF8&qid=1552676057&sr=8-2&keywords=online%20algorithms

http://www14.in.tum.de/personen/albers/papers/inter.pdf

https://m.tau.ac.il/~nivb/download/pd-survey.pdf

https://simons.berkeley.edu/programs/uncertainty2016

http://www.lorentzcenter.nl/lc/web/2014/680/info.php3?wsid=680&venue=Oort

http://math.haifa.ac.il/lea/MOLI.htm

More references• 4. Recent Lecture Notes:

- Advanced Algorithms, Fall 2018 (by Anupam Gupta, CMU). - Algorithms and Uncertainty (by Nikhil Bansal at Berkeley’16). - Competitive analysis via convex optimization (UW’18 by Bubeck and Lee) - -- Decision Making at Scale - 2018 (by Kamesh Munagala at Duke) - On-line and Approximation algorithms, Yossi Azar (TAU 2018)- Online Algorithms (U. Manitoba by Shahin Kamali 2018).• 5. Other special topics:

- Online matching and ad allocation survey, by Aranyak Mehta - Approximation and Online Algorithms for Multidimensional Bin Packing, by Christensen et al. - Online Scheduling Survey, by Susanne Albers - Technique to Obtain Hardness Results for Randomized Online Algorithms. - Online Algorithms with Advice: A Survey.

Thank you!

Documents

Introduction to Online Algorithms - tfocs.iiitb.ac.in