Novel algorithms for peer-to-peer optimization in networked systems

ACCESS Group meeting Mikael Johansson mikaelj@ee.kth.se

Novel algorithms for peer-to-peer optimization

in networked systems

Björn Johansson and Mikael Johansson, Automatic Control Lab, KTH, Stockholm, Sweden

Joint work with M. Rabi, C. Caretti, T. Keviczky and K.-H. Johansson

Content

• Motivation• Decomposition review• A framework for peer-to-peer optimization• Markov-randomized incremental subgradient method• Combined consensus-subgradient method• Experiences from implementation• Conclusions

MotivationLarge-scale optimization problem…

Decomposed into several small subproblems• Potentially large computational savings• Foundation for distributed decision-making

– fi performance of agent i, depends on action of others

– challenge: avoid coordinator, obey communication constraints

CoordinatorCoordinator

Application: multi-agent coordination

Find jointly optimal controls and rendez-vous point

”DMPC” – Distributed model-predictive consensus.

Application: distributed estimation

Node v measures yv, cooperates to find network-wide estimate

Solution is average, algorithm solves ”consensus” problem– Directly extends to Huber’s M-function (robust estimator)

Insert ”physical” pictureof estimation network here

Application: resource allocation

Throughput maximization under global bandwidth constraint

Global constraint, not global variable complicates problem.

Insert ”physical” pictureof estimation network here

Content

Decomposition review

Techniques for decomposing large-scale problem into many small

Trivial case: separable problems

Separable problems

Each node v can find xv by itself, no coordinator needed.

– Reality often more complex (and interesting!)

Complicating variables

Consider unconstrained problem in variables (x1, x2, ):

Here, is complicating (or coupling) variable.

Observation: when fixed, problem is separable in (x1, x2)

– how can this be exploited?

Primal decomposition

Fix complicating variable , define

To evaluate functions i we need to solve associated subproblems.

Original problem is equivalent to the master problem

in variable . Convex when original problem is. Possibly non-smooth.

Called primal decomposition– master problem (coordinator) optimizes primal variable.

Dual decomposition

Introduce new variables 1, 2 and consider

Here, 1 and 2 are local versions of complicating variable

The constraints 1=2 enforces consistency.

Key observation: Lagrangian

is separable (can minimize over local variables separately)

Dual decomposition

Hence, the dual function has the form

where each part of the dual can be evaluated locally,

(evaluation requires solving dual subproblems)

Dual problem

is convex, but not necessarily differentiable.

Subgradient methods

A subgradient of a convex function f at x is any that satisfies

• affine global underestimators• coincide with gradient if f smooth

Projected subgradient method

Converge if bounded and

Incremental subgradient methods

Apply to problems on the form

(e.g. our general form, by letting )

Algorithm: (v,k subgradient of fv at k)

Update by cyclic componentwise (negative) subgradient steps– can use fixed (e.g. 1…V) or random update order

Content

Our frameworkA convex (possibly non-smooth) optimization problem

A connected communication graph• local variables xv at each node v • global variables • per-node loss function fv(xv, )

Peer-to-peer:• Nodes can only communicate with neighbors

Quiz and challenge

Quiz: Which of the techniques we described are peer-to-peer?– Primal decomposition? – Dual decomposition? – Incremental subgradient methods?

Challenge: develop simple and efficient p2p optimization techniques!

Content

Peer-to-peer incremental subgradients?

Incremental subgradients not peer-to-peer– Estimate of optimizer forwarded in ring, or to arbitrary node

Is it possible to develop method that only forwards to neighbors?

Unbiased random walk on graph

Need to construct “unbiased” random walk– Visit every node with equal probability

(has stationary uniform probability)– Transition matrix can be computed via Metropolis-Hastings

(dv is the degree of node v, i.e. number of links)

– Can be computed using local info only!

Markov-randomized algorithm

Repeat:

• Update estimate

(vk state of Markov chain, vk subgradient of fvk

• Pass estimate to random neighbor using Markov chainP=[Pv,w] computed via Metropolis-Hasting

Conceptually simple idea. What can we say about its properties?

Main result

Proof highlights:• Sample sequence when chain in state v• Establish: all nodes visited w. equal probability during return time• Use conditional expectations• Invoke supermartingale theorem

Example: robust estimation

Content

Consensus-subgradient method

Key trick for distributing dual decomposition

Dual decomposition: relax consistency requirements

Alternative idea: “neglect and project” – Each node has local view of global decision variables– Updates in direction of (negative) subgradient– Coordinate with neighbors to achieve consistency

Will apply consensus iterations

Basic algorithm

Repeat

1. Predict next iterate using subgradient method

(v subgradient of f at v(k))

1. Execute I consensus iterations to approach consistency

2. Project (locally) on constraint set

Main result (unconstrained case)

Proof: based on results from approximate subgradient methods

Similar, somewhat more complex, results for constrained case.

Example

Simple 5-node network (left) non-smooth functions fv (right)

Example

Iterates for one (left) and 11 consensus iterations per step

To think about…

What is the right aggregation primitive in the network?– Sampling via unbiased random walk?– Consensus/gossiping?– Spanning-trees?

Has implication on– Implementation complexity/accuracy– Privacy (internal models, objectives private or shared?)– Information dissemination (who knows what in the end)

Content

Implementation experiences

Wireless sensor network testbed at KTH

The ultimate test: – can we make these algorithms run on our WSN nodes?

Wireless communication

Sensors communicate using 802.15.4 compliant radios

Basic primitives: – Unicast: a node addresses a

single neighbor at a time– Broadcast: communication with

(possibly) all neighbors

Exist in reliable and unreliable versions

Problem and solution candidates

We considered quadratic loss functions in nodes – consensus iterations one way to find optimum

Implemented three alternatives– P2P incremental subgradient, using reliable unicast– Dual decomposition using unreliable broadcast– Gossiping algorithm by Boyd et al, reliable broadcast

Nodes maintain local estimate of optimizer

1. Broadcasts current iterate to neighors

2. Updates Lagrange multipliers for some links(based on disagreement with neigbors)

3. Updates local estimate

Unreliable broadcast, since algorithm

can tolerate some packet losses

[Rabbat et al, IEEE SPAWC 2005]

Algorithm I: dual

The classical consensus iteration

1. Broadcasts current iterate to neighors

2. Updates local estimate

Reliable broadcast for consistency

[Xiao et al, IPSN 2005]

Algorithm II: consensus iteration

Algorithm III: p2p incremental

Our peer-to-peer incremental subgradient method

1. Update estimate using subgradientwith respect to local loss function

2. Pass estimate to random neigbour(forwarding decision based on Metropolis)

Reliable unicast (important not to loose token)

Ns2 simulations

fv quadratic (consensus), NS2 evaluation of three schemes

Dual, Markov-incremental subgradient, Xiao-Boyd.

Real implementation

Experiences

• Works surprisingly well

• Basic primitives not so basic– Reliable broadcast– Neighbor discovery

• Challenging the model– Link assymetry!– Packet loss, – Time/energy-efficiency.

Need to go back and revise theory (and implementation!)

Conclusions

Distributed optimization in networked systems– Important and useful– Many challenges remain!

Novel peer-to-peer optimization algorithms– Markov-modulated incremental subgradient method– Consensus-subgradient

Practical implementation in WSN testbed

Implementation and application challenges drive next iteration!

Novel algorithms for peer-to-peer optimization in networked systems

Documents

Networked Parallel Algorithms for Robust Convex … Networked Parallel Algorithms for Robust Convex Optimization via the Scenario Approach Keyou You and Roberto Tempo Abstract—This

Trust and Reputation Algorithms for Hierarchically ... › bitstream › 10393 › 36381 › 9 › ...hierarchically structured Peer-to-Peer (P2P) networks with Chord, a scalable P2P

Examination of Routing Algorithms in Distributed Hash ... · ENSC835: COMMUNICATION NETWORKS, FINAL PROJECT PRESENTATION Spring 2008 Peer-to-Peer (P2P) Networks & Distributed Hash

Swarm Robotics Introduction Swarming Testbed …Swarm Robotics Introduction Swarming Testbed Communication Peer to Peer 2010 Peer to Peer 2011 Algorithms Background Path Planning Swarming

Peer to Peer I - Brown Universitycs.brown.edu/courses/cs138/s17/lectures/02peer2peer1.pdf · • Start with Peer-to-Peer systems, ... • Our job is to design systems and algorithms

Knapsack problem based piece-picking algorithms for layered content in peer-to-peer networks

Scalable AOI-cast for Peer-to-Peer Networked Virtual Environments

Approximate algorithms for efficient indexing, clustering, and classification in Peer-to-peer networks Odysseas Papapetrou 18 April 2011 L3S Research Center,

UNIVERSITY OF JYVÄSKYLÄ Topology Management Algorithms in Chedar InBCT 3.2 Peer-to-Peer communication Cheese Factory -project

Peer to peer network schemes and finding algorithms

GRID MODERNIZATION INITIATIVE PEER REVIEW · zonal approach in multiple loosely - and tightly -networked microgrids • Lessons learned and roadmap to develop networked microgrids

Erasmus+ - Europa · Technologies (DLT) such as consensus algorithms, smart contracts; cryptography, peer-to-peer network, privacy technologies, etc… • "Non-technical skills"

An evaluation of piece picking algorithms for layered content in bittorrent-based peer-to-peer systems

1 Scalable AOI-cast for Peer-to-Peer Networked Virtual Environments Jehn-Ruey Jiang, Yu-Li Huang, and Shun-Yun Hu CSIE, National Central University, Taiwan

◦ List the advantages of networked computing relative to stand-alone computing ◦ Distinguish between Client/Server and Peer-to- Peer networks ◦ List elements

Distributed Algorithms for Peer-to-Peer Systems Ronaldo Alves Ferreira PhD Thesis Advisors: Ananth Grama and Suresh Jagannathan Department of Computer

Peer-to-Peer Systems and GRID Computing ID2210 · 2017. 3. 27. · ID2210 - Introduction Topics of Study •Fundamental results in large-scale distributed algorithms. •Overview

A Survey of Search Algorithms for Peer-to-Peer File

Revision Lecture. 2 Topics Peer-to-peer computing –Algorithms & issues –Gnutella –Scalability –Security –Freenet –JXTA –BitTorrent Agent-based computing

Project-Team ASAP As Scalable As Possible: Foundations of ... · 6.4.9. Finding good partners in availability-aware p2p networks 18 6.5. Peer-to-peer wireless autonomic networked