Software Architectures, Week 5 - Advanced Architectures

Week 5

Advanced Architectures

COAST architecture: http://isr.uci.edu/projects/coast/about.html

Data Silos• Typical in many 'small' data architectures.

• Data are persisted in different databases with imposed borders.

• They are created due to organizational boundaries and divergent goals between individuals or teams.

Data Silos (cont)

A silo breaker, the data lake

Data Lake Architecture

Credit: EMC

Data Lake Architecture

Credit: Zaloni

Data Lake vs Data Warehouse

Credit: Martin Fowlerr

Data Lake Architecture, Batch Processing

Credit: EMC

Data Lake Architecture, Stream Processing

https://www.confluent.io/blog/stream-data-platform-1/

Data Lake Architecture, Stream Processing (cont.)


Data Lake Architecture, Stream Processing (cont.)


What happens when we want to combine batch and stream processing?

λ!• Invented by Nathaniel Marz, it is intended to combine the pros of both worlds.

• Processing is separated but persistence is common.

• It introduces more complexity.

• Many more moving parts.

• Start getting out of favour in the last couple of years.

Lambda Architecture (cont.)

Lambda Architecture (cont.)

Lambda Architecture for Web Apps

Credit: Amazon

How far we can go in reducing elements?

Serverless Architecture• Do we still need servers? • - Seems not.

• Do we need storage clusters? • - Seems not.

• Do we need developers?• - Seems yes.

Serverless Architecture (cont.)

Credit: Amazon

Serverless Architecture (cont.)• Serverless systems don't need to be limited to web services or

contellations of microservices.

• They can also be used for Continuous Deployment (CI), Continuous Delivery (CD) or monitoring.

Since we mentioned Serverless Microservices

• Functionality of Lambda functions can be grouped together and form serverless microservices (logically).

• Same principles regarding communication and separation of concerns apply here.

• Operations become much, much simpler and cheaper.

Serverless Microservices (cont.)

Credit: Amazon

Serverless Continuous Delivery• Jenkins is not needed any more.

• There is no limiation in compiling/building capacity (except your credit cards).

• Highly available (no more dead Jenkins workers).

• No risk of over or under-provisioning.

• No OS to maintain or worry about.

https://stelligent.com/2016/03/17/serverless-delivery-architecture-part-1/

Serverless Continuous Delivery (cont.)


Bottom line...


COAST Architecture• COmputAtional State Transfer (COAST).

• The principal goal is the construction of Internet-scale decentralized applications.

• Computation exchange is the bilateral exchange of computations among peers (code to data).

• A PoC has been implemented in Racket.

http://isr.uci.edu/projects/coast/about.html

COAST Architecture (cont.)

http://isr.uci.edu/projects/coast/about.html

Zeta Architecture• An enterprise-scale architecture aiming to meet always rising

expectations.

• Inspired by Google's top-level architecture.

• There are seven pluggable components of the architecture, and all of them must work together.

https://www.oreilly.com/ideas/zeta-architecture-hexagon-is-the-new-circle

Zeta Architecture in Google (cont.)


Poor man's Zeta (cont.)


Zeta Architecture (cont.)• Data protection schemes, backing up data, recovering from failures,

load balancing, and even running multiple versions of software are simplified with Zeta.

• These properties are offered as services and are available to every guest application.

• Resource allocation is optimized and results in lower HW requirements.


Web Service before Zeta


Web Service after Zeta


Unikernels• Unikernels are specialised, single-address-space machine images

constructed by using library operating systems.

• They are built by compiling high-level languages directly into specialised machine images that run directly on a hypervisor.

• Unikernels benefits are improved security, smaller footprints, more optimisation and faster boot times.

http://unikernel.org

Unikernels (cont.)

http://unikernel.org

ClickOS, exercise in lightness• "A minimalistic, tailor-made, virtualized operating system to run Click-based

middleboxes."

• Small (6MB)

• Boot quickly (in about 30 milliseconds)

• Quick response time (45 microseconds)

• Targeting C++

http://cnp.neclab.eu/clickos/

MirageOS, Docker's successor?• The most widely used Unikernel system.

• In production as a microservice platform.

• Unikernel Systems has been acquired by Docker, making it an inherent part of the Docker ecosystem.

• Used in serverless, non-AWS deployment scenarios.

https://mirage.io

Looking in the (not so deep) future

https://mirage.io

Self-organizing Architectures• Self-adaptive software architecture.

• Autonomously exploring all possible architectures that can be used to realise a given software system.

• Monitoring that system in execution in terms of its performance and its operating environment.

• Identifying the optimal architecture for each set of operating environment conditions that are encountered

http://ieeexplore.ieee.org/document/7573144

Train your server

https://www.wired.com/2016/05/the-end-of-code/

Deepcoder Learning to Write Programs

https://arxiv.org/abs/1611.01989

• A Cambridge Project.

• Deep Neural network which is not performing function aproximation but assembling simple pieces of code which approximate the function.

• It can solve the simplest problems on programming competition websites.

Deep Learning Architecture – High Level

http://www.slideshare.net/odsc/arno-candel-scalabledatascienceanddeeplearningwithh2oodscboston2015-48909623

Deep Learning Architecture – Neural Net

http://www.slideshare.net/odsc/arno-candel-scalabledatascienceanddeeplearningwithh2oodscboston2015-48909623

Deep Learning Architecture – Infrastructure

Thank you!

Documents

Software Architectures, Week 5 - Advanced Architectures