Reactive Java for The Realtime - Reactive Javafor The Realtime Java data streaming for the (soft) realtime

  • View
    6

  • Download
    0

Embed Size (px)

Text of Reactive Java for The Realtime - Reactive Javafor The Realtime Java data streaming for the (soft)...

  • April 19, 2019 Dreamix Demo

    Reactive Java for The Realtime

    Trayan Iliev tiliev@iproduct.org http://iproduct.org

    Copyright © 2003-2019 IPT - Intellectual Products & Technologies

    mailto:tiliev@iproduct.org http://iproduct.org/

  • 2

    Reactive Javafor The Realtime  Java data streaming for the (soft) realtime applicatins  Async programming alternatives  Introduction to FRP, Reactive Streams spec  RxJava Project Reactor  REST services with JAX-RS and Reactor  End-to-end non-blocking reactive SOA with Netty  Realtime event streaming to JS clients using SSE  IoT event streaming demo

  • 3

    Reactive Javafor The Realtime  Java data streaming for the (soft) realtime applicatins  Async programming alternatives  Introduction to FRP, Reactive Streams spec  RxJava Project Reactor  REST services with JAX-RS and Reactor  End-to-end non-blocking reactive SOA with Netty  Realtime event streaming to JS clients using SSE  IoT event streaming demo

    Best Exp

    laine d in

    Cod e

  • About me

    4

    Trayan Iliev

    – CEO of IPT – Intellectual Products & Technologies (http://iproduct.org/)

    – Oracle® certified programmer 15+ Y

    – end-to-end reactive fullstack apps with Java, ES6/7, TypeScript, Angular, React and Vue.js

    – 12+ years IT trainer

    – Voxxed Days, jPrime, jProfessionals, BGOUG, BGJUG, DEV.BG speaker

    – Organizer RoboLearn hackathons and IoT enthusiast (http://robolearn.org)

    http://iproduct.org/ http://robolearn.org/

  • 5

    Since 2003: IT Education Evolved. Courses:

     Java SE/Web/EE, JPA / Hibernate, Spring 5

     Reactive event stream processing with Reactor / RxJava / RxJS

     Node.js + Express + React + Redux + GraphQL

     Angular + TypeScript + GraphQL

     SOA & REST HATEOAS

     DDD & Reactive Microservices

    IPT - Intellectual Products & Technologies http://www.iproduct.org

    http://iproduct.org/ http://www.iproduct.org/ http://iproduct.org/

  • Where to Find the Demo Code?

    6

    Reactive IoT demos available @ GitHub: https://github.com/iproduct/ipt-reactive-spring-iot-demo

    YouTube Video for the IoT demo: https://www.youtube.com/watch?v=AB3AWAfcy9U

    https://github.com/iproduct/ipt-reactive-spring-iot-demo https://www.youtube.com/watch?v=AB3AWAfcy9U

  • Data / Event / Message Streams

    7

    “Conceptually, a stream is a (potentially never-ending) flow of data records, and a transformation is an operation that takes one or more streams as input, and produces one or more output streams as a result.”

    Apache Flink: Dataflow Programming Model

  • Data Stream Programming

    8

    The idea of abstracting logic from execution is hardly new -- it was the dream of SOA. And the recent emergence of microservices and containers shows that the dream still lives on.

    For developers, the question is whether they want to learn yet one more layer of abstraction to their coding. On one hand, there's the elusive promise of a common API to streaming engines that in theory should let you mix and match, or swap in and swap out.

    Tony Baer (Ovum) @ ZDNet - Apache Beam and Spark: New competition for squashing the Lambda Architecture?

  • Realtime Event Processing

    9

    Distributed realtime event processing becomes a hot topic:  IoT,  Service/process monitoring,  Realtime analytics, fraud detection  Click stream analytics  Stock-trading analysis  Supply chain and transportation alerts  ...

  • Lambda Architecture - I

    10

    https://commons.wikimedia.org/w/index.php?curid=34963986, By Textractor - Own work, CC BY-SA 4

  • Lambda Architecture - II

    11

    https://commons.wikimedia.org/w/index.php?curid=34963987, By Textractor - Own work, CC BY-SA 4

  • Lambda Architecture - III

    12

     Data-processing architecture designed to handle massive quantities of data by using both batch- and stream-processing methods

     Balances latency, throughput, fault-tolerance, big data, real-time analytics, mitigates the latencies of map-reduce

     Data model with an append-only, immutable data source that serves as a system of record

     Ingesting and processing timestamped events that are appended to existing events. State is determined from the natural time-based ordering of the data.

  • Lambda Architecture: Projects - I

    13

     Apache Spark is an open-source cluster-computing framework. Spark Streaming, Spark Mllib

     Apache Storm is a distributed stream processing – streams DAG

     Apache Apex™ unified stream and batch processing engine.

  • Lambda Architecture: Projects - II

    14

     Apache Flink - open source stream processing framework – Java, Scala

     Apache Kafka - open-source stream processing (Kafka Streams), real- time, low-latency, high-throughput, massively scalable pub/sub

     Apache Beam – unified batch and streaming, portable, extensible

  • Direct Acyclic Graphs - DAG

    15

  • Synchronous vs. Asynchronous IO

    16

    DB

    Synchronous

    A

    A

    B

    B

    DB

    Asynchronous

    A

    B

    C

    D

    A

    B

    C

    D

  • Example: Internet of Things (IoT)

    17

    CC BY 2.0, Source: https://www.flickr.com/photos/wilgengebroed/8249565455/

    Radar, GPS, lidar for navigation and obstacle avoidance ( 2007 DARPA Urban Challenge )

  • IoT Services Architecture

    18

    Devices: Hardware + Embedded Software + Firmware

    UART/ I2C/ 2G/ 3G/ LTE/ ZigBee/ 6LowPan/ BLE

    Aggregation/ Bus: ESB, Message Broker

    Device Gateway: Local Coordination and Event Aggregation

    M2M: HTTP(/2) / WS / MQTT / CoAP Management: TR-069 / OMA-DM / OMA LWM2M

    HTTP, AMQP

    Cloud (Micro)Service Mng. Docker, Kubernetes/

    Apache Brooklyn

    Web/ Mobile Portal PaaSDashboard

    PaaS API: Event Processing Services, Analytics

  • 19

     Performance is about 2 things (Martin Thompson – http://www.infoq.com/articles/low-latency-vp ): – Throughput – units per second, and – Latency – response time

     Real-time – time constraint from input to response regardless of system load.

     Hard real-time system if this constraint is not honored then a total system failure can occur.

     Soft real-time system – low latency response with little deviation in response time

     100 nano-seconds to 100 milli-seconds. [Peter Lawrey]

    What's High Performance?

    http://www.infoq.com/articles/low-latency-vp

  • 20

     Callbacks – asynchronous methods do not have a return value but take an extra callback parameter (a lambda or anonymous class) that gets called when the result is available. Ex.: Swing’s EventListener

     Futures, Promises – asynchronous methods return a (Completable)Future immediately. The value is not immediately available, and the object can be polled Ex.: Callable task

     Reactive Streams (functional, non-blocking) – Observable (RxJava), Flowable (RxJava2), Flux & Mono (Project Reactor):  Composability and readability  Data as a flow manipulated with a rich vocabulary of operators  Lazy evaluation – nothing happens until you subscribe ()  Backpressure – consumer can signal to producer that the rate is high  High level but high value abstraction that is concurrency-agnostic

    How to Do Async Programming?

    Source: https://projectreactor.io/docs/core/release/reference

    https://projectreactor.io/docs/core/release/reference

  • Futures in Java 8 - I

    21

     Future (implemented by FutureTask) – represents the result of an cancelable asynchronous computation. Methods are provided to check if the computation is complete, to wait for its completion, and to retrieve the result of the computation (blocking till its ready).

     RunnableFuture – a Future that is Runnable. Successful execution of the run method causes Future completion, and allows access to its results.

     ScheduledFuture – delayed cancelable action that returns result. Usually a scheduled future is the result of scheduling a task with a ScheduledExecutorService

  • Future Use Example

    22

    Future future = executor.submit( new Callable() {

    public String call() { return searchService.findByTags(tags); }

    } );

    DoSomethingOther();

    try { showResult(future.get()); // use future result } catch (ExecutionException ex) { cleanup(); }

  • Futures in Java 8 - II

    23

     CompletableFuture – a Future that may be explicitly completed (by setting its value and status), and may be used as a CompletionStage, supporting dependent functions and actions that trigger upon its completion.

     CompletionStage – a stage of possibly asynchronous computation, that is triggered by completion of previous stage or stages (CompletionStages form Direct Acyclic Graph – DAG). A stage performs an action or computes value and completes upon termination