12 Factors, AI-ML and Beyond in Java Lohitashwa Thyagaraj STSM, Master Inventor, DevOps & Cloud Automation Architect IBM

Saravanan Devendra STSM, Chief Technology Architect for Cognitive Assist & Automation IBM

Monolithic – Service Oriented - MicroServices

UI

MS MS MS MS

DB DB DB DB

Micro Services

Enterprise Service Bus

UI

Service

DB

Service Oriented

Service Service

Enterprise Service Bus

UI

Business Layer

Integration Layer

Monolith

DB

Monolithic

MicroServices – A quick view

User

Web App

Oauth Server

Api gateway

User Service

Alert Service

Logger Service

Mailer Service

REST

REST

REST Socket

REST

Messaging

REST/Socket

MicroServices – Two sided view…

Oauth Server

• Loosely Coupled

• Independently Deployable

• Organized around business

capabilities

• Highly Maintainable and Testable

• Resilient • Scalable

• Technology Heterogeneity

• Self Sustained

• Carefully handle requests between your

services • Forced to write extra code to avoid

disruption

• Remote calls experience latency

• Multiple database and transaction Mgmt can be

painful • Deploying Microservices can be complex

The Pros The Flip Side

Deploying MicroServices

Application

OS Image Java Base Image Application Image

Deployment Solutions

Multiple Services instances per host

Single Service instances per host

Service instances per VM Service instances per container Host

VM VM

VM VM Container

Serverless Deployment

Hetro Languages

Multiple service instances Independently deployable and scalable

Quickly Build and Deploy Monitor the Behavior Cost effective

Isolation

MicroServices – Discipline – 12 Factor App • 12 factor App is a methodology that outlines rules guiding application developers on how to

build cloud native applications

1. Code base • One codebase tracked in revision control, many deploys

2. Dependencies • Explicitly declare and isolate dependencies

3. Config • Store config in the environment

MicroServices – Discipline – 12 Factor App • 12 factor App is a methodology that outlines rules guiding application developers on how to

build cloud native applications

4. Backing Services • Treat backing services as attached resources

5. Build, Release & Run • Strictly separate build and run Stages

6. Processes • Execute the app as one ore more stateless processes

MicroServices – Discipline – 12 Factor App • 12 factor App is a methodology that outlines rules guiding application developers on how to

build cloud native applications

7. Port Binding • Export services via port binding

8. Concurrency • Scale out via the process model

9. Disposability • Maximize robustness with fast startup and graceful shutdown

MicroServices – Discipline – 12 Factor App • 12 factor App is a methodology that outlines rules guiding application developers on how to

build cloud native applications

10.Dev/Prod Parity • Keep development, staging, and production as similar as possible

11.Logs • Treat logs as event streams

12.Admin Process • Run admin/management tasks as one-off processes

Three major types of machine learning

Supervised

Unsupervised

Reinforcement

Machine Learning Algorithms

14

Java Machine Learning Libraries

15

app developer or SME

pre-trained AI

app developer or SME

transfer learning

data scientist

custom AI

your domain

data

+

There are 3 techniques for building AI

1 2 3

your domain

data

+

pre-Trained model

+

pre-trained model

+

custom model

+

Visual Recognition Natural Language Classifier Tools like Watson Knowledge Studio Speech to Text …

Visual Recognition Natural Language Understanding Speech to Text Text to Speech …

- Deep learning models using WML - Jupyter notebooks - SPSS Modeler

© 2018 IBM Corporation

Machine Learning as a Service (MLaaS)

Adopting artificial intelligence software and services, businesses can enhance product capabilities, better interact with customers, streamline business operations, and create predictive and precise business strategies.

Developers can build quickly and efficiently with MLaaS offerings, because they have access to pre-built algorithms and models that would take them extensive resources to build otherwise.

There are a number of MLaaS offerings for businesses to choose from, including natural language processing (NLP), computer vision, AI platforms and other machine learning APIs

IBM, Microsoft, Amazon & Google are the top companies offers MLaaS

17

18

https://github.com/watson-developer-cloud/java-sdk

Building a Chatbot using IBM Watson Assistant Service

19

import com.ibm.watson.assistant.v1.Assistant; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.util.HashMap; import java.util.List; import java.util.Map; public class BVTService { public static void main( String[] args ) { BufferedReader br = null; MessageResponse response = null; Map context = new HashMap(); try { br = new BufferedReader(new InputStreamReader(System.in)); while (true) { System.out.print(“Enter something : “); String input = br.readLine(); if (“q”.equals(input)) { System.out.println(“Exit!”); System.exit(0); } response = conversationAPI(input,context); System.out.println(“Watson Response:”+ response.getText().get(0)); context = response.getContext(); System.out.println(“———–“); } } catch (IOException e) { e.printStackTrace(); } }

public static MessageResponse conversationAPI(String input,Map context) { ConversationService service = new Assistant(“2017-02-03”); IamOptions options = new IamOptions.Builder() .apiKey("<iam_api_key>") .build(); service.setIamCredentials(options); String workspaceId = “<<Workspace ID>”; MessageInput newMessage = new MessageInput(); newMessage.setText(input); newMessageOptions = new MessageOptions.Builder() .workspaceId(workspaceId) .input(input) .context(context) .build(); MessageResponse response = service.message(newMessageOptions).execute().getResult(); return response; } }

IBM Watson Services

20

21

22 IBM Confidential 3 May 2019

What is 12 Factor App and why it is important – A Modern Day SDLC

A Good piece of Software is always backed by a strong

DevOps pipe line, has a

Platform which supports

the NFR’s, sound processes and most importantly is composed with

Microservices and not Monoliths – And this is where

the 12 Factor Methodology shines

23 IBM Confidential 3 May 2019

What are Microservices & their Design Patterns

“Enterprise Systems has undergone a paradigm shift in how they were designed and built over the last couple of years with the introduction of cloud computing, big data, stream computing, microservices, devops and agile methodologies”

• It is an approach to developing a single

application as a suite of small services, each running in its own process.

• Services often communicate using a HTTP resource API and lightweight mechanisms.

• These services are built around business capabilities and are independently

deploy-able by fully automated deployment machinery.

• There is a bare minimum of centralized management of these services

• Multiple services may be composed together to create composite microservice. This is like service composition.

• Each of the services may be written in

different programming languages and may use different data storage technologies.

24 IBM Confidential 3 May 2019

Microservices Design Patterns

Typical ML Pipe-line

Narrow AI

2010 2015

General AI

2050–Beyond We are here

Broad AI

Advances in AI

© 2019 IBM Corporation

• Learn from less data

• Adapt learning to new domains without forgetting old ones

• Embed security & ethics

• Protect AI data, models, and applications from cyber attacks

• Prevent human biases from propagating to AI systems

• Explain AI decisions

• Substantiate decisions, build trust, and comply with regulations

Making AI robust for enterprises

MicroServices – Discipline – 12 Factor App • 12 factor App is a methodology that outlines rules guiding application developers on how to

build cloud native applications

• Code base • One codebase tracked in revision control, many deploys

• Dependencies • Explicitly declare and isolate dependencies

• Config • Store config in the environment

• Backing Services • Treat backing services as attached resources

• Build, Release & Run • Strictly separate build and run Stages

• Processes • Execute the app as one ore more stateless processes

MicroServices – Discipline – 12 Factor App • 12 factor App is a methodology that outlines rules guiding application developers on how to

build cloud native applications

• Code base • One codebase tracked in revision control,

many deploys

• Dependencies • Explicitly declare and isolate

dependencies

• Config • Store config in the environment

• Backing Services • Treat backing services as attached

resources

• Build, Release & Run • Strictly separate build and run

Stages

• Processes • Execute the app as one ore more

stateless processes

MicroServices – Discipline – 12 Factor App • 12 factor App is a methodology that outlines rules guiding application developers on how to

build cloud native applications

• Port Binding • Export services via port binding

• Concurrency • Scale out via the process model

• Disposability • Maximize robustness with fast startup and graceful shutdown

• Dev/Prod Parity • Keep development, staging, and production as similar as possible

• Logs • Treat logs as event streams

• Admin Process • Run admin/management tasks as one-off processes