Mobile Offline First for inclusive data that spans the data divide

Presenter Full Name, Title, Company

CONSTRAINTSConstraints with modern web-based applications

Applications in the old days

We have come a long way since the client server days in which applications were installed on each computer. No access to the network, meant no access to the application

Applications over the web

Web based applications have changed application design for government introducing 3(+) tiered

applications.

No/poor access to the internet means no/poor access to the application in the current state.

Constraints with current application architectures

1. Latency or poor internet gives a bad user experience: connectivity

2. The data architecture is closely related to the application server design: data silos

Client applications

Application / Web servers

Data

Web clients

Result of application architectureData Silos

OFFLINE FIRST!New application architecture pattern

New application design approach

Data cloud, sync gateway, services processors

Offline First Design Approach

Separate the application into two distinct components:1. Data objects together

with the business rules relating to the specific data object

2. Business processes that tie the data objects together

Global services

processorsData

Web clients

services Data

Mobile

Dataservices

Self Aware Data Objects [SDO](Micro – data – applications)

• The key to the approach is that (most) applications can be re-modelled as a set of atomic data objects

• Declaratively using xml (or json) to define the SDO. (Relational databases are not suitable)

• With javascript and html renderers to interpret and execute the SDO as a micro application.

SDO wrapperIn global SDO

registry

View models with business

rules

Data model (json)

Json schema to validate

New NoSQL database technologyis a pre-requisite for the architecture

1. NoSQL database architecture stores data as documents

2. Data in the document stored as hierarchies and arrays

3. Inbuilt local datastores and automatic replication and version control enable offline work out of the box

4. Open source and commercial5. Relational databases not suitable

Document centric data modelling

Local data store with replication provides for decentralised data

Database cluster with replication provides for global data cloud

SELF AWARE DATA OBJECTSCore of the new application architecture pattern

SDO Essentials: Define and render the SDO

SDO declaratively specifies• business rules• data models• user interfaces

for editing and viewing

• Schemas for validation

Rendered

1. Html and javascript templates to view and edit objects

2. Data objects stored in single json document in a generic format that is applied to all data objects

SDO definition objects: Template in XML

SDO definition objects: Datamodel in XML

SDO Essentials: SDO data objects

Data objects stored in single json document• Wrapper (derived from the atom rss feed model) • Data model (sequenced for history)• With a universal uuid key to allow global data access

and referencing• And a global category referencing the registry of data

objects• Links to link to other data objects and therefore

create a network of data objects

SDO Example: Budget object – Non edit view

SDO Example: Budget object – Edit view

SDO Benefits: Enable offline work

Data objects stored in couchdb / couchbase are replicated automatically to all the ‘designated’ devices• The edit / view models are stored in the database as well and

replicated seamlessly• Because all the business logic is contained in the SDO view and

data models (and schema) it can be executed anywhere• Synchronisation processor can pre-process any new arriving

document to verify eligibility and access rights to edit the document

• Data may be encrypted within the document without affecting the wrapper and therefore the ability to manage the document

SDO Benefits: Enable data integration

Data objects stored in database are defined in terms of the global data registry• The objective is to define data objects and data

models globally, and share these objects between applications and users

• Assemble new applications using global data objects. Inherit and extend where appropriate

• Or import / export from these data objects using API to the data

EXAMPLESNew application architecture pattern in practice

GIZ Harmonization of M&E Data

• This project is a prototype implementation of the architecture needed to manage a single data cloud

• DPME (Ministerial Data Centre) is the department that is driving this initiative

• Consists of a Data Registry prototype to register users, organisations, applications and SDOs

• Provision for users to adopt SDO, and make data from SDOs available to other users

• API to add data to the data cloud

DEA Working for Water integration to EPWP

• One of the examples of the Harmonisation project• A set of 20 SDOs have been defined for the EPWP3

reporting system• Working for Water data is captured on client server

style applications into SQL servers. Databases backed up and shipped to DEA

• Using the API data objects are migrated into the EPWP3 couchbase cloud

DEA data integrated, by default available offline on mobile or laptop

DEA data integrated, by default available offline on mobile or laptop

• EPWP3 reporting system has mobile offline component

• Register participants• Biometrically enabled• Foundation for a

government wide beneficiary master database

THANK YOUWillem van der Westhuizen, Kwantu Information

Technology