Offline first development - Glasgow PHP - January 2016

@glynn_bird

Offline-First DevelopmentGlasgowPHP – January 2016

Glynn Bird, Developer Advocate, IBM

@glynn_bird

@glynn_bird

NoSQL…

@glynn_bird

Relational Databases• Defined Schema

• Tables/Columns/Rows

• Constraints/Stored Procedures/Triggers

• Indexes/Views

• Structured Query Language (SQL)

@glynn_bird

NoSQL Databases• Any database that isn't a RDBMS!• Types include:• Document stores• Key/Value stores• Graph databases• Often schemaless• Often without SQL

@glynn_bird

…On the Move

@glynn_bird6

@glynn_bird7

@glynn_bird8

@glynn_bird

Image Credit: Joan Touzet (@wohali), ASF Member, CouchDB PMC Member

9

@glynn_bird

Frameworks

10

@glynn_bird

Image Credit: Device landscape by Jeremy Keith, on Flickr

11

Not just mobile first…

@glynn_bird

Image Credit: NASA New Horizons

12

Offline First

Offline-First

@glynn_bird

Offline, online and somewhere in-between

13

@glynn_bird

The Eight Fallacies of Distributed Computing1. The network is reliable

2. Latency is zero

3. Bandwidth is infinite

4. The network is secure

5. Topology doesn't change

6. There is one administrator

7. Transport cost is zero

8. The network is homogeneous

14

Text Credit: The Eight Fallacies of Distributed Computing by Peter Deutsch | Image Credit: Pneumatic Central by Sleestak, on Flickr

@glynn_bird15

Offline-first is the only way

to achieve a true, 100% always-on user

experience.**assuming the device is reliable

@glynn_bird

Benefits of Offline First

16

• Better, faster user experience, both offline and online• Allow your users to work offline or with limited connectivity• Potentially saves battery life and bandwidth usage

@glynn_bird

Offline Patterns & Anti-Patterns• Don't return an error for no reason

• Do let users view cached/saved data

• Do synchronize data when connected

• Consider letting your users decide when to sync

• Think about the UX of users seeing stale data

17

@glynn_bird

Difficulties of Offline First

18

Image credit http://www.sneakerheadvc.com/wp-content/uploads/2012/02/Apple_iSync1.png

@glynn_bird

Introducing CouchDB & IBM Cloudant

@glynn_bird

Apache CouchDB• JSON document store

• HTTP API

• Replication

• Free, open-source

20

@glynn_bird

Apache CouchDB – built to replicate• MVCC for document versioning

• Replication

• One-way or Two-way

• One-shot or continuous

21

@glynn_bird

Apache CouchDB – built to replicate

22

2.0

multi-node clustering

Cloudant Geo

Cloudant Query (Mango)

Cloudant Search (Lucene)

Dashboard

@glynn_bird

IBM Cloudant – built for scale

23

@glynn_bird

IBM Cloudant• Globally distributed data layer for

web and mobile applications• Run as-a-service

• MongoDB-style queries

• Advanced geospatial capabilities

• Full text search indexing

24

@glynn_bird

Mobile Apps - PouchDB

@glynn_bird

PouchDB• A database in your web browser

• Can synchronize with any database that implements the CouchDB Replication Protocol

• Makes create, read, update and delete operations extremely quickly

• Free, open-source

26

@glynn_bird

Getting started with PouchDB

27

<script src="https://cdn.jsdelivr.net/pouchdb/5.1.0/pouchdb.min.js"></script>

<script type="javascript">

var db = new PouchDB("smart-meter");

</script>

@glynn_bird

Adding documents - callbacks

28

db.post({ date: "2014-11-12T23:27:03.794Z", kilowatt_hours: 14}, function(err, data) { console.log(err,data);});

@glynn_bird

Adding documents - bring your own id

29

var db = new PouchDB("smart-meter");var obj = { _id: "abc123", timestamp: "2014-11-12T23:27:03.794Z", kilowatt_hours: 14};db.put(obj, callback);

https://github.com/bradley-holt/offline-first/blob/master/pouchdb/04-create-document-put.js

@glynn_bird

Getting a document

30

db.get("abc123", callback);

// calls back with // {// _id: "abc123",// _rev: "1-27695d5f483ac267d03ad0e3cb54bd2c",// timestamp: "2014-11-12T23:27:03.794Z",// kilowatt_hours: 14// }

@glynn_bird

Getting multiple documents

31

db.allDocs({include_docs:true}, callback);

// calls back with // {// "offset": 0,// "total_rows": 24,// "rows": [{...},{...}]// }

@glynn_bird

Querying

32

• Primary Index

• MapReduce

• PouchDB-find plugin

• PouchDB-quick search plugin

@glynn_bird

Querying a Database with PouchDB Find• Based on Cloudant Query (Mango)

• MongoDB-style query language

33

Image Credit: Mango with section on a white background by bangdoll, on Flickr

db.find({ selector: { name: 'Mario' debut: { '$gt': 1990 } }, fields: ['_id', 'lastname'], sort: ['lastname']})...

@glynn_bird

Replicating a PouchDB Database

34

var db = new PouchDB("smart-meter");var remoteDb = new PouchDB("https://bradley-holt.cloudant.com/smart-meter");db.replicateTo(remoteDb);

https://github.com/bradley-holt/offline-first/blob/master/pouchdb/08-replicate-database.js

@glynn_bird

Bidirectionally Replicating a PouchDB Database

35

db.sync(remoteDb).on("change", function(info) { // Replication has written a new document console.log(info); }).on("complete", function(info) { // Replication has completed or been cancelled console.log(info); });

https://github.com/bradley-holt/offline-first/blob/master/pouchdb/09-replicate-database-bidirectional.js

@glynn_bird

Listening for Database Changes

36

var changes = remoteDb.changes({ since: "now"}).on("change", function(change) { // A document has changed console.log(change);}).on("complete", function(info) { // changes() was canceled console.log(info);});

https://github.com/bradley-holt/offline-first/blob/master/pouchdb/11-database-changes.js

@glynn_bird

PouchDB Framework Adapters

37

@glynn_bird

Web Apps Going Offline

@glynn_bird

HTML5 Offline Application Cache• Enables fully-functional offline

web apps

• Stores files and assets for offline browsing

• Makes page loads very fast, even when online

39

@glynn_bird

Cache Manifest File

40

<html manifest="example.appcache"> …</html>

CACHE MANIFEST# v1 - 2015-01-08index.htmllogo.pngapp.cssapp.js

@glynn_bird

Service Workers (Beta)

41

Client-side scripting framework for

•programmable cache

•sync

•push messaging

•geo-fencing

•background tasks

https://jakearchibald.github.io/isserviceworkerready/

@glynn_bird

Hybrid or Responsive Mobile Web Apps

@glynn_bird

Hybrid Mobile Web Apps• Native mobile web apps built with

HTML5, CSS and JavaScript

• Good for:• Fully-featured, cross-platform native apps

• High-fidelity prototypes

43

@glynn_bird

Responsive Mobile Web Apps• HTML5, CSS and JavaScript mobile

web apps

• Responsive design

• Enhanced to enable offline usage

44

@glynn_bird

Native iOS & Android Apps

@glynn_bird

Cloudant Sync• Library for iOS and Android

• Provides local storage and query API

• Optional sync to Cloudant

• Open-source and free to use

@glynn_bird

Cloudant Sync• Stores data using SQLite

• TouchDB provides MVCC

• Replication to Cloudant over HTTPS

• Cloudant Query API

@glynn_bird

Getting started with Cloudant Sync - iOS

48

CDTDatastore *ds = [manager datastoreNamed:@"mydb" error:&error];

pod "CDTDatastore"

• Install library

• Create database

@glynn_bird

Creating data Cloudant Sync - iOS

49

CDTDocumentRevision *rev = [CDTDocumentRevision revisionWithDocId:@"doc1"];

rev.body = [@{ @"description": @"Buy milk", @"completed": @NO, @"type": @"com.cloudant.sync.example.task"} mutableCopy];

CDTDocumentRevision *revision = [ds createDocumentFromRevision:rev error:&error];

@glynn_bird

Fetching data - Cloudant Sync - iOS

50

CDTDocumentRevision *retrieved = [datastore getDocumentWithId:@"doc1" error:&error];

@glynn_bird

Indexing data - Cloudant Sync - iOS

51

// create index on name, age and pet species NSString *name = [ds ensureIndexed:@[@"name", @"age", @"pet.species"] withName:@"basic"]

// create full-text index on name and comment NSString *name = [ds ensureIndexed:@[@"name", @"comment"] withName:@"basic_text_index" type:@"text"];

@glynn_bird

Querying data - Cloudant Sync - iOS

52

NSDictionary *query1 = @{ @"pet.species": @"cat" };

NSDictionary *query2 = @{ @"$or": @[ @{ @"pet.species": @{ @"$eq": @"dog" } }, @{ @"age": @{ @"$lt": @30 } } ]};

CDTQResultSet *result = [ds find:query];

@glynn_bird

Getting started with Cloudant Sync - Android

53

repositories { mavenLocal() maven { url "http://cloudant.github.io/cloudant-sync-eap/repository/" } mavenCentral()}

dependencies { compile group: 'com.cloudant', name: 'cloudant-sync-datastore-core', version:'0.14.0' compile group: 'com.cloudant', name: 'cloudant-sync-datastore-android', version:'0.14.0'}

• Install library

@glynn_bird

Creating a database - Cloudant Sync - Android

54

import com.cloudant.sync.datastore.*;

// Create a DatastoreManager using application internal storage path…

Datastore ds = manager.openDatastore("mydb");

@glynn_bird

Storing data - Cloudant Sync - Android

55

MutableDocumentRevision revision = new MutableDocumentRevision();

Map<String, Object> body = new HashMap<String, Object>();

body.put("animal", "cat");

revision.body = DocumentBodyFactory.create(body);

BasicDocumentRevision saved = ds.createDocumentFromRevision(revision);

@glynn_bird

Pro Forma• Define fields you want to collect

• Renders form saving data to PouchDB

• Replicates data to Cloudant

• Demohttps://glynnbird.github.io/proforma/

56

@glynn_bird

MD• Offline word processor

• Saves Markdown documents to PouchDB

• Replicates data to Cloudant

• Demohttp://mddoc.mybluemix.net/

57

@glynn_bird

Gutenberg• Offline e-book reader

• Replicates book list from server

• Each book is a Cloudant database

• Demohttp://glynnbird.github.io/gutenberg/

58

@glynn_bird

www.glynnbird.com• My home page

• Cloudant database of articles

• Replicated to PouchDB

• Appcache for offline first

• http://www.glynnbird.com/

59

@glynn_bird

Volt• Password vault in a Chrome

extension

• Data stored in encrypted in PouchDB

• Optional back to CouchDB/Cloudant

• https://github.com/glynnbird/volt

60

@glynn_bird

Location Tracker• Stores data locally in PouchDB

• Front end built with AngularJS

• Authentication logic built with Node.js

• User interface built with Leaflet

• Replicates location data to Cloudant

• More info:https://cloudant.com/location-tracker/

61

@glynn_bird

Glynn BirdDeveloper Advocate, Cloud Data Servicesglynn.bird@uk.ibm.com@glynn_birdgithub.com/glynnbird

@glynn_bird

Image Credits

63

• Joan Touzet (@wohali), ASF Member, CouchDB PMC Member<https://twitter.com/wohali/status/595689720933445632>

• Device landscape by Jeremy Keith, on Flickr<https://www.flickr.com/photos/adactio/6153481666>

• NASA New Horizons<https://www.nasa.gov/sites/default/files/thumbnails/image/nh-surface.jpg>

• Pneumatic Central by Sleestak, on Flickr<https://www.flickr.com/photos/dlanod/235990854>

• Mango with section on a white background by bangdoll, on Flickr<https://www.flickr.com/photos/bangdoll/5665235102>

• Grunge Warning Sign - Do Not Read This Sign by Nicolas Raymond, on Flickr<https://www.flickr.com/photos/80497449@N04/7417352980>