Picking the right API for the right job – Couchbase Connect 2016

©2016 Couchbase Inc. 1

Picking the right API for the right Job


Michael NitschingerSenior Software Engineer, [email protected]

daschl

IMAGE GOES HERE

mailto:[email protected]



daschl

http://www.wienerwald.at/wp-content/uploads/2016/01/schnitzel_pommes.jpg




daschl

http://www.wienerwald.at/wp-content/uploads/2016/01/schnitzel_pommes.jpg



One SDK API to rule them all


One SDK API to rule them all

http://vignette3.wikia.nocookie.net/lotr/images/e/e1/Gollum_Render.png/revision/20141216091433

©2016 Couchbase Inc.©2016 Couchbase Inc.

Services

STORAGE

Couchbase Server 1

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache

Cluster ManagerCluster Manager

Data Service STORAGE

Couchbase Server 2

Managed Cache



Couchbase Server 3

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache



Couchbase Server 4

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Index Service STORAGE

Couchbase Server 5

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


STORAGE

Couchbase Server 6

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Query Service

FTS Service

Index Service


Services

Data Views N1QL FTS Analytics


Services

DataBinary11210

ViewsJSON8092

N1QLJSON8093

FTSJSON8094

AnalyticsJSON8095


Balance

10https://upload.wikimedia.org/wikipedia/commons/b/b7/Unico_Anello.png

Language Specifics Uniformity across SDKs


Balance



Balance



Balance



Balance


©2016 Couchbase Inc. 15©2016 Couchbase Inc.

RFCs

• Semi-Formal process to advance SDKs in both API and behavior• One Shepherd• Collaborative consensus (hard!) compromises• Public: https://github.com/couchbaselabs/sdk-rfcs

https://github.com/couchbaselabs/sdk-rfcs

https://github.com/couchbaselabs/sdk-rfcs


Prioritizing Uniformity

• Program Flow


Prioritizing Uniformity


Prioritizing Language Specifics


Navigating the API Landscape


Key/Value

• Fastest way to access & mutate documents• Access by Document ID only• Includes subdocument manipulation• Efficient binary protocol

Latency Flexibility


Key/Value – Full Doc Access


Key/Value – Full Doc Access


Key/Value – Sub Doc Access


Views

• Distributed Map-Reduce• Great for Aggregations (fast)• Spatial Support

Latency Flexibility


Views


N1QL

• Very flexible, but get your index right• Global Index instead of Map/Reduce• Similar to SQL

Latency Flexibility


N1QL


Full Text Search

• Very Flexible, but more concepts to learn initially• Inverted Index• Still Developer Preview• Similar to Elasticsearch

Latency Flexibility


Full Text Search


Analytics

• Highly capable analytical storage engine• Similar to N1QL for querying• Performs great on huge datasets and aggregations• Developer Preview announced at Connect!

Latency Flexibility


Streaming/Changes Feed

• Based on the DCP protocol• Very fast and efficient• Low level -> Building Block

Latency Flexibility


Michael’s Super Sophisticated Flowchart to Success

Know the Doc ID(s)? SuccessUse KV

Pre GroupedAggregations?

Use Views

Need Full Text Queries?

Use FTS Use N1QL





Use Views


Use FTS Use N1QL





Use Views


Use FTS Use N1QL



Know the Doc ID(s)? SuccessKV


Views


FTS N1QL


(Not So) Dirty Tricks for Fun and Profit


The “Cache It While Its Hot”

• Transparently cache expensive N1QL/FTS/View results with a TTL and load while hot


The “Cache It While Its Hot”

• Transparently cache expensive N1QL/FTS/View results with a TTL and load while hot


The “Custom Bulk Loader”

• Perform a N1QL, FTS or View query to just fetch the doc IDs, then perform a bulk KV fetch


The “Custom Bulk Loader”

• Perform a N1QL, FTS or View query to just fetch the doc IDs, then perform a bulk KV fetch


The “Tarzan of the Docs”

• “Leap” from Doc to Doc via references and use bulk fetches for higher throughput

• Scales with increasing batch size• Allows to “code” intermediate steps with UDFs


Summary

TradeoffLatency vs. Flexibility

RememberPowerful Patterns

Unsure?Think of the Flowchart

Participatein the RFCs!


Thank You!