Standardizing Our Drivers Through Specifications: A Look at the CRUD API

STANDARDIZING OUR DRIVERS THROUGH SPECS:

A LOOK AT THE CRUD APIOPENING TUESDAY, JUNE 2ND

IN SELECT CONFERENCE ROOMS.

Jeremy Mikolajmikola

What’s the deal with specs?

A Cursory IntroductionSHAMELESSLY BORROWED FROM…

MongoDB has a lot of drivers…

Drivers Today

APIs evolved over many yearsIdiomatic for their languageInconsistent with each otherSubtle behavioral differences

Our support team loves this!

Specifications

AuthenticationSDAM, server selectionWire protocol, write commandsUser-facing APIs

CRUD, enumeration, $out

End Goals

Consistency across driversBehavior and API

More intuitive documentationIncreased developer productivityGuidance for third-party drivers

End Goals (Continued)

Mitigate Parkinson’s law of triviality

Available on GitHub

Internal WIPs → Released SpecsSee: mongodb/specifications

Mailing lists are still in vogue: and mongodb-drivers mongodb-dev

CRUD Specification

What are we addressing?

Collection-level read/write methods.

Variations in naming and semantics.

Inconsistent functionality and APIs.

Default Behaviors (Exhibit A)

// Updates one documentdb.things.update( { "type": "human" }, { "$set": { "organic": true } });

// Removes all matching documentsdb.things.remove( { "type": "robot" });

Option Names (Exhibit B)

// Updates all documentsdb.things.update( { "type": "human" }, { "$set": { "organic": true } } { "multi": true });

// Removes one matching documentdb.things.remove( { "type": "robot" }, { "justOne": true });

Method Signatures (Exhibit C)

// According to the documentationfunction (query, update, options) { // ...}

// Actual implementationfunction (query, update, upsert, multi) { if ( typeof(upsert) === "object" ) { // Unpack options... }

// ...}

Legacy baggage with update()

CRUD

CreateReadUpdateDelete

insert()find()

update()remove()

But Wait… There’s More!

CRUDCRADBOoMFaM

CreateReadUpdateDeleteCountReplaceAggregateDistinctBulk, One or ManyFind and Modify

Our entirely reasonable andlevel-headed approach…

BEAR WITH ME.

Terminology

Collection: class or interface representing a collection.Spec defines methods to be implemented on this object.

Iterable: some iterable object or structure.Cursor or cursor-like abstraction for reads.

Vector or array for write method arguments.

Operations

Methods to be implemented on the Collection object.

These have required and optional parameters.

Deviations

Spec is flexible with naming and option handling.

This permits idiomaticity. (real word)☜

Naming Deviations

“Root” words are non-negotiable.

batchSize, batch_size ʕ•ᴥ•ʔbatchCount щ（ﾟДﾟщ）

maxTimeMS, maxTime °͡ ʖ͜ °͡maximumTime? (╯°□°）╯︵ ┻━┻

FindOptions, FindArgs ح(ﾟヮﾟ)ﾉQueryParams? (ノಠ益ಠ)ノ彡┻━┻

ordered, isOrdered ᕕ( ᐛ )ᕗ

Option Handling

Required options are positional arguments.

For optional options, you have some options:

Named parametersDictionary or hash objectsOption classes (e.g. UpdateOptions)

May be consolidated, sharedFluent builders for find(), aggregate()

Document when order of operations applies!

What we’re not doing

Method overloading.

Encapsulating params in “Model” classes.

Codifying inconsistent APIs for BC.

Diving into the API

Chapter 1: Reads

Queryingfind(filter: Document, options: FindOptions): Iterable<Document>;

filter is criteria (i.e. $query meta operator).

Support other operators through options.

Iterable is obviously a cursor here.

FindOptions

allowPartialResults: BooleanbatchSize: Int32comment: StringcursorType: CursorTypelimit: Int32maxTimeMS: Int64modifiers: DocumentnoCursorTimeout: BooleanoplogReplay: Booleanprojection: Documentskip: Int32sort: Document

Abstracting Internal Details

CursorType enum may be normal,tailable, or tailable and awaitable.

Today’s wire protocol flags aretomorrow’s command options.

Users shouldn’t care andit’s not worth future API breaks.

Other Read Methodsaggregate(pipeline: Document[], options: AggregateOptions): Iterable<Document>;

count(filter: Document, options: CountOptions): Int64;

distinct(fieldName: string, filter: Document, options: DistinctOptions): Iterable<any>;

AggregateOptions

allowDiskUse: BooleanbatchSize: Int32maxTimeMS: Int64useCursor: Boolean

useCursor default varies by server version.

May affect the kind of Iterable returned.

Diving into the API

Chapter 2: Writes

We’re basically 50% done at this point…

Write MethodsinsertOne(document: Document): InsertOneResult;

insertMany(Iterable<Document> documents, options: InsertManyOptions): InsertManyResult;

deleteOne(filter: Document): DeleteResult;

deleteMany(filter: Document): DeleteResult;

One or many behavior is explicit andfacilitates self-documenting code.

Eliminates inconsistency between multi and justOnedefaults for update() and remove(), respectively.

insertMany()

insertMany is syntactic sugarfor bulkWrite() with inserts.

Spec doesn’t address legacybatch OP_INSERT operations.

InsertManyOptions

ordered: Boolean

Write Methods (Continued)

replaceOne(filter: Document, replacement: Document, options: UpdateOptions): UpdateResult;

updateOne(filter: Document, update: Document, options: UpdateOptions): UpdateResult;

updateMany(filter: Document, update: Document, options: UpdateOptions): UpdateResult;

Same points about explicit,self-documenting code apply.

Trivial to validate if replacement orupdate documents contain operators.

UpdateOptions

upsert: Boolean

Bulk WritesbulkWrite(requests: WriteModel[], options: BulkWriteOptions): BulkWriteResult;

Remember initializeOrderedBulkOp(),or the really old fluent API from 2013?

WriteModel

Also, remember when we said we weren’t doing“model” classes that encapsulate all arguments?

WriteModel

Models include required and optional (if any)arguments from single write methods.

bulkWrite()’s requests argumentallows users to specify all of their writes

at once and in a single method call.

Trivial to make a fluent API atop this,although it’s not in the spec.

UpdateManyModel (f.e.)

filter: Document requiredupdate: Document requiredupsert: Boolean optional

BulkWriteOptions

ordered: Boolean

Basically the same thing as InsertManyOptions.

Result Classes

Insert results may report driver-generated IDsDelete results include countsUpdate results include counts andserver-generated IDs from upsertsBulk results aggregate all of the above

Results are optional for unacknowledged writes.(e.g. Optional<BulkWriteResult>, isAcknowledged boolean)

Since all fields within insert results are optional,insertOne() and insertMany() may be void!

Write Errors

Spec is flexible on how errors are reportedMainly concerned that info is accessibleunder consistent fields and names.Doesn’t address merging errors

We have a bulk write spec for that

Diving into the API

Chapter 3: Find and Modify

I’ll keep this short…

Find and Modify MethodsfindOneAndDelete( filter: Document, options: FindOneAndDeleteOptions): Document;

findOneAndReplace( filter: Document, replacement: Document, options: FindOneAndReplaceOptions): Document;

findOneAndUpdate( filter: Document, update: Document, options: FindOneAndUpdateOptions): Document;

Option classes contain onlythe relevant command options.

Preemptive Q&A

Read Preferences?

Generally, queries on same collectionwill use the same read preference.

Spec assumes it’s set onclient, database, or collection.

Per-operation read preferences are permitted;the spec simply doesn’t define it.

Write Concerns?

Everything we just said about read preferences…

findOne() et al.

Drivers are free to keep existing methodsand add new ones, too.

Please keep options and naming consistent.

Thanks!

FIN.

Questions?

Image Credits

and http://www.instructables.com/id/Egg-Cream/Giphy Reaction Gifshttps://octodex.github.com/wheres-waldocat/