Upload
jorge-bay-gondra
View
1.361
Download
2
Tags:
Embed Size (px)
Citation preview
Microserviceswith Node.js and Apache Cassandra
1
Jorge Bay Gondra
@jorgebg
Software Engineer, Drivers team at DataStax
© 2015 DataStax, All Rights Reserved. Company Confidential
Topics
1- Microservices
2- Why Node.js and Cassandra
3- The Node.js driver
4- Putting it all together: killr-service
2
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
Defining microservices
4
- Service as software component
- 1 service per process
- Different data storage per service
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
From monolithic server architecture
5
- Single process contains all the logic
- Simple to develop
- Simple to deploy
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
From monolithic server architecture
6
- Large code bases
- Hard add new features
- Hard to train new developers
- Long term technology commitment
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
Why Microservices
7
- Codebase <---> teams
- Independently deployable: Versioning
- Fault isolation
- Best technology for the job
- Long lived systems, short lived services
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
But some things get harder
8
- Inter-service communication
- Deployment
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
9
Monolithic architecture
UI
Services
Data Access
Browser Monolithic Application Relational database
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
10
Monolithic architecture
Store UI Order UI Customer UI
Catalog Svc Order Svc Customer Svc
Product DA Order DA Customer DA
Browser Monolithic Application Relational database
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
11
Microservices architecture
Browser Front end Services Data storage
Order
Product
Customer
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
12
Inter-service communication
Products
Orders
Message Broker
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
Inter-service communication
13
- Event based
- Async
- Start simple
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
14
Microservices architecture
Browser Front end Services Data storage
Order
Product
Customer
Message broker
© 2015 DataStax, All Rights Reserved. Company Confidential
1- Microservices
Recap
15
- Service as component
- Deploy / Versioning
- Own your data
© 2015 DataStax, All Rights Reserved. Company Confidential
2- Why Node.js and Cassandra
Node.js
17
- Single paradigm: async IO / event loop
- Minimum overhead per connection
- Predictable amount of memory usage under load
© 2015 DataStax, All Rights Reserved. Company Confidential
2- Why Node.js and Cassandra
Cassandra
18
- Tunable consistency
- No master / slave
- Fault tolerant
© 2015 DataStax, All Rights Reserved. Company Confidential
3- The Node.js Driver for Cassandra
Features
20
- Automatic failover
- Node discovery
- Tunable policies
- Request pipelining
- TLS
© 2015 DataStax, All Rights Reserved. Company Confidential
3- The Node.js Driver for Cassandra
Automatic failover
21Client 1
DC 1
DC 2
Requests
© 2015 DataStax, All Rights Reserved. Company Confidential
3- The Node.js Driver for Cassandra
Node discovery
22
Client
DC 1
Request
Events
Response
© 2015 DataStax, All Rights Reserved. Company Confidential
3- The Node.js Driver for Cassandra
Tunable policies
23
- Load balancing
- Retry
- Reconnection
© 2015 DataStax, All Rights Reserved. Company Confidential
3- The Node.js Driver for Cassandra
Load balancing policy sample
24
function BlackListPolicy(blackListedHost, childPolicy) {this.blackListedHost = blackListedHost;this.childPolicy = childPolicy;
}util.inherits(BlackListPolicy, LoadBalancingPolicy);
BlackListPolicy.prototype.newQueryPlan = function (keyspace, queryOptions, callback) {this.childPolicy.newQueryPlan(keyspace, queryOptions, function (err, iterator) {callback(err, filter(iterator));
});};function *filter () {/**/}
© 2015 DataStax, All Rights Reserved. Company Confidential
3- The Node.js Driver for Cassandra
Ecmascript 6
25
- Load balancing policies: generators
- Encoding / decoding: ES6 Map / Set
© 2015 DataStax, All Rights Reserved. Company Confidential
3- The Node.js Driver for Cassandra
Request pipelining
26
Client
Sending first request
(1)
Cassandra Nodetim
e
Received first response
(2)
© 2015 DataStax, All Rights Reserved. Company Confidential
3- The Node.js Driver for Cassandra
Recap
27
- Failover built-in
- Connection pooling
- Defaults
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Functionality
29
- Video Comments
- Video Ratings
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Schema
30
CREATE TABLE comments_by_video (
videoid uuid,
commentid timeuuid,
userid uuid,
comment text,
PRIMARY KEY (videoid, commentid))
WITH CLUSTERING
ORDER BY (commentid DESC);
CREATE TABLE comments_by_user (
userid uuid,
commentid timeuuid,
videoid uuid,
comment text,
PRIMARY KEY (userid, commentid))
WITH CLUSTERING
ORDER BY (commentid DESC);
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Schema
31
CREATE TABLE video_rating (
videoid uuid,
rating_counter counter,
rating_total counter,
PRIMARY KEY (videoid));
CREATE TABLE video_ratings_by_user (
videoid uuid,
userid uuid,
rating int,
PRIMARY KEY (videoid, userid));
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Service methods
32
- GET comments by video
- POST comment
- GET rating by video
- POST rating
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
GET comments by video
33
var cassandra = require('cassandra-driver');var client = new cassandra.Client({ contactPoints: ['localhost']});var query = 'SELECT videoid, commentid, userid, comment FROM comments_by_video WHERE videoid = ?';app.get('/v1/comments/:videoId([a-f0-9\\-]{36})', function (req, res, next) {
client.execute(query, [req.params.videoId], { prepare: true }, function (err, result) {if (err) return next(err);res.json(result.rows);
});});
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
GET comments by video
34
var repository = new Repository(client);//...app.get('/v1/comment/:videoId([a-f0-9\\-]{36})', function (req, res, next) {
repository.getCommentsByVideo(req.params.videoId, function (err, comments) {if (err) return next(err);res.json(comments);
});});
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
POST comment
35
app.post('/v1/comment/:videoId([a-f0-9\\-]{36})', function (req, res, next) {
repository.insertComment(req.params.videoId, req.body.userId, req.body.comment, (err, id) {
if (err) return next(err);
res.send(id.toString());
});
});
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Repository: Insert comment
36
Repository.prototype.insertComment = function (videoId, userId, comment, callback) {
var commentId = cassandra.types.TimeUuid.now();
var queries = [
{ query: 'INSERT INTO comments_by_video (videoid, commentid, userid, comment) VALUES (?, ?, ?, ?)',
params: [videoId, commentId, userId, comment]},
{ query: 'INSERT INTO comments_by_user (userid, commentid, videoid, comment) VALUES (?, ?, ?, ?)',
params: [userId, commentId, videoId, comment]}];
this.client.batch(queries, { prepare: true }, callback);
});
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Repository: Insert comment
37
Repository.prototype.insertComment = function (videoId, userId, comment, callback) {
var commentId = cassandra.types.TimeUuid.now();
var queries = [/*...*/];
var bus = this.bus;
this.client.batch(queries, { prepare: true }, function (err) {
if (!err) {
bus.publishNewComment(videoId, commentId, userId, comment);
}
callback(err, commentId);
});
});
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Message broker client
38
function Bus() {
}
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Message broker client
39
function Bus() {
}
Bus.prototype.publishNewComment = function (videoId, /*...*/) {
//Insert magic here
};
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Message broker client
40
function Bus() {
events.EventEmitter.call(this);
var self = this;
serverLib.on('whatever.topic.event.name', function (data) {
self.emit('whatever-client-name', data);
});
}
util.inherits(Bus, events.EventEmitter);
Bus.prototype.publishNewComment = function () {/*...*/};
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Seneca sample: GET comments
41
seneca.add( {role: 'comment', cmd: 'get'}, function (args, done) {repository.getCommentsByVideo(args.id, done);
});
© 2015 DataStax, All Rights Reserved. Company Confidential
4- Microservices Demo
Recap
42
- 3 classes
- ~150 LoC
- Boundaries
© 2015 DataStax, All Rights Reserved. Company Confidential
Thanks
@jorgebg
github.com/jorgebay/killr-service
datastax.com/dev/blog
bit.ly/nodejs-cassandra-user
43
© 2015 DataStax, All Rights Reserved. Company Confidential
Thanks
Further reading
- Sample project: killr-service
- Presentation: Implementing Micro-Service Architecture by Fred George
- Book: Building Microservices By Sam Newman (O'Reilly)
- Article: Microservices Architecture By James Lewis and Martin Fowler
44