© Copyright SELA software & Education Labs Ltd. | 14-18 Baruch Hirsch St Bnei Brak, 51202 Israel | www.selagroup.com

SELA DEVELOPER PRACTICEMAY 31 – JUNE 4, 2015

Nir NoyConsultant, Web, Sela@noynir

Node.js

Agenda

Introduction - What is Node.js.File system.Building servers.Building APIs using modules, events and packages.Express Web framework.

Introduction – What is Node.js ?

What is Node.js

A JavaScript runtime that is designed for asynchronous IO operationsVery lightweight and fastBeing used by a growing number of companies:

The Node.js ecosystem

Node.js has a rapidly growing ecosystem:Web frameworks:

ExpressSocket.io

Database supportSql DatabasesNoSql Databases

Hosting and Cloud environmentsIIS, AzureHerokuJoyent

Synchronous server operations

// GET api/countriespublic string Get(){

var client = WebRequest.Create("http://.../");

var response = client.GetResponse();var stream = response.GetResponseStream();var reader = new StreamReader(stream);return reader.ReadToEnd();

}Blocking I/O operation

The Cost of I/O

I/O Operations are expensive.

I/O Source CPU Cycles

L1 – Cache 3

L2 – Cache 14

RAM 250

Disk 41,000,000

Network 240,000,000

Handling I/O

Waiting for I/O operations to complete is one of the most common performance bottlenecks.There are several common solutions to handle it:

SynchronousPros: simple.Cons: blocking, can hold up other requests.

Fork a new processPros: efficient, not holding up other requests.Cons: does not scale, ~100 connections=~100 processes .

Handling I/O

Threads:Pros:

efficient, not holding up other requests but more lightweight then executing another process

Cons: memory-expensive - ~1MB per thread, complicated, need to worry about controlling access and shared resources.

Traditional web-servers (IIS, Apache) today use an optimized thread-per-connection model that uses thread pools.

Handling I/O in Node.js

Node.js runs your code on a single thread.All I/O operations are asynchronous and Non-Blocking

Internally threads and processes are used but not explicitly exposed to your code.

When an I/O operation completes, an event is triggered and the result is passed to a callback function.

Why Node.js

Lightweight and efficientusing non-blocking I/O keeps memory consumption stable by reducing the number of threads.Less context switching overhead.

Concurrencyusing non-blocking I/O keeps the main thread responsive and allowing it support tens of thousands of concurrent connections.

One Languageallows reuse of resources between the client and server.

Demo

Async server in Node.js

What Node.js code looks like

Node.js Event loop

I/O operations are evented and external events are handled by Node’s Event loop.

Javascript functions are passed to a queue and invoked when the return value from their current I/O calls is available.

I/O Callbacks are invoked synchronously by the order they returned.

Node.js Event loop

Server

Register Callback

AsyncWorkers

DDB, FS, Network,

etc…

Request

Response

Request

ResponseRegister Callback

Demo

Node.js single thread

Node.js Usage

Running CPU intensive tasks (e.g. long running loops) on Node.js’s main thread should be avoided as it will block handling other requests to the server.

Running CPU intensive tasks can be scaled out to multiple node processes.

Node.js can run a managed cluster of multiple processes to scale out an entire application.

The Node.js process

Unlike client-side JavaScript Node.js does not run inside a browser window.

There are no window or document global objects

The Node process provides the runtime for node programs

Node.js process architectureNode.exe

V8

CommonJS – Module loader

JavaScript Application

libuv

OS

Core Javascript modules for file system, network, http, etc…

Responsible for loading modules and manage dependencies. Using CommonJS’s module definition.

Multi-platform library written in C that handles all async I/O operations.

Google’s Javascript Engine.Compiles the Javascript code to native machine code.

The process global object

The process global object provides an API for interacting with the process

Access command-line argumentsGet the execution path, current platform, node.js versionEnvironment variablesAccessing standard input/output

Demo

Running Node from the command line

File System

File System

Node.js provides the fs module for file system operations:

Reading\writing filesGetting file metadataReading directoriesCreating a file watcher

Most fs functions have synchronous APIs

Reading Directories

var fs = require('fs');

// this method reads the content of a directoryfs.readdir(process.argv[2], function(err,files){

files.forEach(function(file){

//Do something with file…

});});

Demo

Inspecting directory’s content using the fs module

Reading files

The fs module prvides different methods for reading file content:

readFileSync reading the whole file synchronouslyreadFile reading the whole file asynchronouslycreateReadStream provides a stream based API for reading files

var fs = require('fs');

//Asyncfs.readFile(process.argv[2],'utf-8', function (err, data)

{ if (err) throw err;

console.log(data)});

//Read file syncvar f = fs.readFileSync(process.argv[2],'utf-8');console.log(f);

Reading Files

Demo

Different ways to read files

StreamsJust like other frameworks Node.js use streams as an abstraction for reading/writing data.

Just like most IO in Node.js, streams provide an event base API using the on method

close, opendataerror

When using streams, data is read in chunks and doesn’t block the main thread for the entire time the data is buffered into memory.

Streams (cont.)var fs = require('fs');

var strm = fs.createReadStream(process.argv[2]);strm.setEncoding('utf8');

strm.on('data', function(data) { console.log('read from strm');});

strm.on('end', function(){console.log('ended strm')

})

The pipe method allows you to connect a readable stream as the destination of a writeable stream

var fs = require("fs"); // Read Filefs.createReadStream("input/people.json")    // Write File    .pipe(fs.createWriteStream("output/people.json"));

Stream.pipe

Demo

Fun with standard IO

Lab

FS Module

Building servers

Building HTTP servers

Node.js was originally designed as a platform for building web servers.

Most Web applications are I/O Bound~90% of a request’s processing time is spent on waiting for I/O to complete.

Node.js async I/O mechanism provides node with high concurrency and a low memory footprint.

Building HTTP servers (cont.)

Node.js is designed to be scaled out to multiple processes and machines which makes it ideal for cloud hosting.

The http module provides basic functionality for handling HTTP (client and server)

Additional frameworks like Express and Restify provide higher level abstractions (like MVC)

var http = require('http');

http.createServer(function(req,res){

var response = 'Hello World'; res.writeHead(200, { 'Content-Type': 'text/plain', 'Content-Length':

response.length }); res.write(response); res.end();

}).listen(3000);

Node Http Server

Demo

Hello HTTP

The http.Server class

The http.createServer() method is used to create a server.

It returns an instance of the http.Server object.

The http.Server provides a set of eventsrequest – fires on every incoming request.Connection -fires on every new tcp connection.Upgrade - fires on every incoming upgrade request.

Demo

Handling the ‘connection’ event

Receiving data

HTTP allows server applications to receive data:

Using the request URIMessage body

Use the url and querystring modules for parsing URLs and query strings.Message body can contain any format

Being a JavaScript runtime, Node.js process JSON natively Other formats are supported through npm packages.

var http = require('http');var url = require('url') ;

http.createServer(function (req, res) {

var queryObject = url.parse(req.url,true).query; console.log(queryObject); res.writeHead(200); res.end('Feel free to add query parameters to the end

of the url');

}).listen(8080);

Parsing URL and Querystring

Demo

Using the URL and QueryString APIs

Http streaming// handling incoming HTTP requests var handleRequests = function(req,res){

// creating an outgoing HTTP request req = http.request(options, responseCallback =

function(response) {

var str = '';response.on('data', function (chunk) {

str += chunk; });

response.on('end', function(){ res.end(str);

}); }); req.end();

};

Chunked transfer encoding

HTTP 1.1 introduced Chunked transfer encoding that allows messages to be sent in chunks.

The Content-Length header is removed and a Transfer-Encoding:chunked header is added to the response.

HTTP can send messages in chunks Write data in small chunks keeping a small memory footprint Freeing node’s single thread to serve more requests

var http = require('http');var handleRequests = function(req,res){ // creating an outgoing HTTP request var req2 = http.request(options, responseCallback =

function(response) {

res.writeHead(200, {'content-type': 'text/xml', 'Content-Encoding':'gzip'})

response.on('data', function (chunk) { res.write(chunk); }); response.on('end', function(){ res.end(); }); });

req2.end();};http.createServer(handleRequests).listen(3000);

Chunked transfer encoding

Demo

HTTP Chunked transfer

Lab

Simple Http Server

Building APIs using modules, events and packages

CommonJS Modules

Node.js uses CommonJS modules for structuring code.

Modules are defined in separate .js files

To consume a module use the require function

Modules and Scopes

Modules provide a simple mechanism for creating scopes in JavaScript

Members defined inside a module are not accessible outside

To expose a member use the exports object

This allow CommonJS to simplify dependency resolution

Demo

Defining and Using Modules

The Global object

Unlike browsers the top-level scope in Node.js is the module not global

To expose objects globally (across modules) you can add them to the global object

Demo

Exposing Objects Globally

Node Package Manager (NPM)

The Node Package Manager (NPM) provides a management mechanism for modules:

Download and installVersion managementDeployment

Demo

Getting Express.js

Managing Dependencies

NPM packages are managed in a file called package.jsonpackage.json schema define fields like:

name, descriptionVersionDependencies

Dependencies are being resolved during NPM install

{ "name": "app", "version": "0.0.0", "private": true, "scripts": { "start": "node ./bin/www" }, "dependencies": { "body-parser": "~1.10.2", "cookie-parser": "~1.3.3", "debug": "~2.1.1", "express": "~4.11.1", "jade": "~1.9.1", "lodash": "^3.3.1", "morgan": "~1.5.1", "serve-favicon": "~2.2.0" }}

A look at Package.json

Demo

A look at package.json

Managing Dependencies

NPM packages are installed inside a folder named node_modules

The folder is created inside the path where the “npm install …” was executed.

When a npm module is required node will look it up inside the node_modules folder of the current path.

If the module is not found it is recursively searched up the folder tree.

Demo

Modules Dependency Resolution

The EventEmitter

Most of Node.js's APIs are event based

The events module provides the event emitter object for emitting events.

Events are fired through the emit method and can be watched using the on method.

Demo

Emitting Events

Lab

Being Modular

Express Web framework

Introduction to Express

Node.js http module provides bare-bones HTTP server functionalityBuilding a modern web application server would require implementing many common features such as:

RoutingSession ManagementCookies & Requests ParsersRequest Logging

Introduction to ExpressExpress is a web application framework inspired by Sinatra.Express middleware provides an expendable Http pipeline on top of Node.js's httpServerAn Express application is essentially a series of middleware callsA middleware is simply a function with access to the request object, response object and a callback to next middleware in line.

Express is available through npm

$ npm install express --save

Express provides a generator tool  to quickly create an application skeleton.

$ npm install express-generator –g

//run $ express -h

Express Installation

Demo

Simple Express Server

Routing is one of the pillars of ExpressTo create a route use the app.verb convention

// respond with "Hello World!" on the homepageapp.get('/', function (req, res) { res.send('Hello World!');})

// accept POST request on the homepageapp.post('/', function (req, res) { res.send('Got a POST request');})

// accept PUT request at /userapp.put('/user', function (req, res) { res.send('Got a PUT request at /user');})

Routing

Routers

An Express application might have a large number of routes defined.

Maintaining All of them in a single file can be a pain.

Routers helps you keep your routes clean, organized and modular.

Routers are defined in a separate node modules and used as middlewares.

A router is an isolated instance of middlewares and routes.

The router can have middleware and http VERB routes added just like an application.

var router = express.Router([options]);

router.get('/events', function(req, res, next) { // .. some logic here .. like any other middleware // ..});

Routers (cont.)

Demo

Simple routing

Routing and Parameters

Express support parameters as part of the URI

Declare a parameter in the URI by using a “:” in front of itAccess query-string parameters using req.query Use regular expressionsTo set a parameter as optional add a “?” suffix.

router.get('/user/:username',function(req,res){//Do something

})

router.get("/page1/:id?",function(req,res){req.params.id=req.params.id || 0;

res.send("query: "+ JSON.stringify(req.query));});

Routes With Parameters

Demo

Passing Parameters to Routes

Express application settings can be set using app.set() and retrieved using app.get() You can also set middleware functions using app.use().

// this middleware will be executed for every request to the app

app.use(function (req, res, next) { console.log('Time: %d', Date.now()); next();})

Configuring Express

Views are template based UI mechanismExpress support many view-engines including:

JadeJSHtmlEJS

View engines are defined by setting the ‘view-engine’ variable

app.set('view engine', 'jade');

// view engine setupapp.set('views', path.join(__dirname, 'views'));

Views

Jade

Jade is Express default view-engine

It is based on Haml it provide a clean syntax for generating HTML

Tab basedFull support for HTML

doctypehtml

headtitle My Page

bodydiv.jumbotron

h1#title This is my Pagep how do you like my styling?

Jade template

Demo

Basic Jade Template

Jade fully support JavaScript using the script element:

Linking to external JS filesEmbedding in-line JavaScript

To setup static content serving in Express use the built-in static middleware.

app.use(express.static(__dirname+'/public'));// multiple static folders can be defined.

Mixing up JavaSript & Jade

Demo

Jade and bootstrap

Blocks

In most applications, some UI components are used throughout the application

Blocks provide a way to provide a common layout that is shared among views

doctypehtml

headtitle My Applicationlink(href='/css/bootstrap.min.css',

rel="stylesheet")link(href='/css/bootstrap-theme.min.css',

rel="stylesheet")body

div.page-headerh1 This is my Application

div.container-fluidblock content

Jade Layout Template

extends layoutblock content

div.containerp this is a containerdiv and some more text here

Jade Template Using Layout

Demo

Using Blocks

Model

Express provides a mechanism to insert data-models into views

The rendering of the complete artifact is up to the view engine

extends layout

block contentdiv.container.bg-danger

h1 oops...,Server errorp #{error}

Rendering data in Jade

Demo

Rendering Data with Jade

Functionality to Express apps are added via third-party middleware installed through npm.

$ npm install cookie-parser

var express = require('express');var app = express();var cookieParser = require('cookie-parser');

// load the cookie parsing middlewareapp.use(cookieParser());

Express Middlewares

Common Express MiddlewaresExpress 4.0 Name

-body parser

compression

-cookie session

morgan

-cookie parser

-express session

-static favicon

-response time

errorhandler

-method override

-connect timeout

vhost

csurf

var express = require('express')var bodyParser = require('body-parser')

var app = express()

// parse application/jsonapp.use(bodyParser.json())

app.use(function (req, res) { res.setHeader('Content-Type', 'text/plain') res.write('you posted:\n') res.end(JSON.stringify(req.body, null, 2))})

The body-parser middleware

var express = require('express')var session = require('express-session')var app = express()

app.use(session({ secret: 'keyboard cat', resave: false, saveUninitialized: true}));

app.use(function(req, res, next) { var sess = req.session if (sess.views) { sess.views++ } else { sess.views = 1 res.end('welcome to the session demo. refresh!') }})

Express-Session

Demo

Login with Express

Lab

Express App

Summary

Node.js is not just for building serversIt allows JavaScript to run outside of browsers and perform asynchronous IOIt is asynchronous by natureNode.js runs your code on a single thread.Lightweight, fast and extremely cool.

nirn@sela.co.il | @noynir

Questions