Error Handling Best Practices

NODE.JS

Yoni Goldbergwww.goldbergyoni.com@nodepractices@goldbergyoni

In this presentation

YONI GOLDBERG2 |

Agenda: The following slides summarize and curate most of the knowledge and patterns gathered to date on Node error handling. It contains more than 35 quotes, code examples and diagrams from the highest ranked blog posts and StackOverflow threads.

Why is this important: Node error handling embodies unique challenges and gotchas – without clear understanding and strategy it might be the Achilles heel of your app

You may read the 10 top ranked blog posts on error handling OR watch the following slides which summarizes them all

Summarizes and curates 6 blog posts

Use promises for async error handling

1

1

2

3

4

5

6

doWork()

.then(doWork)

.then(doOtherWork)

.then((result) => doWork)

.catch((error) => throw error)

.then(verify);

TL;DR

YONI GOLDBERG4 |

Handling asynchronous errors in callback style is probably the fastest way to hell (a.k.a the pyramid of doom). The best gift you can give to your code is using instead a reputable promise library which provides much compact and familiar code syntax like try-catch

Code example – using promises to catch errors

1

2

3

4

5

6

doWork()

.then(doWork)

.then(doOtherWork)

.then((result) => doWork)

.catch((error) => throw error)

.then(verify);

Otherwise

YONI GOLDBERG5 |

Node.JS callback style, function(err, response), is a promising way to un-maintainable code due to the mix of error handling with casual code and over-nested code patterns, see below

Anti pattern code example – callback style error handling

1

2

3

4

5

6

7

8

9

10

11

12

getData(someParameter, function(err, result){

if(err != null)

//do something like calling the given callback function and pass the error

getMoreData(a, function(err, result){

if(err != null)

//do something like calling the given callback function and pass the error

getMoreData(b, function(c){

getMoreData(d, function(e){

if(err != null)

//you get the idea? });

});

One paragraph explainer

YONI GOLDBERG6 |

Callbacks don’t scale as they are not familiar to most programmers, force to check errors all over, deal with nasty code nesting and make it difficult to reason about the code flow. Promise libraries like BlueBird, async, and Q pack a standard code style using RETURN and THROW to control the program flow. Specifically, they support the favorite try-catch error handling style which allows freeing the main code path from dealing with errors in every function

From the blog pouchdb.com, ranked 11 for the keywords “Node Promises”

…And in fact, callbacks do something even more sinister: they deprive us of the stack, which is something we usually take for granted in programming languages. Writing code without a stack is a lot like driving a car without a brake pedal: you don’t realize how badly you need it, until you reach for it and it’s not there. The whole point of promises is to give us back the language fundamentals we lost when we went async: return, throw, and the stack. But you have to know how to use promises correctly in order to take advantage of them.

“We have a problem with promises”

What other bloggers say

YONI GOLDBERG7 |

From the blog pouchdb.com, ranked 11 for the keywords “Node Promises”

…And in fact, callbacks do something even more sinister: they deprive us of the stack, which is something we usually take for granted in programming languages. Writing code without a stack is a lot like driving a car without a brake pedal: you don’t realize how badly you need it, until you reach for it and it’s not there. The whole point of promises is to give us back the language fundamentals we lost when we went async: return, throw, and the stack. But you have to know how to use promises correctly in order to take advantage of them.

Blog Quote: “We have a problem with promises”

From the blog gosquared.com, ranked 5 for the keywords “Node.JS error handling”

…The promises method is much more compact, clearer and quicker to write. If an error or exception occurs within any of the ops it is handled by the single .catch() handler. Having this single place to handle all errors means you don’t need to write error checking for each stage of the work.

Blog Quote: “The promises method is much more compact”

What other bloggers say

YONI GOLDBERG8 |

From the blog StrongLoop, ranked 7 for the keywords “Node.JS error handling”

…Callbacks have a lousy error-handling story. Promises are better. Marry the built-in error handling in Express with promises and significantly lower the chances of an uncaught exception. Promises are native ES6, can be used with generators, and ES7 proposals like async/await through compilers like Babel

Blog Quote: “Promises are native ES6, can be used with generators”

From the blog Benno’s, ranked 13 for the keywords “Node.JS error handling”

…One of the best things about asynchronous, callback based programming is that basically all those regular flow control constructs you are used to are completely broken. However, the one I find most broken is the handling of exceptions. Javascript provides a fairly familiar try…catch construct for dealing with exceptions. The problems with exceptions is that they provide a great way of short-cutting errors up a call stack, but end up being completely useless of the error happens on a different stack…

Blog Quote: “All those regular flow control constructs you are used to are completely broken”

Summarizes and quotes 6 sources

Use only the built-in Error object

2

12

3

45

if(!productToAdd)

throw new Error(“invalidInput“ , 400, “No product provided”);

TL;DR

YONI GOLDBERG10 |

Many throws errors as a string or as some custom type - this complicates the error handling logic and the interoperability between modules. Whether you reject a promise, throw exception or emit error – using only the built-in Error object will increases uniformity and prevents loss of information

Code example - doing it right

12345

//'throwing' an Error from a Promise return new promise(function (resolve, reject) {

Return DAL.getProduct(productToAdd.id).then((existingProduct) =>{ if(existingProduct != null)

reject(new Error("Why fooling us and trying to add an existing product?"));)};

Otherwise

YONI GOLDBERG11 |

When invoking some component, being uncertain which type of errors come in return – makes it much harder to handle errors properly. Even worth, using custom types to describe errors might lead to loss of critical error information like the stack trace!

Code example - Anti Pattern

123

//throwing a String lacks any stack trace information and other important properties if(!productToAdd) throw ("How can I add new product when no value provided?");

//throwing as custom object also lack the stack traceif(price < 0)

throw {errorType:invalidInput};

One paragraph explainer

YONI GOLDBERG12 |

The permissive nature of JS along with its variety code-flow options (e.g. EventEmitter, Callbacks, Promises, etc) pushes to great variance in how developers raise errors – some use strings, other define their own custom types. Using Node.JS built-in Error object helps to keep uniformity within your code and with 3rd party libraries, it also preserves significant information like the StackTrace. When raising the exception, it’s usually a good practice to fill it with additional contextual properties like the error name and the associated HTTP error code. To achieve this uniformity and practices, consider extending the Error object with additional properties (see code examples on next slides)

From the blog devthought.com, ranked 6 for the keywords “Node.JS error object”

…passing a string instead of an error results in reduced interoperability between modules. It breaks contracts with APIs that might be performing instanceof Error checks, or that want to know more about the error. Error objects, as we’ll see, have very interesting properties in modern JavaScript engines besides holding the message passed to the constructor…

  

“A string is not an error”

Code Example – extended Error object

YONI GOLDBERG13 |

Code example – doing it even better with extended error object

1

2

3

4

5

6

7

8

9

10

11

12

//centralized error object that derives from Node’s Error

function (name, httpCode, description, isOperational) {

Error.call(this);

Error.captureStackTrace(this);

this.name = name;

//...other properties assigned here

};

appError.prototype.__proto__ = Error.prototype;

module.exports.appError = appError;

//client throwing an exception

if(user == null)

throw new appError(commonErrors.resourceNotFound, commonHTTPErrors.notFound, "further explanation", true)

What other bloggers say

YONI GOLDBERG14 |

From Node.JS official documentation

…All JavaScript and System errors raised by Node.js inherit from, or are instances of, the standard JavaScript Error class and are guaranteed to provide at least the properties available on that class. A generic JavaScript Error object that does not denote any specific circumstance of why the error occurred. Error objects capture a “stack trace” detailing the point in the code at which the Error was instantiated, and may provide a text description of the error. All errors generated by Node.js, including all System and JavaScript errors, will either be instances of, or inherit from, the Error class…k.

Blog Quote: “All JavaScript and System errors raised by Node.js inherit from Error”

From the blog Ben Nadel, ranked 5 for the keywords “Node.JS error object”

Personally, I don’t see the value in having lots of different types of error objects – JavaScript, as a language, doesn’t seem to cater to Constructor-based error-catching. As such, differentiating on an object property seems far easier than differentiating on a Constructor type

Blog Quote: “I don’t see the value in having lots of different types”

What other bloggers say

YONI GOLDBERG15 |

From the blog machadogj, ranked 6 for the keywords “Node.JS error management”

…One problem that I have with the Error class is that is not so simple to extend. Of course you can inherit the class and create your own Error classes like HttpError, DbError, etc. However that takes time, and doesn’t add too much value unless you are doing something with types. Sometimes, you just want to add a message, and keep the inner error, and sometimes you might want to extend the error with parameters, and such…

Blog Quote: “Inheriting from Error doesn’t add too much value”

Summarizes and quotes 4 sources

Distinguish operational vs programmer errors

3

1

23

4

5

//error handling code within middleware

process.on('uncaughtException', function(error) {

if(!error.isOperational)

process.exit(1);

TL;DR

YONI GOLDBERG

Operational errors (e.g. API received an invalid input) refer to known cases where the error impact is fully understood and can be handled thoughtfully. On the other hand, programmer error (e.g. trying to read undefined variable) refers to unknown code failures that dictate to gracefully restart the application

Code example – marking an error as operational (trusted)

123

//marking an error object as operational var myError = new Error("How can I add new product when no value provided?");myError.isOperational = true;

//killing the process only if an error is not trusted (not operational)

//error handling code within middleware

process.on('uncaughtException', function(error) {

if(!error.isOperational)

process.exit(1);

Otherwise

YONI GOLDBERG18 |

You may always restart the application when an error appear, but why letting ~5000 online users down because of a minor, predicted, operational error? the opposite is also not ideal – keeping the application up when unknown issue (programmer error) occurred might lead to an unpredicted behavior. Differentiating the two allows acting tactfully and applying a balanced approach based on the given context

From the blog debugable.com, ranked 3 for the keywords “Node.JS uncaught exception”

…So, unless you really know what you are doing, you should perform a graceful restart of your service after receiving an “uncaughtException” exception event. Otherwise you risk the state of your application, or that of 3rd party libraries to become inconsistent, leading to all kinds of crazy bugs…

“Otherwise you risk the state of your application”

One paragraph explainer

YONI GOLDBERG19 |

Always distinguish between the two: operational errors refer to situations where you understand what happened and the impact of it – for example, a query to some HTTP service failed due to connection problem. On the other hand, programmer errors refer to cases where you have no idea why and sometimes where an error came from – it might be some code that tried to read undefined value or a DB connection pool that leaks memory. Operational errors are relatively easy to handle – usually logging the error is enough. Things become hairy when a programmer errors pop-up, the application might be in an inconsistent state and there’s nothing better you can do than restart gracefully + analyze quickly the reason for those failures

From the blog Joyent, ranked 1 for the keywords “Node.JS error handling”

…The best way to recover from programmer errors is to crash immediately. You should run your programs using a restarter that will automatically restart the program in the event of a crash. With a restarter in place, crashing is the fastest way to restore reliable service in the face of a transient programmer error…

“Programmer errors are bugs in the program”

Code Example – marking an error as operational

YONI GOLDBERG20 |

Code example

123

4567891011

12

1314151617

//centralized error object that derives from Node’s Error

function (name, httpCode, description, isOperational) {

Error.call(this);

Error.captureStackTrace(this);

this.name = name;

//...other properties assigned here

};

appError.prototype.__proto__ = Error.prototype;

module.exports.appError = appError;

//marking an error object as operational var myError = new Error(“invalidInput”, 400, How can I add new product when no value provided?“, true); //error handling code within middlewareprocess.on('uncaughtException', function(error) { if(!error.isOperational) process.exit(1);});

What other bloggers say

YONI GOLDBERG21 |

From Node.JS official documentationBlog Quote: “No safe way to leave without creating some undefined brittle state”

…By the very nature of how throw works in JavaScript, there is almost never any way to safely “pick up where you left off”, without leaking references, or creating some other sort of undefined brittle state. The safest way to respond to a thrown error is to shut down the process. Of course, in a normal web server, you might have many connections open, and it is not reasonable to abruptly shut those down because an error was triggered by someone else. The better approach is to send an error response to the request that triggered the error, while letting the others finish in their normal time, and stop listening for new requests in that worker.

From the blog: JS Recipes”

…There are primarily three schools of thoughts on error handling:1. Let the application crash and restart it.2. Handle all possible errors and never crash.3. Balanced approach between the two

Blog Quote: “There are three schools of thoughts on error handling”

Summarizes and quotes 4 sources

Handle errors centrally, through but not within middleware

4

12

3

4

module.exports.handler = new errorHandler();

function errorHandler(){

this.handleError = function (error) {

YONI GOLDBERG

Error handling activities such as mail notification to admin and logging should be encapsulated in a dedicated and centralized object that all end-points (e.g. Express middleware, cron jobs, lambda functions, unit-testing) call when an error comes in

Code example – handling errors within a dedicated object

1

2

3

4

5

6

TL;DR

module.exports.handler = function (){

this.handleError = function (error) {

return logger.logError(err).then(sendMailToAdminIfCritical).then(saveInOpsQueueIfCritical).then(determineIfOperationalError);

}

Otherwise

YONI GOLDBERG24 |

Handling error within modules or HTTP routes (e.g. Express) will lead to duplicate code and greater chances of errors that are handled improperly

Code example – Anti Pattern: handling errors within the middleware

1

2

3

4

5

6

7

8

//middleware handling the error directly, where will handle Cron jobs or AMQP subscriber errors?

app.use(function (err, req, res, next) {

logger.logError(err);

if(err.severity == errors.high)

mailer.sendMail(configuration.adminMail, "Critical error occured", err);

if(!err.isOperational)

next(err);

});

One paragraph explainer

YONI GOLDBERG25 |

Without one dedicated object for error handling, greater are the chances of important errors hiding under the radar due to improper handling. The error handler object is responsible for making the error visible, for example by writing to a well-formatted logger, sending events to some monitoring product or email to admin directly. A typical error flow might be: Some module throws an error -> API router catches the error -> it propagates the error to the middleware (e.g. Express, KOA) who is responsible for catching errors -> a centralized error handler is called -> the middleware is being told whether this error is untrusted error (not operational) so it can restart the app gracefully. Note that it’s a common, yet wrong, practice to handle error within Express middleware – doing so will not cover errors that are thrown in non-web interfaces

From the blog Daily JS, ranked 14 for the keywords “Node.JS error handling”

…You should set useful properties in error objects, but use such properties consistently. And, don’t cross the streams: HTTP errors have no place in your database code. Or for browser developers, Ajax errors have a place in code that talks to the server, but not code that processes Mustache templates…

“HTTP errors have no place in your database code”

Code Example – a typical Error flow

YONI GOLDBERG26 |

Code example – handling errors within a dedicated object12345

678910111213141516

17181920212223

//DAL layer, we don't handle errors hereDB.addDocument(newCustomer, (error, result) => { if (error) throw new Error("Great error explanation comes here", other useful parameters)}); //API route code, we catch both sync and async errors and forward to the middlewaretry { customerService.addNew(req.body).then(function (result) { res.status(200).json(result); }).catch((error) => { next(error) });}catch (error) { next(error);} //Error handling middleware, we delegate the handling to the centralized error handlerapp.use(function (err, req, res, next) { errorHandler.handleError(err).then((isOperationalError) => { if (!isOperationalError) next(err); });});

From the blog Joyent, ranked 1 for the keywords “Node.JS error handling”

…You may end up handling the same error at several levels of the stack. This happens when lower levels can’t do anything useful except propagate the error to their caller, which propagates the error to its caller, and so on. Often, only the top-level caller knows what the appropriate response is, whether that’s to retry the operation, report an error to the user, or something else. But that doesn’t mean you should try to report all errors to a single top-level callback, because that callback itself can’t know in what context the error occurred…

Blog Quote: “Sometimes lower levels can’t do anything useful except propagate the error to their caller”

What other bloggers say

YONI GOLDBERG27 |

From the blog JS Recipes, ranked 17 for the keywords “Node.JS error handling”

…In Hackathon Starter api.js controller alone, there are over 79 occurrences of error objects. Handling each err individually would result in tremendous amount of code duplication. The next best thing you can do is to delegate all error handling logic to an Express middleware…  

“Handling each err individually would result in tremendous duplication”

Summarizes and quotes 2 source

Document API errors using Swagger

5

12

3

4

responses:

"405":

description: Validation exception

"404":

description: Pet not found

"400":

YONI GOLDBERG

Let your API callers know which errors might come in return so they can handle these thoughtfully without crashing. This is best done with REST API documentation frameworks like Swagger

TL;DR

An example of API errors documentation using swagger

Otherwise

YONI GOLDBERG30 |

An API client might decide to crash and restart only because he received back an error he couldn’t understand. Note: the caller of your API might be you (very typical in a microservices environment)

Code example – acting thoughtfully on a given HTTP error response

1//Javascript example: treating an error thoughtfully improves the flow and UXlet newPet = {name:”Mike”, age:3};let serviceURI = `http://myDoamin.com/api/pets/`;httpRequest({method: 'POST', uri: serviceURI, resolveWithFullResponse: true, body: newPet, json: true}).then((response) => {

//http error 409 = a conflictif(response.httpCode === 409)

notificationService.showError(“The given pet name already exist, kindly choose a new one?”);});;

One paragraph explainer

YONI GOLDBERG31 |

REST APIs return results using HTTP code, it’s absolutely required for the API user to be aware not only about the API schema but also about potential errors – the caller may then catch an error and tactfully handle it. For example, your API documentation might state in advanced that HTTP status 409 is returned when the customer name already exist (assuming the API register new users) so the caller can correspondingly render the best UX for the given situation. Swagger is a standard that defines the schema of API documentation with eco-system of tools that allow creating documentation easily online, see prtscn screens below

From the blog Joyent, ranked 1 for the keywords “Node.JS logging”

We’ve talked about how to handle errors, but when you’re writing a new function, how do you deliver errors to the code that called your function? …If you don’t know what errors can happen or don’t know what they mean, then your program cannot be correct except by accident. So if you’re writing a new function, you have to tell your callers what errors can happen and what they mean…

“You have to tell your callers what errors can happen”

An Example

YONI GOLDBERG32 |

Defining a Swagger endpoint that returns status 405 upon an invalid input

Summarizes and quotes 4 sources

Shut the process gracefully when a stranger comes to town

6

12

3

4

process.on('uncaughtException', (error) =>{

//check if the error is safe (operational)

process.exit(1)

});

YONI GOLDBERG

When a non-operational error occurs (see best practice number #3) - there is uncertainty about the application healthiness. A common practice suggests restarting the process carefully using a ‘restarter’ tool like Forever, PM2 or Linux systemd

Code example: deciding whether to crash

TL;DR

1

2

3

4

5

6

7

//deciding whether to crash when an uncaught exception arrives

//Assuming developers mark known operational errors with error.isOperational=true, read best practice #3

process.on('uncaughtException', function(error) {

errorManagement.handler.handleError(error);

if(!error.isOperational)

process.exit(1)

});

Otherwise

YONI GOLDBERG35 |

Some developer errors will lead to crazy and unpredicted behavior. For example, consider an event emitter which is used globally and not firing events anymore due to some internal failure

Code example - swallowing errors is an anti-pattern

1234567

891011

//error happened? let's swallow it and prevent crashes! (don't do that)process.on('uncaughtException', function(error) { logger.log(error)});

One paragraph explainer

YONI GOLDBERG36 |

Somewhere within your code, an error handler object is responsible for deciding how to proceed when an error comes in – if the error is trusted (i.e. operational error, see further explanation within best practice #3) then writing to log file might be enough. Things get hairy if the error is not familiar – this means that some component might be in a fault state and all future requests are subject to failure. For example, a singleton, stateful token issuer service that threw an exception and lost its state – from now it might behave unexpectedly and cause all requests to fail. Under this scenario, kill the process and use a ‘Restarter tool’ (like Forever, PM2, etc) to start with a clean slate.

From the blog Joyent, ranked 1 for the keywords “Node.JS error handling”

…The best way to recover from programmer errors is to crash immediately. You should run your programs using a restarter that will automatically restart the program in the event of a crash. With a restarter in place, crashing is the fastest way to restore reliable service in the face of a transient programmer error…

“The best way is to crash”

Code Example – deciding when to crash

YONI GOLDBERG37 |

Code example – handling errors within a dedicated object1

234

567

8910

111213

1415

1617

//deciding whether to crash when an uncaught exception arrives

//Assuming developers mark known operational errors with error.isOperational=true, read best practice #3process.on('uncaughtException', function(error) { errorManagement.handler.handleError(error);

if(!errorManagement.handler.isTrustedError(error)) process.exit(1)});

//centralized error handler encapsulates error-handling related logic function errorHandler(){ this.handleError = function (error) {

return logger.logError(err).then(sendMailToAdminIfCritical).then(saveInOpsQueueIfCritical).then(determineIfOperationalError); }

this.isTrustedError = function(error) {

return error.isOperational; }

What other bloggers say

YONI GOLDBERG38 |

From the blog: JS Recipes”

…There are primarily three schools of thoughts on error handling:

1. Let the application crash and restart it.

2. Handle all possible errors and never crash.

3. Balanced approach between the two

Blog Quote: “There are three schools of thoughts on error handling”

From Node.JS official documentation

…By the very nature of how throw works in JavaScript, there is almost never any way to safely “pick up where you left off”, without leaking references, or creating some other sort of undefined brittle state. The safest way to respond to a thrown error is to shut down the process. Of course, in a normal web server, you might have many connections open, and it is not reasonable to abruptly shut those down because an error was triggered by someone else. The better approach is to send an error response to the request that triggered the error, while letting the others finish in their normal time, and stop listening for new requests in that worker.

Blog Quote: “No safe way to leave without creating some undefined brittle state”

Summarizes and quotes 2 sources

Increase error visibility using advanced logging tools

7

123456789

//your centralized logger objectvar logger = new winston.Logger({ level: 'info', transports: [ new (winston.transports.Console)(), new (winston.transports.File)({ filename: 'somefile.log' }) ] });

YONI GOLDBERG

A set of tools like mature logging libraries and log aggregators (Winston, Bunyan, ElasticSearch, AWS CloudWatch etc) will speed-up error discovery and understanding. So forget about console.log.

Code example – Winston Logger in action

TL;DR

1234567

89

//your centralized logger objectvar logger = new winston.Logger({ level: 'info', transports: [ new (winston.transports.Console)(), new (winston.transports.File)({ filename: 'somefile.log' }) ] }); //custom code somewhere using the loggerlogger.log('info', 'Test Log Message with some parameter %s', 'some parameter', { anything: 'This is metadata' });

Otherwise

YONI GOLDBERG41 |

Skimming through console.logs or manually hunting an exception within messy text files in ~10 servers might keep you busy at work until late

Code example – anti pattern, using console.log

//let's degrades performance and visibility (don't do that)

console.log("this log statement is not persistent and can't get

aggregated to a centralized log files repository");;

123

One paragraph explainer

YONI GOLDBERG42 |

We all loovve console.log but obviously a reputable and persisted Logger like Winston, Bunyan or L4JS is mandatory for serious projects. A set of practices and tools will help to reason about errors much quicker – (1) log frequently using different levels (debug, info, error) (2) when logging, provide contextual information as JSON objects, see example in next slides. (3) use a log aggregator solution like AWS CloudWatch, ELK, Splunk (each cloud provider has its own aggregator service) that provide a unified view of all logs. Otherwise you'll have to hunt production bugs by SSH into multiple servers (4) a dashboard that curates logs and provides insights like which errors happen most, which API endpoints are slower than others and much more.

From the blog Strong Loop, ranked 1 for the keywords “Node.JS logging”

Lets identify a few requirements (for a logger):1. Time stamp each log line. This one is pretty self explanatory – you should be able to tell when each log entry occured.2. Logging format should be easily digestible by humans as well as machines.3. Allows for multiple configurable destination streams. For example, you might be writing trace logs to one file but when an error is encountered, write to the same file, then into error file and send an email at the same time…

“Logger Requirements”

CloudWatch - AWS service for viewing aggregated logs

YONI GOLDBERG43 |

Watch and filter log entries that are aggregated from all servers using AWS CloudWatch service

Taking it higher – ELK logs dashboard view

YONI GOLDBERG44 |

A Kibana (part of ELK) dashboard that make sense of logs

Code Example – querying log files using a logger library

YONI GOLDBERG45 |

Code example – handling errors within a dedicated object1

234

5

67

8

var options = {

from: new Date - 24 * 60 * 60 * 1000, until: new Date, limit: 10, start: 0, order: 'desc', fields: ['message'] };

// Find items logged between today and yesterday.

winston.query(options, function (err, results) { //callback with results

});

Summarizes and quotes 1 source

Test error flows using your favorite test framework

8

123

describe("Facebook chat", () => { it("Notifies on new chat message", () => { var chatService = new chatService();

YONI GOLDBERG

Whether professional automated QA or plain manual developer testing – Ensure that your code not only satisfies positive scenario but also handle and return the right errors. Testing framework like Mocha & Chai can handle this easily (see code examples)

Code example – ensuring the right exception is thrown using Mocha & Chai

TL;DR

describe("Facebook chat", () => { it("Notifies on new chat message", () => { var chatService = new chatService(); chatService.participants = getDisconnectedParticipants(); expect(chatService.sendMessage.bind({message: "Hi"})).to.throw(ConnectionError); });});

1234567

Otherwise

YONI GOLDBERG48 |

Without testing, whether automatically or manually, you can’t rely on our code to return the right errors. Without meaningful errors – there’s no error handling

Lightweight testing option – Postman (Chrome extension) allows testing HTTP API in few minutes and even export the testing as a console script that can be included in your CI process

One paragraph explainer

YONI GOLDBERG49 |

Testing ‘happy’ paths is no better than testing failures. Good testing code coverage demands to test exceptional paths. Otherwise, there is no trust that exceptions are indeed handled correctly. Every unit testing framework, like Mocha & Chai, has a support for exception testing (code examples below). If you find it tedious to test every inner function and exception – you may settle with testing only REST API HTTP errors.

Code example – ensuring API returns the right HTTP error code

12345678910

it("Creates new Facebook group", function (done) { var invalidGroupInfo = {}; httpRequest({method: 'POST', uri: "facebook.com/api/groups", resolveWithFullResponse: true, body: invalidGroupInfo, json: true }).then((response) => { //oh no if we reached here than no exception was thrown }).catch(function (response) { expect(400).to.equal(response.statusCode); done(); }); });

Summarizes and quotes 2 sources

Discover errors and downtime using APM products

9

123

NPM install newrelic

//a single line of code is all what it takes to benefit a dashboard that analyzes your app performance

//app.js ->

Require(‘newrelic’);

YONI GOLDBERG

Monitoring and performance products (a.k.a APM) proactively gauge your codebase or API so they can auto-magically highlight errors, crashes and slow parts that you were missing

TL;DR

Read at Wikipedia here

In the fields of information technology and systems management, Application Performance Management (APM) is the monitoring and management of performance and availability of software applications. APM strives to detect and diagnose complex application performance problems to maintain an expected level of service. APM is “the translation of IT metrics into business meaning ([i.e.] value)

“Wikipedia about APM”

OtherwiseYou might spend great effort on measuring API performance and downtimes, probably you’ll never be aware which are your slowest code parts under real world scenario and how these affects the UX

Performance monitoring for example – “newrelic”, a commercial product, highlights the worst UX experience in your app by measuring from the end-user perspective

One paragraph explainer

YONI GOLDBERG53 |

Exception != Error. Traditional error handling assumes the existence of Exception but application errors might come in the form of slow code paths, API downtime, lack of computational resources and more. This is where APM products come handy as they allow with minimal setup to detect a wide variety of ‘burried’ issues proactively. Among the common features of APM products are – alerting when HTTP API returns errors, detect when API response time drops below some threshold, detection of ‘code smells’, monitor server resources, operational intelligence dashboard with IT metrics and many other useful features. Most vendors offer a free plan.

“Major products and segments”APM products constitues 3 major segments:

1. Website or API monitoring – external services that constantly monitor uptime and performance via HTTP requests. Can be setup in few minutes. Following are few selected contenders: Pingdom, Uptime Robot, and New Relic

2. Code instrumetation – products family which require to embed an agent within the application to benefit feature slow code detection, exceptions statistics, performance monitoring and many more. Following are few selected contenders: New Relic, App Dynamics

3. Operational intelligence dashboard – these line of products are focused on fasciliatitating the ops team with metrics and curated content that helps to easily stay on top of application peroformance. This is usually involves aggregating multiple sources of information (application logs, DB logs, servers log, etc) and upfront dashboard design work. Following are few selected contenders: Datadog, Splunk

Example: Up time monitoring using UpTimeRobot.Com

YONI GOLDBERG54 |

Up time monitoring products specializes in detecting service accessibility issues including high latency

Example: performance monitoring with AppDynamic

YONI GOLDBERG55 |

Performance monitoring product takes an holistic approach of gauging the system behavior from multiple angles including from the user’s device

Summarizes and quotes 2 sources

Catch unhandled promise rejections

10

1

2

3

4

5

6

DAL.getUserById(1).then((johnSnow) =>

{

//this error will just vanish!

if(johnSnow.isAlive == false)

throw new Error('ahhhh');

});

Anti pattern code example – Catching unresolved and rejected promises1

2

3

4

5

6

7

8

9

10

process.on('unhandledRejection', function (reason, p) {

//I just caught an unhandled promise rejection, since we already have fallback handler for unhandled errors (see below), let throw and let him handle that

throw reason;

});

process.on('uncaughtException', function (error) {

//I just received an error that was never handled, time to handle it and then decide whether a restart is needed

errorManagement.handler.handleError(error);

if (!errorManagement.handler.isTrustedError(error))

process.exit(1);

});

TL;DR

YONI GOLDBERG57 |

Any exception thrown within a promise will get swallowed and discarded unless a developer didn’t forget to explictly handle. Even if you’re code is subscribed to process.uncaughtException! Overcome this by registering to the event process.unhandledRejection

Otherwise*

58 |

Your errors will get swallowed and leave no trace. Nothing to worry about

Code example - Shockingly, these errors will leave no trace

1

2

3

4

5

6

DAL.getUserById(1).then((johnSnow) =>

{

//this error will just vanish

if(johnSnow.isAlive == false)

throw new Error('ahhhh');

});

*Update: As of Node 6.6, this behavior was partially improved and uncaught promises will get logged to the console. Though this increases the chances of discovering errors, still you’re error handling code won’t have the chance of treating this error like any other. Consequently, this practice is still valid and important

One paragraph explainer

YONI GOLDBERG59 |

Typically, most of modern Node.JS/Express application code runs within promises – whether within the .then handler, a function callback or in a catch block. Surprisingly, unless a developer remembered to add a .catch clause, errors thrown at these places disappear without leaving any trace(!). They will not get caught even by app.uncaughtException. The straightforward solution is to never forget adding .catch clause within each promise chain call and redirect to a centralized error handler. However building your error handling strategy only on developer’s discipline is somewhat fragile. Consequently, it’s highly recommended using a graceful fallback and subscribe to process.on(‘unhandledRejection’, callback) – this will ensure that any promise error, if not handled locally, will get its treatment.

From the blog James Nelson

Let’s test your understanding. Which of the following would you expect to print an error to the console?Promise.resolve(‘promised value’).then(function() {throw new Error(‘error’);});Promise.reject(‘error value’).catch(function() {throw new Error(‘error’);});The problem with being human is that if you can make a mistake, at some point you will. Keeping this in mind, it seems obvious that we should design things in such a way that mistakes hurt as little as possible, and that means handling errors by default, not discarding them

“If you can make a mistake, at some point you will”

Summarizes and quotes 2 sources

Fail fast, validate arguments using a dedicated library

11

1

23

456

7

8

var memberSchema = Joi.object().keys({

password: Joi.string().regex(/^[a-zA-Z0-9]{3,30}$/),

birthyear: Joi.number().integer().min(1900).max(2013)};

Joi.validate(newMember, memberSchema)

YONI GOLDBERG

This should be part of your endpoint best practices (Express, hapi, KOA) – Assert API input to avoid nasty bugs that are much harder to track later. Validation code is usually tedious unless using a very cool helper libraries like Joi

Code example – validating complex JSON input using ‘Joi’

TL;DR

//Using Joi, a very popular NPM package, to define input schema and validate itvar memberSchema = Joi.object().keys({ password: Joi.string().regex(/^[a-zA-Z0-9]{3,30}$/), birthyear: Joi.number().integer().min(1900).max(2013), email: Joi.string().email()}); function addNewMember(newMember){ //assertions comes first if(Joi.validate(newMember), memberSchema, (err, value) => throw Error("Invalid input)); //other logic here}

12345

67891011

Otherwise

YONI GOLDBERG62 |

Consider this – your function expects a numeric argument “Discount” which the caller forgets to pass, later on your code checks if Discount!=0 (amount of allowed discount is greater than zero), then it will allow the user to enjoy a discount. Can you see the nasty bug hiding between the lines?

Anti-pattern: no validation yields nasty bugs

//if the discount is positive let's then redirect the user to print his discount couponsfunction redirectToPrintDiscount(httpResponse, member, discount){ if(discount != 0) httpResponse.redirect(`/discountPrintView/${member.id}`);} redirectToPrintDiscount(httpResponse, someMember);//forgot to pass the parameter discount, why the heck was the user redirected to the discount screen?

123456789

One paragraph explainer

YONI GOLDBERG63 |

We all know how checking arguments and failing fast is important to avoid hidden bug. If not, read about explicit programming and defensive programming. In reality, we tend to avoid it due to the annoyance of coding it (e.g. think of validating hierarchical JSON object with fields like email and dates) – libraries like Joi and Validator turns this tedious task into a breeze.

From the blog: Joyent, ranked #1 in Google keywords “Node.JS error handling”

A degenerate case is where someone calls an asynchronous function but doesn’t pass a callback. You should throw these errors immediately, since the program is broken and the best chance of debugging it involves getting at least a stack trace and ideally a core file at the point of the error. To do this, we recommend validating the types of all arguments at the start of the function.

“You should throw these errors immediately”

Wikipedia: Defensive Programming

YONI GOLDBERG64 |

Read at Wikipedia

Defensive programming is an approach to improve software and source code, in terms of: General quality – reducing the number of software bugs and problems. Making the source code comprehensible – the source code should be readable and understandable so it is approved in a code audit. Making the software behave in a predictable manner despite unexpected inputs or user actions.

“Wikipedia: Defensive Programming”

This best practice is obsolete now as not domains are officially deprecated. It’s not recommended to use domain for any scenario

[Deprecated] Use Node.jS domain to isolate errors

12

Thanks For ReadingSee many others Node.JS best practices at my Twitter @nodepractices account or my Facebook page at facebook.com/nodepracticesBe my guest at: www.goldbergyoni.com

Seek Node.JS consultation or training? I’m here for you at me@goldbergyoni.com