DevOps and Node.js Chetan Desai, Intuit DevOps Architect, TurboTax @chetanddesai 11/17/2015

Intuit’s SOA

Application Services

Capability Services

Utility Services

User Experience Multi-device user experiences (cares about layout)

Application specific services (data interaction for UX)

Re-usable capabilities across applications (tax engine)

Data platform type services (login & identity)

My First Reaction •  I come from deploying in containers like JBoss & Tomcat • What do you mean the whole thing dies if there is an exception?! –  Run 30 million+ customers through it.

• Consult some experts: NodeSource • Our DevOps practices was integral to our success

Enterprise Considerations

1.   Shared build farms 2.   Build once / deploy multiple times 3.   Minimize failure points during deployment 4.   Availability

Shared Build Farms

Problem: global npm dependencies • npm WARN prefer global <pkgName>@<ver> should be installed with –g –  No root access or sudo privileges –  Other node services with different versions

CLI Tools • Solution: package.json npm scripts!

• Add global dependencies to devDependencies section

• Add your CLI calls as npm scripts • Execute Scripts –  npm run <script>

The power of the package.json

Other Benefits 1.   Local setup documentation:

- scripts for building, testing, running. 2.   Obtain consistency between developer workstations

- at least compatible 3.   Works in shared build environments!

- dependencies localized

Build Once / Deploy Multiple

• Reproducibility is a requirement • Problem: Transitive Dependencies in package.json • semver major.minor.patch –  ~1.1.1 "Approximately equivalent to version” –  ^1.1.1 "Compatible with version” –  1.1.1 "Specific version”

• Did you install version 2.0.0 or version 2.0.0?

npm shrinkwrap

• Solution: npm shrinkwrap –dev –  Full transitive dependency list and versions installed

• Somewhere in the middle –  Don’t check in shrinkwrap.json –  Generate it once at build time

• How to balance dev speed and compliance? |-----------------------------------------| Developer Speed Reproducibility / Compliance

Fast & Reliable Deployments

• Problem: –  massive node_modules folder compared to the code size –  dependency on an npm registry

• Solution: –  Our build OS (RHEL) is same as runtime OS –  Reduce size of node_modules with npm prune –production –  Zip up the remaining contents with service code •  Never run npm at deployment time

Availability

• Problem: Process Management –  Let it die and restart –  Lots of options: pm2,

forever, strongloop pm, cluster

• http://strong-pm.io/compare/

Availability

• Solution: Who can manage processes better than the OS itself? –  RHEL6 upstart & RHEL7 systemd –  Enterprises have heterogeneous set of languages •  Deploy & monitor features needs to work across stack

• Upstart Approach –  Multiple stateless processes, 1:1 with CPUs –  Load balanced & SSL termination with nginx –  If the process dies, upstart restarts it •  Configurable respawn window •  Splunk for log monitoring

Recap: Enterprise Considerations

1.   Shared build farms - local dependencies and npm scripts

2.   Build once / deploy multiple times - zip and version deployable with modules - shrinkwrap for reproducibility

3.   Minimize failure points during deployment - only run npm at build time

4.   Availability - upstart (RHEL6), systemd (RHEL7)

The solutions will evolve…

continue the conversation.

Thank you!

Chetan Desai, Intuit DevOps Architect @chetanddesai