Late to the NodeJS party

So at work we started looking at the current flavour of the month, NodeJS (or just Node from here on out). Turns out its actually quite cool but also quite frustrating.

For those not yet gulping down the Node coolaid, i will try to summarise what we’ve found so far and why we think it might actually be very useful for us.

Non blocking / Event driven

One thing you hear about Node is its “non blocking”, great! erm… what?

So, imagine you need to build a complex object from LOTS of database tables, some which due to many-many relationships you need to run several queries to get e.g.

• Query 1: Get basic listing (10ms)
• Query 3: Get tags (10ms)
• Query 2: Get child items (15ms)
• Query 4: Get location information (10ms)

In a blocking language, say PHP, assuming php is instant (and that your not using the new non blocking framework http://reactphp.org/), you would run them in order, meaning it takes a total of 45ms.

But, in Node, you can run all of them at the same time, so the total time would be 15ms!!

Of course, you CAN do this in other languages using threads (or even the same event driven / non blocking technique Node uses) but in Node its forced on you and just how you work with the language.

It takes a while for this to sink in and get fully groked but its worth the effort.

Thats not to say its always better. You can very quickly end up with “callback hell” and it can make debugging / following the flow of the execution very hard to follow.

But after a while it starts to click into place and you start looking at the “old” way you wrote code and see all the places this technique could have helped a lot.

NPM (Node Packaged Modules)

Also known as “there is a module for that”.

https://npmjs.org/

Modules are first class citizens in the node world. They are pre-written code that you can include into your system to make life easier.

The NPM website also acts as a registry, keeping track of what modules depend on what (https://npmjs.org/browse/depended). This is very useful information when selecting modules as it lets you see which modules have wide adoption (and likely better tested).

Lots of people are writing modules for Node, some good, some not so good. But its great to find that your DB flavour of choice is already supported!

Callbacks

Making everything non blocking is critical. As there is only one thread, anything that makes the system wait is really bad. NodeJS fixes this by making any blocking calls use callbacks :

Do this; once its finished run this callback

This works fine for simple examples :

db.query("select * from posts where id = 1", function(error, data) { console.log(data); });

But, once you start getting a few callbacks deep, it can get quite ugly “callback hell”.

Fortunately the 2nd most depended on module has a number of patterns to help with exactly this : https://npmjs.org/package/async

There is a great blog post / tutorial on how to use the above module here : http://www.sebastianseilund.com/nodejs-async-in-practice

Clustering

With only one thread per application, you’ve only got a single CPU core working for you, which on todays huge machines is quite a waste. Fortunately, its very easy to cluster your application.

NodeJS has a cluster module, which you can enable :

// Include the cluster module
var cluster = require('cluster');

// Code to run if we're in the master process
if (cluster.isMaster) {

    // Count the machine's CPUs
    var cpuCount = require('os').cpus().length;

    // Create a worker for each CPU
    for (var i = 0; i < cpuCount; i += 1) {
        cluster.fork();
    }

    // Listen for dying workers
    cluster.on('exit', function (worker) {

        // Replace the dead worker,
        // we're not sentimental
        console.log('Worker ' + worker.id + ' died :(');
        cluster.fork();

    });

// Code to run if we're in a worker process
} else {

    // Add your normal app code here
    ...

}

The above code was borrowed from a nice blog post about topic : http://rowanmanning.com/posts/node-cluster-and-express/ its targeted to Express, but works for any node app you might write.

Past clustering several threads on a single machine, you can use tools like HA proxy to balance between several ports / machines. The nodeUP podcast has a good conversation with Trello who go into a bit of detail of how they clustered/scaled their nodeJS app http://nodeup.com/fiftyfour

Streams

The other “killer feature” of Node is its streams. Stream instances are basically Unix pipes.

The author of JQuery will explain it much better than i can : http://ejohn.org/blog/node-js-stream-playground/

He also made a very cool playground to mess about with streams and see what they can do : http://nodestreams.com/

Frameworks

http://nodeframework.com/

People have been taking the NPM components and running with them to build entire frameworks, the site above lists a few of the more popular projects.

Performance, performance, performance

Speed is a key feature of Node and its measured and tracked by a range of tools. Google alum’s and others will tell you that the faster your site is, the more people will use it. So being fast is important.

People are writing a bunch of cool tools to test your own code, to make sure its still fast, one we’ve found called Flod which in our very limited testing seems to work quite well.

https://npmjs.org/package/flod

Its still Javascript

So you probably, sort of, know how to write it already ! After all,
JavaScript Guru Douglas Crockford famously said “JavaScript is the only language people feel like they don’t need to learn to use.”

While it is still Javascript, NodeJS comes with a nice core API (http://nodejs.org/api/documentation.html) which is then extended by the first class modules available via NPM (http://npmjs.org/).

Proven tech + large community

Everyone from Paypal to major banks, Walmart to Ebay is already using it, in production, for critical systems.

There is also a thriving community of developers around it, both in the US and rest of the world. e.g. http://www.meetup.com/Seattle-Node-js/ or http://lnug.org/ ;)

What are you waiting for?

Have a play, there are LOTS of resources out there already e.g. http://nodeschool.io/ or http://nodeup.com/ to help you on your path to node ninjadom. Its easy to get up and running on your own machine and really rather cool.