How to unit test NodeJS HTTP requests?

This Node.js tutorial will walk you through how to set up your testing tool, how to unit test your HTTP GET and POST requests as well as how to test a failure scenario.

I have a Node.js app where I wanted to unit test some HTTP requests. Node usually makes things simple, so I expected this to be like that too… but I soon discovered that this was not the case at all.

When unit testing, you don’t want HTTP requests to go out and affect the result. Instead you create a fake request – a test double – to replace the real one, but because node’s streams are hard to fake, this is where I hit the wall.

It took some work, but found a way to make it simple! Let me show you how with some examples based on real live production code.

• Sending a GET request and testing response handling
• Sending a POST request and testing the parameter behavior
• Testing that failures are handled correctly

Example Scenario

I based this and the tests on real world code and simplified them so they’re easy to follow. But the core techniques are the same as I use in real production code.

I’m going to call this file api.js

var http = require('http');
 
module.exports = {
  get: function(callback) {
    var req = http.request({
      hostname: 'jsonplaceholder.typicode.com',
      path: '/posts/1'
    }, function(response) {
      var data = '';
      response.on('data', function(chunk) {
        data += chunk;
      });
 
      response.on('end', function() {
        callback(null, JSON.parse(data));
      });
    });
 
    req.end();
  },
 
  post: function(data, callback) {
    var req = http.request({
      hostname: 'jsonplaceholder.typicode.com',
      path: '/posts',
      method: 'POST'
    }, function(response) {
      var data = '';
      response.on('data', function(chunk) {
        data += chunk;
      });
 
      response.on('end', function() {
        callback(null, JSON.parse(data));
      });
    });
 
    req.write(JSON.stringify(data));
 
    req.end();
  }
};

This should look familiar if you’ve used the http module. One function for fetching data, another for sending data. The code uses the JSONPlaceholder API, a simple REST API useful for this kind of prototyping and testing.

Necessary Libraries

To make testing easy, we’ll use some libraries to help. We’ll use Mocha as the testing framework and Sinon to create the test doubles. Streams are a hassle, but we can use mock-req and mock-res to fix that.

npm install mocha sinon mock-req mock-res

Setting Up the Test Suite

First, let’s create the test suite. I’m going to put this file into tests/apiSpec.js.

var assert = require('assert');
var sinon = require('sinon');
var MockReq = require('mock-req');
var MockRes = require('mock-res');
var http = require('http');
 
var api = require('../api.js');
 
describe('api', function() {
  beforeEach(function() {
    this.request = sinon.stub(http, 'request');
  });
 
  afterEach(function() {
    http.request.restore();
  });
 
 
  //We will place our tests cases here
 
});

The key things here are beforeEach and afterEach. beforeEach creates a stub to replace http.request, and afterEach restores the original functionality.

A stub is a test double that lets us define behavior and track usage – set return values, parameters, check the call count, etc.

But how does this help? We created the stub here, not in the api module! NodeJS caches required modules, so the changes here are reflected in other modules. In other words, api.js is forced to use our stub.

We save the stub into this.request, so we can reference it later.

Testing the GET Request

Let’s start by adding a test that validates the get request handling. When the code runs, it calls http.request. Since we stubbed it, the idea is we’ll use the stub control what happens when http.request is used, so that we can recreate all the scenarios without a real HTTP request ever happening.

it('should convert get result to object', function(done) {
  var expected = { hello: 'world' };
  var response = new MockRes();
  response.write(JSON.stringify(expected));
  response.end();
 
  var request = new MockReq();
 
  this.request.callsArgWith(1, response)
              .returns(request);
 
  api.get(function(err, result) {
    assert.deepEqual(result, expected);
    done();
  });
});

First, we define the expected data that we will use in our test and create a mock response object. To copy how http.request works, we write a JSON version of our expected data into the response and end it.

We create a mock request, but we only need it as a return value.

A successful http.request will pass the response to the callback. Looking at the earlier example, the callback is the second parameter, so we tell the stub to call it with the response as a parameter. The numbering starts from 0, so using 1 refers to the second parameter. A call to http.request always returns a request, so we tell it to return our mock request.

With the setup out of the way, we call api.get with a callback which verifies the behavior. The assert module is built-in to node, but if you prefer you can use your favorite assertion library, such as chai, instead.

We can run the test using mocha. You should find the test passes with a green light.

Testing the POST Request

For the POST scenario, we want to ensure the parameters are passed correctly to the http request.

it('should send post params in request body', function() {
  var params = { foo: 'bar' };
  var expected = JSON.stringify(params);
 
  var request = new MockReq({ method: 'POST' });
  var write = sinon.spy(request, 'write');
 
  this.request.returns(request);
 
  api.post(params, function() { });
 
  assert(write.withArgs(expected).calledOnce);
});

When calling api.post, it should send parameters in the request body. That’s why in this test, we only need to verify request.write is called with the proper value.

Like previously, we define the expected data first. Doing this helps make the test easier to maintain, as we’re not using magic values all over the place.

We then define a mock request, making sure it uses the POST method. To make sure the expected data is written, we need a way to check the function was called. This calls for another test double, in this case a spy. Just like real secret agents, a spy will give us information about its target – in this case, the write function.

The http.request function only needs to return the mocked request in this test.

We call api.post, passing in our parameters and an empty callback. Why an empty callback? Again, we only test what we need, and here we don’t need to test the callback.

Finally, we use assert to check the values from the spy: We check that the spy was called once with the correct parameters.

We can run the tests again using mocha, and they’ll pass.

Note: in a real app, you might want to have a test for the callback, but it should go into its own test case. A single test case should ideally have one assertion only. The test for a callback would resemble the test with a get request, so I won’t go into detail on it here.

Testing a Failure Scenario

And last but not least, we should include a test for a failure.

Every other http testing library I tried stumbled here, but with mock-req it’s straightforward.

it('should pass request error to callback', function(done) {
  var expected = 'some error';
  var request = new MockReq();
 
  this.request.returns(request);
 
  api.get(function(err) {
    assert.equal(err, expected);
    done();
  });
 
  request.emit('error', expected);
});

This should be starting to look familiar. We define the expected data, create a mock request, tell our stub to return it and call the api. The callback passed to the api checks the error parameter is correct.

The new thing here is the last line. When an http request fails due to network errors, http parsing errors or such, it emits an error event. To simulate an error, we emit one from the test.

Run tests with mocha, and they’ll… fail? Huh?

This is why unit tests are important, and especially testing the failure cases! It turns out a bug snuck into the code.

No problem, let’s add handling for the error event.

get: function(callback) {
  var req = http.request({
    hostname: 'jsonplaceholder.typicode.com',
    path: '/posts/1'
  }, function(response) {
    var data = '';
    response.on('data', function(chunk) {
      data += chunk;
    });
 
    response.on('end', function() {
      callback(null, JSON.parse(data));
    });
  });
 
  req.on('error', function(err) {
    callback(err);
  });
 
  req.end();
},

The only thing added here is the three lines to handle the error event.
Run tests with mocha and now they will pass. Done!

Conclusion

Writing tests like these takes a bit of practice. You need to know how the stubbed and mocked functions work, but with experience it will become easy. The great thing is that you can apply these methods to test other things too – MySQL queries, Redis lookups, Ajax calls, TCP sockets… the list goes on and on.

I’ve made the example module and tests available for download from here for your convenience: api.js, apiSpec.js.