Overview
Testing
This section goes over different techniques for testing Nitric applications locally, and in the cloud.
We attempt to make the local run of nitric as similar as possible to the cloud environments, so these tests should act the same locally as they do in the cloud. However, running them on the cloud may incur costs.
Unit testing
A unit test is testing small sections of logically isolated code, usually a function, subprogram, method or property. Unit tests provide confidence in the code that has been written. This confidence gives you the ability to create new features, without fear of breaking older code.
It is important that unit tests are fast, replicable, and isolated. It shouldn't touch or talk to a database, network, or file system, and should be able to run parallel to any other test.
Writing the API
We will write a small API to do some testing on.
First, define the resources:
// resources/apis.ts
import { api } from '@nitric/sdk';
export const helloApi = api('main');
// resources/buckets.ts
import { bucket } from '@nitric/sdk';
export const imageBucket = bucket('images');
The trick for unit testing is that we want to define our functions separately from the routes, as testing anonymous functions in the route callback is difficult. Thus, we'll split them like this:
// functions/hello.ts
import { imageBucket } from '../resources/buckets';
import { helloApi } from '../resources/apis';
export const imageWriter = imageBucket.for('writing');
export const handleHello = async (ctx: any) => {
const { name } = ctx.req.params;
ctx.res.body = `Hello ${name}`;
return ctx;
};
export const handleAddImage = async (ctx: any) => {
const { name } = ctx.req.params;
await imageWriter.file(name).write(Buffer.from(name));
ctx.res.body = `Successfully added '${name}' image to bucket!`;
return ctx;
};
helloApi.get('/:name', handleHello);
helloApi.post('/:name', handleAddImage);
Writing the Tests
Asserting
Put the unit tests in a test directory, and name it unit.test.ts
. The logic being split into separate functions makes it very easy to test our API.
We are first testing that if the function is passed a context with a set name parameter, it should return the same context, but with a body added to the response. The function handleHello is asynchronous, therefore we await it, expecting it to resolve to the expected value.
import { handleHello } from '../functions/hello';
describe('Testing Hello Function', () => {
const ctx = { req: { params: { name: 'Test' } }, res: { body: '' } };
describe('Given we want a greeting', () => {
test('responds with Hello Test', async () => {
await expect(handleHello(ctx)).resolves.toEqual({
...ctx,
res: {
body: 'Hello Test',
},
});
});
});
});
Mocking
Our next function handleAddImage becomes more complex because it uses a bucket. This will need to be mocked as we want our unit tests to be fast, replicable, and isolated. By mocking a production system, we have an isolated environment that won't be affected by changes in the system. The trade-off is that the unit tests won't catch system-level failures. However, this is covered by integration tests.
We'll go over it section by section, but here is the full example of these tests:
...
describe('Given we want to add a new image to a bucket', () => {
let writerSpy;
let bucketSpy;
let handleAddImage;
beforeAll(async () => {
writerSpy = jest.spyOn(File.prototype, 'write');
bucketSpy = jest.spyOn(BucketResource.prototype, 'for');
// Doing dynamic imports to mock the BucketResource.for
const helloModule = await import('../functions/hello');
handleAddImage = helloModule.handleAddImage;
})
afterAll(() => {
jest.resetAllMocks();
});
beforeEach(() => {
jest.clearAllMocks();
})
test('adds a new image to the bucket', async () => {
writerSpy.mockResolvedValue();
await expect(handleAddImage(ctx))
.resolves.toEqual({
...ctx,
res: {
body: "Successfully added 'Test' to user collection"
}
})
expect(writerSpy).toBeCalledTimes(1)
});
test('throws an error', async () => {
writerSpy.mockRejectedValue(new Error('Mock error'))
await expect(handleAddImage(ctx)).rejects.toEqual(Error('Mock error'))
expect(writerSpy).toBeCalledTimes(1)
});
});
Let's start by looking at the mocks. These are done in the beforeAll function, which means it happens before all the tests are run.
The functions to create a bucket 'for' and the call to write to a bucket 'write' will make gRPC calls on the backend. This means that they need to be mocked, as there is nothing for it to call to, and thus will result in an UNAVAILABLE error. We mock these using jest's spyOn functions on the object prototype.
This works fine for mocking the write
method as it's within the handleAddImage
function, but the for
method is called when the module is imported. Therefore, we add a dynamic import after for
is mocked.
import { File, BucketResouce } from '@nitric/sdk';
...
let writerSpy;
let bucketSpy;
let handleAddImage;
beforeAll(async () => {
writerSpy = jest.spyOn(File.prototype, 'write');
bucketSpy = jest.spyOn(BucketResource.prototype, 'for');
// Doing dynamic imports to include the BucketResource.for method mock
const helloModule = await import('../functions/hello');
handleAddImage = helloModule.handleAddImage;
})
We also add afterAll
and beforeEach
functions to reset the mocks.
afterAll(() => {
jest.resetAllMocks();
});
beforeEach(() => {
jest.clearAllMocks();
});
Now that we have the mocks, we can use them for the tests. The first test is testing that the function will succeed. On a success, the promise will both resolve and have a success message in the body of the context's response.
Update the mock so that it resolves, and check that the write
function was only called once. The mock can change in the test, as it will be reset before the next test is run.
test('adds a new image to the bucket', async () => {
writerSpy.mockResolvedValue();
await expect(handleAddImage(ctx)).resolves.toEqual({
...ctx,
res: {
body: "Successfully added 'Test' to user collection",
},
});
expect(writerSpy).toBeCalledTimes(1);
});
The next test will instead test a write failure. The failure may be caused by any number of reasons, and therefore we should make sure our function is fit to handle them when they occur.
Change the mock this time so that it rejects with a new error. Still check that the write was called once.
test('throws an error', async () => {
writerSpy.mockRejectedValue(new Error('Mock error'));
await expect(handleAddImage(ctx)).rejects.toEqual(Error('Mock error'));
expect(writerSpy).toBeCalledTimes(1);
});
Running the tests
The tests can now be run. Add to your package.json
a test script.
"scripts": {
"test": "jest"
}
You can then run the tests locally with:
npm run test
The output should be something like this:
Testing Hello Function
Given we want a greeting
✓ responds with Hello Test (4 ms)
Given we want to add a new image to a bucket
✓ adds a new image to the bucket (5 ms)
✓ throws an error (3 ms)
Test Suites: 1 passed, 1 total
Tests: 3 passed, 3 total
Snapshots: 0 total
Time: 4.292 s, estimated 7 s
Ran all test suites.
✨ Done in 6.18s.
Integration Testing
Integration tests are important for your systems as they catch system-level issues that unit tests miss, and are faster than end-to-end tests.
When writing these tests we want to make sure that these tests are reliable, able to be reproduced and can isolate our system failures. This is why we want to use a testing framework like jest, and use an HTTP testing library like supertest.
npm install --save-dev jest supertest @types/jest @types/supertest
Writing the API
We will write a small API to do some testing on.
First, define the resources:
// resources/apis.ts
import { api } from '@nitric/sdk';
export const helloApi = api('main');
// resources/buckets.ts
import { bucket } from '@nitric/sdk';
export const imageBucket = bucket('images');
Then define our routes and our functions:
// functions/hello.ts
import { imageBucket } from '../resources/buckets';
import { helloApi } from '../resources/apis';
export const imageWriter = imageBucket.for('writing');
export const handleHello = async (ctx: any) => {
const { name } = ctx.req.params;
ctx.res.body = `Hello ${name}`;
return ctx;
};
export const handleAddImage = async (ctx: any) => {
const { name } = ctx.req.params;
await imageWriter.file(name).write(Buffer.from(name));
ctx.res.body = `Successfully added '${name}' image to bucket!`;
return ctx;
};
helloApi.get('/:name', handleHello);
helloApi.post('/:name', handleAddImage);
Writing the Tests
We will put the integration test in a test directory, and name it integration.test.ts
. For the test, we want to create an agent using supertest. We'll point the agent at the URL of our API.
// tests/integration.test.ts
import supertest from 'supertest';
describe('Testing Hello Api', () => {
const api = supertest('http://localhost:4001');
});
We can then add some tests to hit this API. We'll start with testing the GET /:name
route.
This request has the name parameter set to 'test'. This means we expect the response to be 'Hello test' and the status code to be 200. We get the done function as a parameter from the test callback, and call that when the test is resolved or encounters errors. This is to stop timeouts on the test, as we are testing an async operation.
// tests/integration.test.ts
import supertest from 'supertest';
import assert from 'assert';
describe('Testing Hello Api', () => {
const api = supertest('http://localhost:4001');
describe('Given a request to GET /:name', () => {
test('responds with Hello test', (done) => {
api
.get('/test')
.expect(200)
.then((res) => {
assert.equal(res.text, 'Hello test');
done();
})
.catch((err) => done(err));
});
});
});
We can add another test case for hitting the POST /:name
route. This will need to test whether the response from the API resolves correctly, as well as whether the image bucket was updated. It makes a post request to our route and then checks the response just like the last one. Additionally, this test reads from the bucket to verify that the correct content was written.
Make sure you remove the call to done from the assertion in the api route. Otherwise, the bucket test will not be run.
// tests/integration.test.ts
import supertest from 'supertest';
import assert from 'assert';
import { imageBucket } from '../resources/buckets';
describe('Testing Hello Api', () => {
const api = supertest('http://localhost:4001')
...
describe('Given a request to POST /:name', () => {
test('adds a new image to the bucket', (done) => {
api
.post('/test')
.expect(200)
.then(res => {
assert.equal(res.text, "Successfully added 'test' image to bucket!")
})
.catch(err => done(err))
imageBucket
.for('reading')
.file('name')
.read()
.then(val => {
assert.equal(val.toString(), 'test')
done();
})
.catch((err) => done(err));
})
});
});
Running the tests
There are two options for running our tests:
- We run them locally
- We run them against a deployed API
Local Tests
For a local run, we need to first run the local API, then run the tests. Add to your package.json
a test script.
"scripts": {
"test": "jest"
}
We can then start the API with nitric start
and then run your function code.
And then in a separate terminal, run the tests:
npm run test
This will produce the output:
Testing Hello Api
Given a request to GET /:name
✓ responds with Hello test (4 ms)
Given a request to POST /:name
✓ adds a new image to the bucket (5 ms)
Test Suites: 1 passed, 1 total
Tests: 2 passed, 2 total
Snapshots: 0 total
Time: 3.292 s, estimated 5 s
Ran all test suites.
✨ Done in 5.18s.
Deployed Tests
We attempt to make the local run of nitric as similar as possible to the cloud environments, so these tests should act the same locally as they do on the cloud. However, running them on the cloud may incur costs.
When you have an API deployed to the cloud, most cloud providers have a feature in the console to do endpoint testing. However, the tests are often just checking if the API resolves to a 200 status code. The process we are following here will lead to much more robust testing and a lot more confidence in your cloud application.
Now for testing. The obvious first step before running our tests is to deploy the resources.
nitric up
Then, to test against the deployed API, we just want to swap out our supertest agent host from the localhost endpoint to our deployed endpoint. This new endpoint will look something like this:
// tests/integration.test.ts
const api = request('https://testerapi.us-east-1.amazonaws.com');
To invoke our tests we can do npm run test
locally. This will run the tests against the deployed instance and return your test results.