Automated Developer Tests

Kinds of Automated Developer Tests

• Unit tests: For every function, small class, or object we export from a code file, we need to write unit tests to justify each line and branch in the implementation. These tests should be “disconnected,” meaning they don’t require any connections or OS handles, including file handles. The goal is to verify that the code unit performs as expected and nothing more. For example, if the result is unsorted, don’t assert that it is sorted. If the unit handles files, we only need to test that it handles a file correctly, not multiple file types.
• Contract tests: Some of our code relies on third-party tools and external resources, such as databases, which can be challenging to mock accurately. To ensure our code interacts with these resources correctly and they behave as expected, we need a few tests. These tests will be run less frequently, take longer, and require the external resources to be available.
• Service tests: Unit tests ensure that each small part of the code works as intended. Contract tests verify that interactions between code units or between code units and external resources function correctly. Service tests, which are fewer in number and take longer to run, check that basic microservice features are working. These tests need the external services and the microservice itself to be running. Similar to smoke tests, they don’t cover every possible scenario but test enough to ensure the service will be healthy if deployed.

We deliberately chose unique names for the test types to ensure they are distinct from those typically written and maintained by our QE team members.

Guidelines for Adding Tests

Our QE experts excel at identifying the scenarios that need testing and can easily interact with running microservices and supporting resources. Their automation tools and skills are capable of handling many of our testing challenges. When considering adding a contract or service test, we should only proceed if there’s a compelling reason why the functionality can’t be tested by our QE team and their automated tests. Valid reasons might include ensuring application security, maintaining application state integrity, and clarifying the API.

If we determine there’s a good reason, we should add the test to the appropriate developer test suite. We should aim for fewer than six service tests and no more than fifty contract tests, though these numbers are approximations. Having six service tests or ten contract tests is acceptable. There should be no hard limit on the number of unit tests; we should include as many as needed to create the simplest, most maintainable code units.