Unit Tests vs Integration Tests
I used to favor automated integration testing. Now, I find myself walking away from it. I no longer find it worth the cost of setting up and maintaining such tests. I now rely mostly on unit testing.
First, I need to define what I meant by those terms. In context of this post, unit tests
In Wikipedia
| unit test | integration test |
|---|---|
| automated, not always | not specified |
| ranging from entire ‘module’ to an individual function | modules tested as a group |
| depends on execution conditions and testing procedures | depends on unit test/implies modules are already unit tested |
From Martin Fowler
| unit test | integration test |
|---|---|
| very ill defined and no absolute answer | term is blurred and needs to be used with care |
| ranging from ‘whatever the team defines’, e.g., a set of classes to a single function | ranging from broad to narrow in scope |
| either socialble or solitary | uses test double to avoid testing against real instance of a dependency service |
| takes between ten few minutes to few seconds to run the entire suite | |
| less coverage than integration tests | may depend on contract tests, end-to-end tests, and QA in production |
One person’s definition of unit test may have a lot of overlap with another person’s definition of integration test. If a test runs quickly and uses test doubles to avoid testing against a real external service, is it still an integration test?
We frequently use “integration tests” to test things like:
- Did we write the SQL query correctly? (This can be hard for complex queries.)
- Example: Did we use the right type of joins? Did we group on the right column?
- Did we pass the right parameters to the query?
- Similar. Did we use the ORM’s features correctly?
- Example: Did we use Hibernate’s first level cache correctly?
- Example: Was the date format match what the service expects? For microservices, was a parameter misunderstood? For
JSON input, does the service expect a missing field and
nullto mean the same thing or different things? - Does the service itself behalf as expected?
- Example: We expect a column’s uniqueness to be enforced by the DB. Was the table setup correctly? We expect uniqueness data in our syste, is this enforced somehow?
Is it worth maintaining integration tests and suffering flakiness in order to verify these things?