Putting the Unit in Unit Tests

Some excellent comments to my last post. Please allow me to respond:

Should you test your persistence layer? Yes.
Should you test your system from end to end? Yes.
Should you test complex edge cases? Yes.

If you assumed the answer to any (sensible) question that starts with ‘Should you test…’ is ‘Yes’, you wouldn’t be too far from the truth.

However….

Should you test any of those things in your Unit tests? No. Unit tests (to me) are the things that the developers write and run many times a day, and should always pass. Testing against DB’s is hideously slow. Orders of magnitude too slow for unit tests. Tests against DB’s can too easily fail for the wrong reasons. Tests that require masses of setup code and mock objects just to run are too brittle. Lots of mock objects can hurt you when refactoring, as they tend to break when you pull their superclasses apart.

Also note that I didn’t say ‘Don’t use Mocks’. Sometimes they are required. MockPersistenceLayer being a good example. Although if you have interface/impl separation (which you should for your persistence layer), then ‘InMemoryPersistenceLayer’ or ‘HashMapBackedPersistenceLayer’ would be equally good names, and less open to misinterpretation than MockPersistenceLayer.

The other kind of Mocks, the ones that verify their methods were called in a certain sequence with certain parameters can be useful, but, as I said, can be brittle and prone to breakage during refactoring, so should be used with caution. They also make it less clear what’s being tested. Chances are the system would benefit from finer grained objects that could be tested individually, which should reduce the need for a full Mock.

Its perfectly ok to have a separate set of JUnit tests that aren’t ‘Unit’ tests, that (for example) test the persistence of each of your objects exactly once. Run them as often as you want, add them to your commit script if you feel the need, just don’t bolt them into the main body of unit tests.

Advertisements

One thought on “Putting the Unit in Unit Tests

  1. That is a good distinction. I still stand by my comment than LOC metrics are not so good, but I think it’s a good point that any test using JUnit isn’t necessarily a unit test.

    However, I think that most interesting tests would be of these non-unit type tests (what to call them? String tests? Integration tests? dunno. They seem like UNIT tests to me, but I also see a difference).

    I mean, your business logic is gonna be mostly data interpretation, validation, storage and retrieval. For a complex system, it’s going to be pretty difficult to test those without a lot of set up. The last project I did had JUnit tests for persistence, business logic and low-level stuff (e.g. testing equals() methods of various classes). I’d say that the business logic and database stuff accounted for 90% of the test cases, and each test failed more than once (i.e. it was useful in finding bugs).

    The other tests, while useful, really didn’t test anything interesting.

    I also think that any serious test is going to break when refactoring. That is the hidden penalty of automated testing, you have do a lot of work for various changes. It is good though.

    So, my roundabout way of getting to the point is that most valueable and interesting tests are necessarily going to require a lot of setup code and be brittle during refactoring/system change.

Comments are closed.