UnitTests for almost nothing

If you are a developer who wants maintainable, easy to change code you definitely write unit tests to ensure the functionality.

All programming languages I develop in (Java, Python, Ruby and Elixir) have unit testing libraries. But are your tests really worth being called unit tests?

In this article I try to cover a case where you test almost nothing but get a big percentage of code coverage.

Testing for coverage

If you work for a serious company (or as a freelancer and you are serious) you use some tools for maintaining a good and stable code base: static code analysis and tests with code coverage. Hopefully you have a system behind all this to automatically calculate these metrics with every build (Hudson / Jenkins provide dashboards for both and Sonar is much better for this case).

And you have a goal defined: code coverage above 75%. Or more better is branch coverage over 75%. And if your tool shows values greater or equal to this you are good. You can clap your shoulders and go for an AWB with co-workers to celebrate your success and shining.

But there is a dark side of testing for coverage…

What is test coverage?

This is an actual question we should answer prior jumping toward the main message of this article.

Test coverage measures interactions with a given object. Very simplified this means which lines of code were called when unit test were executed.

If I lookup in Wikipedia I get the following description:

In computer science, code coverage is a measure used to describe the degree to which the source code of a program is tested by a particular test suite. A program with high code coverage has been more thoroughly tested and has a lower chance of containing software bugs than a program with low code coverage.

Well, this is not really true. You can achieve good code coverage even if you do not test your code thoroughly. And if you do this you are doing UnitTests for almost nothing.

For almost nothing?

It sounds harsh but this is the truth. Your unit test is not worth the beer you drink when you get over 90% coverage with this approach.

This is because you can fire up method calls and do not track the results if the calls worked as they have to.

Let’s imagine a DTO. This should be a very primitive DTO with some primitive fields, getters and setters, constructors and a toString method.

The code should look like this:

package unittest;

import java.text.MessageFormat;

/**
 * Simple example class to demonstrate test coverage
 *
 * @author GHajba
 */
public class ExampleDTO {

  private int positionX;
  private int positionY;
  private String someText;
  private long uniqueIdentifier;

  public ExampleDTO() {
     this(0, 0, "Wellhello", 123456);
  }

  public ExampleDTO(int positionX, int positionY, String someText, long uniqueIdentifier) {
     setPositionX(positionX);
     setPositionY(positionY);
     setSomeText(someText);
     setUniqueIdentifier(uniqueIdentifier);
  }

  public int getPositionX() {
     return positionX;
  }

  public void setPositionX(int positionX) {
     this.positionX = positionX;
  }

  public int getPositionY() {
     return positionY;
  }

  public void setPositionY(int positionY) {
     this.positionY = positionY;
  }

  public String getSomeText() {
     return someText;
  }

  public void setSomeText(String someText) {
     this.someText = someText;
  }

  public long getUniqueIdentifier() {
     return uniqueIdentifier;
  }

  public void setUniqueIdentifier(long uniqueIdentifier) {
     this.uniqueIdentifier = uniqueIdentifier;
  }

  @Override
  public String toString() {
    return MessageFormat.format("{3} is {2} at [{0},{1}]",
        getPositionX(), getPositionY(), getSomeText(), getUniqueIdentifier());
  }
}

Let’s create a simple JUnit test case for this class:

package unittest;

import org.junit.Test;

/**
 * Simple unit test file for almost nothing
 *
 * @author GHajba
 */
public class ExampleDTOTest {

  @Test
  public void testToString() {
    new ExampleDTO().toString();
  }
}

As you can see, this code calls only one constructor and the toString method. If I run the test I get a passed result and if I run it with JaCoCo I get the following result:

Test coverage of ExampleDTO

Well, nice thing couldn’t do it better myself at work.

However this code does nothing to ensure the functionality of the class being tested.

Almost for nothing

Now it is time to see why nothing.

Let’s make a minor change in the code above and see what happens. Nothing. The code coverage stays 100% but you have a not working code — even with a change like this:

public void setPositionY(int positionY) {
     this.positionX = positionY;
  }

A simple typo — but without any cause.

Again, this can lead to hours of debugging even if you brag that your test has 100% coverage so this is not your fault.

Conclusion

You can achieve 100 test coverage (in some short-circuit conditional branches you can end up missing one or two because of the condition but give it a try) but it depends on your conscience that you do some valuable testing which throws an error if the implementation changes.

If you do code reviews in your team look for tests which do not have any assertions and verifications in them. They tend to be error prone.

Naturally tests with assertions can bear errors but not as much as tests not verifying anything.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s