Problem with random test data

java

Some time ago I noticed new library in our code base - Random Beans which as the name suggest is tool developed to easily create random data and random objects for testing purposes. Unfortunately we used it in the wrong way. Here’s how we backed up from the the random test data to regain control over testing.

tl;dr

Don’t use random data to populate objects with logic. Thanks captain obvious ;). Random tests input is not good idea because:

  • if with particular test input you expected specification to fail then its corner case and should be handled in dedicated and descriptive test case
  • if input doesn’t really matter why not provide it in plain text for the sake of readability. If you really need random data why not roll the dice ;)
  • if tests will fail for some reason would you be able to reproduce input which caused failure or will it be expected ‘X’ but got ‘Y’ and that’s all?
  • are you sure that random input doesn’t make your tests harder to understand and easier to break?

The story

To quickly bootstrap project we used random data generator to produce domain objects - I know it sounds bad. At first it looked promising and saved few key strokes, but it didn’t took long to fail. The problem reveled itself when I introduced to initially stupid simple domain some business logic. We had few tests already in place but they were not depending on internal object state, but rather focusing on interactions. I created small subcontext for new feature in new package and once I started to integrate my code into existing domain testes started to fail randomly. I’m not sure what I expected. I asked for some randomness didn’t I?

The solution

At first I wanted to exclude some fields from generation process and set values manually and then I realised that’s no good because there are a lot of tests and with each one of them I will be forced to break encapsulation (which is dangerously easy in groovy…). Then I decided it is time to fallback to old good object factories. With object factories you are sure that initialized instances are valid business objects and with a bit of effort you’ll be able to use the same code for tests objects creation, which is big plus.

Consider flowing class (in our production application we had more fields to populate and state which revealed problem):

class Article {
  private final Clock clock;

  @Getter
  private LocalDateTime lastUpdateTime;
  private State articleState;

  private String content;

  public Article(Clock clock) {
    this.clock = clock;

    this.articleState = State.IN_PROGRESS;
    this.lastUpdateTime = getCurrentDateTime();
  }

  public void updateContent(String newContent) {
    Preconditions.checkState(articleState.canSave());
    this.content = newContent;
    this.lastUpdateTime = getCurrentDateTime();
  }

  public void sendForApproval() {
    Preconditions.checkState(articleState.canSendForApproval());
    this.lastUpdateTime = getCurrentDateTime();
    this.articleState = State.WAITING_FOR_APPROVAL;
  }

  private LocalDateTime getCurrentDateTime() {
    return LocalDateTime.now(clock);
  }

  private enum State {
    APPROVED, REJECTED, IN_PROGRESS, WAITING_FOR_APPROVAL;

    boolean canSave() {
      return this == IN_PROGRESS || this == REJECTED;
    }

    boolean canSendForApproval() {
      return this == State.IN_PROGRESS;
    }
  }
}

In perfect world you should be able to just do: new Article() and be done with it, but sometimes it’s not that simple and object creation might depend on external factors which are inconvenient or impossible to provide in constructor, especially when you create article instances from many places, or you want to mock some stuff for testing purposes. That’s when object factories come into play.

You can easily create article instances with ArticlesFactory:

@RequiredArgsConstructor
class ArticleFactory {
  private final Clock systemClock;

  public ArticleFactory() {
    this(Clock.systemDefaultZone());
  }

  public Article newArticle() {
    return new Article(systemClock);
  }
}

Now when you have production objects factory you can reuse it’s logic for test data creation:

public class TestArticleFactory {
  private static Clock fixedTime = new MutableClock();

  public static Article newArticle(Clock clock) {
    return new ArticleFactory(clock).newArticle();
  }

  public static Article newApprovedArticle() {
    return newApprovedArticle(fixedTime);
  }

  public static Article newApprovedArticle(Clock clock) {
    final Article article = newArticle(clock);
    article.sendForApproval();
    return article;
  }
}

And thats all. Note that from now on you can easily add additional logic to you test data factories to porpelly initialize objects and prepare them for testing phase. You’ll need produce some extra code (comparing to raw groovy solution) but you are sure that objects created for your tests are the same objects you’ll use in production application and in the longer run when domain changes all you’ll need to do is fix factory to produce what is expected from it.

samples

13 Jun 2017 #java #tdd