Introduction and Factorial Example

Testing Skeleton

In this tutorial we’ll show you the basic ideas behind unit-testing by testing a trivial program that most programmers have probably written at least once. Along the way we’ll see how unit-testing makes the function more robust by helping us to think about what kinds of inputs our function can receive and what the expected outputs should be.

To start let’s say we’ve been tasked with writing a factorial function with the following signature:

int factorial(int n);

This function should compute the factorial of the number n, where the factorial is defined recursively as:

0! = 1
n! = n * (n – 1)!

Hey, these look a lot like test cases! We’ll see how to use them as such in a moment, but for now let’s just get together a some skeleton code:

#include "lemon.h"

int factorial(int n) {
  // compute the factorial
}

int main(int argc, char* argv[]) {
  lemon::test<> lemon(0);

  // Tests go here....

  lemon.end();

  return 0;
}

Alright so now we have the basic setup for a Lemon unit test. The value passed into the lemon constructor tells it how many unit tests we’re expecting to run, and has to be updated manually each time a test is added. The factorial function is defined in the unit test file for demonstration purposes, but it could just as easily be included from elsewhere.

Recursive Factorial

Now that we have the skeleton, let’s write our first test and bump the number of tests in the Lemon constructor up to 1:

int main(int argc, char* argv[]) {
  lemon::test<> lemon(1);

  // Test 1: The factorial of 0 is 1
  lemon.is(factorial(0), 1, "0! = 1");

  lemon.end();

  return 0;
}

As you can see we use the assertion Lemon::is above which uses operator == to check the first two parameters for equality. This also showcases an important part of lemon which is that each assertion is given a descriptive name. The title of this assertion is “0! = 1”. If we try to compile the program right now we’ll get an error saying that factorial returns no value, so let’s fix this really quick. Typically you want to write a failing test first and then make it pass, so let’s writing a failing factorial function:

int factorial(int n) {
  return 0;
}

Now let’s try compiling and running the test, this is what we get:

$ ./factorial_test
1..1
not ok 1 - 0! = 1
#   Failed test '0! = 1'
#         got: '0'
#    expected: '1'
# Looks like you failed 1 of 1 tests.

And there you have we failed one test because the expected value was 1 but a 0 was returned. Now let’s fix our factorial function and see what happens. The goal is to add just enough functionality to get the test to pass.

int factorial(int n) {
  return 1;
}

Now let’s run our unit-test again:

$ ./factorial_test
1..1
ok 1 - 0! = 1
# Looks like you passed all 1 tests

Awesome now we’ve passed our first test, so let’s go add another. This time we’ll check that the factorial function really equals the factorial of some value:

// Test 2: 3! = 3 * 2 * 1
lemon.is(factorial(3), 3 * 2 * 1, "3! = 3 * 2 * 1");

Let’s run this with our factorial function and see what happens:

$ ./factorial_test
1..2
ok 1 - 0! = 1
not ok 2 - 3! = 3 * 2 * 1
#   Failed test '3! = 3 * 2 * 1'
#         got: '1'
#    expected: '6'
# Looks like you failed 1 of 2 tests.

Wow, that output is all kinds of wrong. Let’s add the recursive part to the factorial function now:

int factorial(int n) {
  if (n == 0) return 1;
  return n * factorial(n - 1);
}

Now let’s run the test again:

$ ./factorial_test
1..2
ok 1 - 0! = 1
ok 2 - 3! = 3 * 2 * 1
# Looks like you passed all 2 tests

Sweet! Now this function seems to work, but let’s consider what other kinds of parameters it can receive. For example, since the parameter is of type int it can receive both positive and negative integer values. So what happens if we pass it a negative integer? Well let’s see:

factorial(-5) = -5 * factorial(-6) = -5 * -6 * factorial(-7) = ...

Uh oh, that looks like an infinite recursion to me, perhaps we should have negative factorials return something safe. For the sake of this example, let’s have them return 1. So we add a test for this:

// Test 3: (-5)! = 1
lemon.is(factorial(-5), 1, "(-5)! = 1");

And we’ll run it:

$ ./factorial_test
1..3
ok 1 - 0! = 1
ok 2 - 3! = 3 * 2 * 1
Segmentation fault

Whoa! I would consider that a failure, and it’s good that we caught this now instead of some point down the road when the factorial function is buried in other code. Now that we’ve identified the problem let’s patch it up:

int factorial(int n) {
  if (n <= 0) return 1;
  return n * factorial(n - 1);
}

Now let’s re-run our unit tests:

$ ./factorial_test
1..3
ok 1 - 0! = 1
ok 2 - 3! = 3 * 2 * 1
ok 3 - (-5)! = 1
# Looks like you passed all 3 tests

Finally, let’s add one more just to make sure that the factorial function is actually working:

// Test 4: 5! = 120
lemon.is(factorial(5), 120, "5! = 120");

And let’s run our unit tests:

$ ./factorial_test
1..4
ok 1 - 0! = 1
ok 2 - 3! = 3 * 2 * 1
ok 3 - (-5)! = 1
ok 4 - 5! = 120
# Looks like you passed all 4 tests

Iterative Factorial

Sweet, now we’ve got a nice suite of unit tests, but we’re not done. Let’s say we’ve decided that this factorial function is too slow since we have overhead from each recursive function call, so now we’ve been tasked with making this function iterative. Let’s see how our unit tests help us to ensure that the factorial function still works.

So we’ll start by making a first crack at the factorial function:

int factorial(int n) {
  if (n <= 0) return 1;

  int result = 1;
  for (int i = 0; i < n; i++) {
    result *= i;
  }

  return 0;
}

Alright, looks good. Let’s compile and run our unit tests:

$ ./factorial_test
1..4
ok 1 - 0! = 1
not ok 2 - 3! = 3 * 2 * 1
#   Failed test '3! = 3 * 2 * 1'
#         got: '2'
#    expected: '6'
ok 3 - (-5)! = 1
not ok 4 - 5! = 120
#   Failed test '5! = 120'
#         got: '24'
#    expected: '120'
# Looks like you failed 2 of 4 tests.

Uh oh, it looks like we made a mistake. Let’s take a minute and look at the errors. For the first test we got 2 when a 3 was expected and for the second one we got 24 when 120 was expected. Let’s compute the factorials up to 5 and see if we can find out what’s wrong:

1! = 1
2! = 2 * 1 = 2
3! = 3 * 2 * 1 = 6
4! = 4 * 3 * 2 * 1 = 24

Notice the pattern? It appears that our factorials fail to multiply by the final number so 3! becomes 2! and 5! becomes 4!. Let’s fix this, and see if we can’t make the solution clearer:

int factorial(int n) {
  if (n <= 0) return 1;

  int result = 1;
  while (n > 1) {
    result *= n;
    --n;
  }

  return result;
}

Now we re-run our tests and look at the results:

1..4
ok 1 - 0! = 1
ok 2 - 3! = 3 * 2 * 1
ok 3 - (-5)! = 1
ok 4 - 5! = 120
# Looks like you passed all 4 tests.

Great! All of our tests passed and they even helped us catch a bug when we “refactored” the factorial function. So now we have a fairly efficient factorial function that we can be reasonably sure is correct, good stuff.

The final result