Testing Branches & Paths
Resources
One important concept to understand when writing if statements and if-else statements is the control flow of the program. Before we learned about conditional statements, our programs had a linear control flow - there was exactly one pathway through the program, no matter what. Each time we ran the program, the same code would be executed each time in the same order. However, with the introduction of conditional statements, this is no longer the case.
Because of this, testing our programs becomes much more complicated. We cannot simply test it once or twice, but instead we should plan on testing our programs multiple times to make sure they work exactly the way we want. So, let’s go through some of the various ways we can test our programs that include conditional statements.
Simple Example
First, let’s consider this example program:
x = int(input("Enter a number: "))
y = int(input("Enter a number: "))
if x + y > 10:
print("Branch 1")
else:
print("Branch 2")
if x - y > 10:
print("Branch 3")
else:
print("Branch 4")
We can represent the two if-else statements in this program in the following flowchart as well:
Branch Coverage
First, let’s consider how we can achieve branch coverage by executing all of the branches in this program. This means that we should test the program using different sets of inputs that will run different branches in the code. Our goal is to execute the code in each branch at least once. In this example, we see that there are four branches, helpfully labeled with the numbers one through four in the print()
statements included in each branch.
So, let’s start with a simple set of inputs, which will store
x
and y
. Which branches will be executed in this case? Before reading ahead, see if you can work through the code above and determine which branches are executed?
In the first if-else statement, we’ll see the x + y
is equal to
True
. So, we’ll execute the code in branch 1 this time through. When we reach the second if-else statement, we’ll compute the value of x - y
to be
False
, so we’ll execute branch 4. Therefore, by providing the inputs 6
and 6
, we’ve executed the code in branches 1 and 4.
So, can we think of a second set of inputs that will execute the code in branches 2 and 3? That can be a bit tricky - we want to come up with a set of numbers that add to a value less than
x
and
y
. When we reach the first if-else statement, we can compute the result of x + y
, which is
False
and we’ll execute the code in branch 2. So far, so good!
Next, we’ll reach the second if-else statement. Here, we’ll compute the result of x - y
. This time, we know that
True
, we’ll run the code in branch 3. That’s exactly what we wanted!
Therefore, if we want to test this program and try to execute all the branches, we only have to provide two pairs of inputs:
and and
However, that’s only one way we can test our program. There are many other methods we can follow!
Path Coverage
A more thorough way to test our programs is to achieve path coverage. In this method, our goal is to execute all possible paths through the program. This means that we want to execute every possible ordering of branches! So, since our program has 4 branches in two different if-else statements, the possible orderings of branches are listed below:
- Branch 1 -> Branch 3
- Branch 1 -> Branch 4
- Branch 2 -> Branch 3
- Branch 2 -> Branch 4
As we can see, there are twice as many paths as there are branches in this case! We’ve already covered ordering 2 and 3 from this list, so let’s see if we can find inputs that will cover the other two orderings.
First, we want to find a set of inputs that will execute branch 1 followed by branch 3. So, we’ll need two numbers that add to a value greater than 10, but their difference is also greater than 10. So, let’s try the value
x
and
y
. In that case, their sum is greater than
x - y
and find that it is also
The last ordering can be done in a similar way - we need a pair of inputs with a sum less than
x
and y
. In this case, their sum is
As we can see, path coverage will also include branch coverage, but it is a bit more difficult to achieve. If we want to cover all possible paths, we should test our program with these 4 sets of inputs now:
and and and and
There’s still one more way we can test our program, so let’s try that out as well.
Edge Cases
Finally, we should also use a set of inputs that test the “edges” of the Boolean expressions used in each if-else statement. An edge is a value that is usually the smallest or largest value before the Boolean expression’s output would change. So, an edge case is simply a set of input values that are specifically created to be edges for the Boolean expressions in our code.
For example, let’s look at the first Boolean logic statement, x + y > 10
. If we are simply considering whole numbers, then the possible edge cases for this Boolean expression are when the sum of x + y
is exactly
False
, but when the result is
true
. So,
True
values and False
values. The same applies to the Boolean expression in the second if-else statement.
So, to truly test the edge cases in this example, we should come up with a set of values with the sum and difference of exactly
x
to
y
to
False
side of the edge. Our program will correctly execute branches 2 and 4.
For the other edge, we can use a similar set of inputs where x
is
y
is still
True
and we’ll execute branches 1 and 3.
Why is this important? To understand that, we must think a bit about what was intended by this program. What if the program should execute branches 1 and 3 if the value is greater than or equal to
>=
instead of the simple greater than symbol >
in our code. So, a minor typo like that can change the entire execution of our program, and we wouldn’t have noticed it in any of our previous tests. This is why it is important to test the edge cases along with other inputs.
In fact, it would probably be a good idea to add a third edge case, where the sum and difference equal
True
and False
values in this program.
So, the final set of values we may want to test this program with are listed below:
and (branch coverage 1 and 4) and (branch coverage 2 and 3) and (path coverage 1 and 3) and (path coverage 2 and 4) and (edge case 2 and 4) and (edge case 2 and 4) and (edge case 1 and 3)
As we can see, even a simple program with just a few lines of code can require substantial testing to really be sure it works correctly! This doesn’t even include other types of testing, where we make sure it works properly with invalid input values, decimal numbers, and other situations. That type of testing is really outside of the scope of this lab, but we’ll learn more about some of those topics later in this course.
Complex Example
Let’s go through one more example to get some more practice with creating test inputs for conditional statements. For this example, let’s consider the following program in Python:
a = int(input("Enter a number: "))
b = int(input("Enter a number: "))
if a // b >= 5:
# Branch 1
print(f"{b} goes into {a} at least 5 times")
else:
# Branch 2
print(f"{a} is less than 5 times {b}")
if a % b == 0:
# Branch 3
print(f"{b} evenly divides {a}")
else:
# Branch 4
print(f"{a} / {b} has a remainder")
This program is a bit trickier to evaluate. First, it accepts two numbers as input. Then, it will check to see if the first number is at least 5 times the second number. It will print an appropriate message in either case. Then, it will check to see if the first number is evenly divided by the second number, and once again it will print a message in either case.
Branch Coverage
To achieve branch coverage, we need to come up with a set of inputs that will cause each print statement to be executed. Thankfully, the branches are numbered using comments in the Python code, so it is easy to keep track of them.
Let’s start with an easy input - we’ll set a
to
b
to
a // b
is equal to
a % b
is exactly
b
will evenly divide the
a
without any remainder. So, we’ve covered branches 1 and 3 with this input.
For another input, we can choose
a
and
b
. In the first if statement, we’ll see that a // b
is now
a % b
is equal to
Therefore, we can achieve branch coverage with just two inputs:
and and
Path Coverage
Recall that there are four paths through a program that consists of two if statements in a row:
- Branch 1 -> Branch 3
- Branch 1 -> Branch 4
- Branch 2 -> Branch 3
- Branch 2 -> Branch 4
We’ve already covered the first and last of these paths, so we must simply find inputs for the other two.
One input we can try is
a // b
will be
a % b
, so we’ll go to branch 4 instead. This covers the second path.
The third path can be covered by inputs
a // b
is now
a % b
is exactly 0.
Therefore, we can achieve path coverage with these four inputs:
and and and and
Edge Cases
What about edge cases? For the first if statement, we see the statement a // b >= 5
, so that clues us into the fact that we’ll probably want to try values that result in the numbers
Likewise, the Boolean statement in the second if statement is a % b == 0
, so we’ll want to try situations where a % b
is exactly
So, with a bit of thinking ahead, we can come up with a set of just 4 inputs that not only achieve full path and branch coverage, but also properly test the various edge cases that are present in the program!