As experienced testers we all know how equivalence classes and boundary values can help us with test case design. Especially boundary value analysis (BVA) directs our attention to values that usually have a high probability of finding defects, which developers might have implemented simply by using the wrong relational operator. Let’s investigate:
Assume we want to test the implementation of the following requirement: “If you are 18 years or older you can apply for car insurance with our company”. The boundary is at “year = 18”, so we would test it with 17 (where we expect NOT to be able to apply for car insurance) and 18 (where we expect to be able to apply for car insurance). You can summarize your test cases (TCs) for this approach in the following table:
The above approach (1) shows the expected result due to a correct implementation. However, let’s see what the results are for the two test values 17 and 18, if the developer has – by error – implemented the decision using a wrong relational operator:
Because TC (1) and (6) give the same results for the input values 17 and 18, the wrongful implementation (x = 18) will not be detected. We would need a third test case for this, e.g. input value 19. TC (1) gives a true, TC (6) a false output for the input value 19, so the wrong relational operator can be detected. So far so good, that’s why we test high risk applications with THREE values per boundary, we all know that. Nothing new here, so where is the catch?
The difference between "coverage values per boundary" and "coverage values per boundary value"
Let’s look at the ISTQB CTFL Syllabus 4.0: “In 3-value BVA, for each boundary value there are three coverage items: this boundary value and both its neighbors. […] To achieve 100% coverage with 3-value BVA, test cases must exercise all coverage items, i.e., identified boundary values and their neighbors.” Well, the new version of the ISTQB syllabus explicitly does not talk about “three coverage values per boundary”, it talks about “three coverage values per boundary VALUE”. And this changes the rules dramatically:
Remember, in the old definition of the 3-value-BVA we tested the boundary of 18 with THREE coverage items: 17, 18 and 19 (as shown above). But now we have to use for the 3-value-BVA three coverage items per boundary VALUE. The boundary VALUES in the above example are 17 and 18. Therefore for the lower boundary value of 17, the coverage items are 16, 17 and 18 and for the upper boundary value of 18 the coverage items are 17, 18 and 19 (the boundary and its neighbors). Therefore: For the correct implementation of 3-value-BVA we need FOUR test cases: 16, 17, 18 and 19!
This approach using four coverage items per boundary has a further advantage. Black box test case design uses the specification as a test basis, hence 18 is the boundary and 17 and 18 are the boundary values. But we testers do not know how the developer has implemented this condition. Instead of “if x >= 18 then…” she might have implemented it like shown in TCs (7): “if x > 17 then…” – which is functionally equivalent. Doing black box testing, we do not know which implementation was chosen. Now look at the following table:
Testing this implementation with the old 3-value-BVA approach, using 17, 18 and 19 as implemented coverage items (7) will not detect the false coding in (12). However, the revised approach using four test cases (16, 17, 18, 19) will show the defect:It results in 16/false for TCs (7) and 16/true for TCs (12).
Testing high risk applications with four coverage items per boundary instead of three can detect defects that might otherwise be overlooked. In the end, you - as a well-educated and experienced tester – have to make a decision, which method to choose. Lots of factors need to be considered for this, now you know one more.
If you would like to learn more about test case design with the help of boundary values ("boundary value analysis"), then attend one of our ISTQB Foundation Level training courses online or at your preferred location. From 2024 according to the new ISTQB Foundation Level syllabus version 4.0.
