Pairwise Testing

Reading through Advanced Software Testing (for Test Analyst) today makes me understand more about pairwise testing, its concepts and how it’s designed both manually or automatically (by tools). Since this is very useful technique which every tester should know at least the basic or the insights, I think it’s worth to make a short notes briefing some important aspects of this test design technique. You can find a lot of information about it overnet but the concepts, examples and expressions are difficult to catch up. Here, I try to explain in a short, simple way to make it easiest for you to follow. Just a little practice and you’re done.

Pairwise Testing

1. Definition:
• A black-box test design technique
• Design test cases to execute all possible combinations of each pair of input parameters.

2. Benefit:
• Prevent combinatorial explosion
• Generate test cases that are much more effective than random selection or guessing

3. Application:
• Frequently used in compatibility and configuration testing

4. Manual process (illustrated by example)
Say, we want to do compatibility testing of a web application based on 3 factors as below:

• The first factor: connection speed (2 options: dial-up and broadband)
• The second factor: operating system (2 options: Windows XP, and Windows Vista.)
• The third factor: browser (2 options: Firefox, Internet Explorer)

First, construct a table called an orthogonal array:

Number of rows = the product of two largest numbers of options. We have all factors have 2 number of options, so number of rows = 2×2 = 4

Second, map the factor options with numbers, replace each instance of 0 with the first option, 1 with the second option and so forth.

We try with pair Speed and OS first, fill in the table possible combinations of pair Speed and OS, we have:

Then, for the pair Speed and Browser, do the same thing: fill in all their possible combinations, we have:

Now, check the pair OS and Browser, obviously we see that there’re duplication in combination of those 2 factors, we have combination 00 in both row 1 and 3, combination 11 in both row 2 and 4.

Thus, we have to edit combination for pair OS and Browser a little bit so that this pair has all possible combination. Try to edit row 3 and 4 and the result is:

Finally, check if we miss any combination for each pair?
• For the first pair of factors, Speed and OS, we have 00, 01, 10, 11, in rows 1, 2, 3, and 4.
• For the second pair of factors, Speed and Browser, we have 00, 01, 11, and 10, in rows 1, 2, 3, and 4. All four pairs, just occurring in a different row.
• For the third pair of factors, OS and Browser, we have 00, 11, 01, and 10, in rows 1, 2, 3, and 4. All four pairs, again, just occurring in a different row than the previous pairs of factors.
Yeah, so you have test cases that execute all possible combinations of each pair of input parameters. That’s the pairwise testing. The final step is just to remap the number to option name and you have the final result of nice test cases below:

5. Automatic process:
Download allpairs tool, read its manual and use it to generate test cases for you. If you learn to run it properly, the result is the same.

6. Further practice:
You may want to practice more because at the end of the day, we never have such simple situations like above example. Let’s roll your sleeves, try with the following exercise:

Design test cases using pairwise technique for a configuration testing based on 4 factors as below:
• The first factor is connection speed. We have two options: dial-up and broadband.
• The second factor is the operating system. We have four options: Mac, Linux, Windows XP, and Windows Vista.
• The third factor is the security settings. We have four options: native operating system security only, Symantec’s product, Trend Micro’s product, and McAfee’s product.
• The fourth factor is the browser. We have three options: Firefox, Internet Explorer, and Opera.

When you’re done, check with the following table to compare your answer. There is only 1 special thing in this exercise is that when you have unequal number of options in every factors, just give every factors number to the maximum. For example, in our case, factor Speed just has 2 options but we have numbers for it 0, 1, 2, 3. Later when you map those numbers with real option name, place ~ (means whatever option) to exceeding numbers.

References:
1. Advanced Software Testing (Test Analyst) – Rocky Nook
2. http://www.satisfice.com/tools.shtml

If you have any question or comments, feel free to drop me a line here or tweet me @khuongtw

Thanks for your reading.