How to trick your Manager in to believing that you have executed all possible test cases

To all testing folks out there, just ask yourselves - How many times have you been in a situation where you could confidently tell your manager / client that you have executed all possible test cases and these are the bugs that were reported and after the bugs were fixed, you verified the bugs and retested the application and completed the rest of your testing activities and claimed that we have achieved 100% Test Coverage?

                But how many of us were actually telling the truth? To illustrate let me give you an example of a standard SAP screen for say “VA01 - Create Sales Order”. 

Assume that you are entering data in four fields present on the screen – Order Type, Sales Organization, Distribution Channel and Division. Now assume that each of the fields – Sales Organization, Distribution Channel and Division have 3 possible values for input data. So to achieve 100% Test Coverage, the tester has to test for 3 * 3 *3 = 27 possible input data combinations. Practically this may be sound, but imagine a situation where there are 6 possible values for input data. In such a scenario, there would be 6 * 6 *6 = 216 possible input data combinations. It may not really be a practical idea to execute the same test case over and over again for 216 iterations.

                To solve problems like this, we can use techniques like “Pairwise Testing” which is a combinatorial testing method. According to Wikipedia, this technique is based on the premise that “the simplest bugs in a program are generally triggered by a single input parameter. The next simplest category of bugs consists of those dependent on interactions between pairs of parameters, which can be caught with all-pairs testing”.

                Using tools like PICT (Pairwise Independent Combinatorial Testing) by Microsoft, Jenny, Hexawise, etc… testers can automate the process of arriving at the input pairs for testing where they are essentially accomplishing “Test Data Automation”. Another feature that these tools provide is – they give the option to testers to define some “input variables” which cannot be paired together and also assign priority to some “input variables” which are deemed to be critical by the tester. Features like this, can also help in cases where testers specifically want to do some negative testing by using some invalid pairs as inputs. The tools also provide an option of combining 3 or more input parameters thereby increasing the number of iterations to be executed.

                So there you have it folks a simple, guilt free, scientific way to trick your manager in to believing that you have executed all possible test cases by using “Test Data Automation” for your input values. Of course, one has to remember that 'Testing is Sapling at its best' and remember not to abuse the technique of using 'Pairwise'.

             

            If you are interested in using the “Pairwise Testing” technique, you can download the PICT tool by Microsoft –

http://download.microsoft.com/download/f/5/5/f55484df-8494-48fa-8dbd-8c6f76cc014b/pict33.msi