Testing Gods – Part I

Whoa!!! The buzz around town was that "James Bach" was coming down to Bangalore to conduct a workshop on testing and hey guess what, he was also coming down to my company – Mindtree for a session and was scheduled to conduct a workshop for a lucky few testing folks (oh by the way, I was one of the lucky few). What I am going to deal with in this post is about the session that James took on ‘Exploratory Testing’.

One of the common misconceptions that people have about exploratory testing is that it is ‘ad – hoc’. James sought to put this misconception out by explaining to us about ‘Structured Exploratory Testing’. In order to simplify understanding, let me list down in points –

Identify what is your current scope of testing which could in itself have many layers. For example, I could currently narrow down my scope to functional testing. For greater granularity, I could also limit my scope to listing down which of the specific functionalities / features I would be testing on


Identify tools that can possibly make your job easier, MS Excel is one awesome tool where the scope for using the tool is limitless. If you are testing on a web application, there are various ‘add – ons’ that you could install to make your browser work for you


Now, we can get started on our testing –


Note the date and time when you start testing, this could come in useful when you are verifying log files


Make notes of whatever actions you do (if possible with the date and time too) and the observations that you make


A good tip is to note down all the defects and then log them in your defect tracking tool, so that you can optimize time


Session based exploratory testing (where you predetermine the test duration and scope) would be a great approach


Creating a Mind Map for each of your sessions on exploratory testing is very powerful and helps you a great deal on making notes, creating reports, traceability and many more


Pair testing with a peer tester is a great approach where you would notice that two brains work better than one


Pair testing with a developer is an even better approach, that way you won’t be wasting precious time in reproducing defects to your developer while also gaining developer confidence and respect

Note – the above list is obviously not exhaustive and is based on a number of assumptions (for more on assumptions, keep following this blog) like the tester knows what the requirements / functionalities of the product / application are, he knows which environment and platform he should be testing on and many more.

Some of the interesting questions that came up during the session were –

Q. How do you estimate testing effort without first getting your hands dirty on the product / application?
Ans. It is possible to estimate testing effort, if it is possible to estimate the number of bugs that are expected to be injected. In other words, just like it is not possible to estimate the number of bugs that can be injected, the same way it is not possible to estimate testing effort without getting your hands dirty first.

Q. Can we do Exploratory Automation Testing?

Ans. We can use Automation tools / browser add – ons, load generators, etc. to optimize Exploratory Testing

Q. Is Exploratory Testing sufficient for my software testing project?

Ans. It is best to have a good balance between Scripted Testing and Exploratory Testing and incorporate the findings (defects) from your Exploratory Testing sessions in to your formal Test Scripts

PS - follow this blog for more stuff on 'Testing Gods'

Risk Based Testing

Recently I came across a colleague of mine who was working on an assignment for a client of his on implementing the ‘Risk Based Testing’ methodology for their clients. The concept sounded very interesting to me and so I asked him to explain it to me further. So this is a brief of what they were implementing for their clients – the objective was to reduce the regression testing cycle by reducing the number of test cases to be executed and hence the amount of time taken to execute one regression test cycle.

          The methodology they used was very simple – traverse through all the test cases and assign a factor of ‘Priority’ to each test case which denotes the significance of the test case from the business standpoint and also assigning a factor of ‘Risk’ to the test case denoting the probability of the test case failing. Now the product of these two factors is taken and based on a predetermined baseline, only those test cases are executed which meet the baseline. Now all this may sound simple but what is very important here to note is the dependence on people who have a thorough understanding of the product or application on which RBT (Risk Based Testing) methodology is being implemented.

          The above approach is purely a mathematical / formula based approach to RBT, another approach to go about RBT is using the following heuristics for testing of the product / application under test – 

·         Losing the Confidence of Customers

·         Loss of Money

·         Loss of Life

·         Loss of Effort / Time

·         Loss of Business

·         Loss of Image / Brand

·         Losing Customers to Competitors

·         Losing Data

·         Not Adapting Quickly

What’s cool about the training aspiring testers / fresher’s are subjected to?

Well, this blog is actually inspired by the excellent training programs that are conjured up to train fresher’s from campus in the field of Software Testing. The investment in terms of time, effort and money is just phenomenal. They typically go through a series of lectures, power point presentations, quizzes, self-reading (more like mugging up definitions for your board exams), etc… for a period of 3 – 6 months at the end of which they take up exams (again very similar to the board exams conducted by our prestigious universities). And the interesting point to note here is that these very same companies lament the fact that the university curriculum does not really prepare students for the industry. Talk about hypocrisy.

          I am not sure how many hours Sachin Tendulkar spent in classrooms, theoretically learning about the nuances of batting; listening to lectures and power point presentations rather than honing his skills on the field. Or for that matter how many lectures Aamir Khan had to listen to learn the art of acting. Well I really don’t think they would have spent too many valuable hours listening to just theoretical concepts. They are excelling in their fields due to the fact that they learnt from their seniors by observing them, picking up tips from their respective coaches, honing their skills with hours of practice, being unafraid to make mistakes with the conviction that it will only help them become a better practitioner in their chosen field.

          This is true of not only sports and the film industry. Look at the people who paint our houses so well, or our excellent carpenters. They did not pick up their skills in the classroom. Which brings us to the question – “Why can’t we do the same in the software industry? Especially Software Testing”. We can introduce fresher’s from campus to Software Testing by first letting them observe how some experienced Testers test applications / products, then engage them in ‘Paired Testing’ sessions with experienced Testers and giving them good direction to do a lot of self-learning in the Software Testing space. One really good link where there is a consolidated source of key things an aspiring tester should learn is – 

http://prabhatnayak.blogspot.com/2011/11/roadmap-for-aspiring-tester.html

          Apart from this, there are also some great Software Testing experts, whose articles one can read and follow their blogs to get regular updates on Software Testing and also new approaches to Testing, some of them are listed below – 

http://www.satisfice.com/blog/

http://jonbox.wordpress.com/

http://testertested.blogspot.com/

http://www.developsense.com/blog/

http://testobsessed.com/

Using a Mind-Map to help you Test better

Well to start off with, this technique is surely not rocket science. I was introduced to this technique by Pradeep Soundararajan, one of the most respected Software Testers in India who blew me off my feet with the sheer simplicity of the technique. A Mind-Map is essentially a very simple way by which a Tester can visually represent all the hard work he has done in a graphical manner. This include the features he has tested, the questions (both resolved and unresolved) he had about the product under test, the bugs that he uncovered during testing and so on.

            Now, you may have a question as to how a Mind-Map can actually help us as Software Testers? Well, how many times have we faced a situation where we have done a lot of sanity / exploratory testing without really documenting them and at a later point in time, not really being too sure about the Test coverage / the features that we have tested. Consider another situation where the testing team has got a clarification from the development team and not documented it anywhere. At a later point in time, there could be an issue where both the development and testing teams claim that they are correct.

            Using a Mind-Map can help us document all this and more as is evident in the Mind-Map below (the Mind-Map shown below is for a GL Posting Entry in the FI module in SAP) – 

 

            We start with creating a central node for the ‘Test Case / Feature’ to be tested.  Next, we attach the ‘Test Case’ document (if available) as a sub node. The ‘Process / Functionality’ (features in scope) is listed as the next sub node. And next we get on to the action – ‘Test Execution’ where we list down the exact test steps that we run, when a particular test cycle has been run, what are the areas where there is an issue and needs to be flagged and so on. ‘Questions’ is also another sub node that can be added, where we document the questions / queries that we have raised along with the responses that we have received. And lastly our favorite node ‘Bugs / Issues’ where we document the bugs that we have uncovered along with the flag number (if applicable) from the ‘Test Execution’ and the ‘Defect Id’ from the defect tracking tool.

How you can go one up on the development team!!!

As testers, have you ever been faced with a situation where the developers think they are much more smarter than you and that you are not really capable of giving them technical suggestions / solutions on how to improve the quality, performance, visual appeal of the website.

Well, here’s a tool that helps you do just that. Well it’s not really a tool so to speak, to be more precise it’s a browser add-on for Firefox that gives suggestions on how to improve the performance of the website. The prerequisite for the add-on – ‘YSlow’ is the installation of the ‘Firebug’ add-on.

The interesting thing about ‘YSlow’ is that when compared with other add-ons like ‘HttpWatch’, it does not only tell you where the problem in performance lies but also tells you how you can address these issues on performance. ‘YSlow’ basically grades sites on a scale of ‘A to F’, A – being the best in terms of Performance and F – being really bad in term of performance. This grading is done considering about 23 rules / best practices that the ‘YSlow’ team has identified which are critical for gauging the performance of a website.

‘YSlow’ assigns a weight for each of the rules and the grading of a website is based on the total weight that a website gains. ‘YSlow’ also assigns a grade for each of the 23 rules so that Developers / Performance Engineers can make informed decisions on how to improve the overall performance of the website. And of course using the ‘YSlow’ add-on, you can always give excellent, well informed suggestions to your developers on how they can tweak their code so that the overall performance of the website improves.

The add-ons for ‘Firebug’ and ‘YSlow’ can be downloaded and installed from the link –

https://addons.mozilla.org/en-US/firefox/addon/yslow/

                In the screenshot above, ‘YSlow’ has graded the ‘Google’ homepage as ‘Grade A’ for the overall rating.  ‘Google’ also gets a grading of ‘A’ for the rule ‘Make fewer HTTP requests’ which is not really surprising considering the fact that the UI of the ‘Google’ home page is quite minimalistic whereas the ‘Google’ home page gets a grading of ‘F’ for the ‘Add Expires headers’ rule where if there is a ‘Expire Header’ assigned to all components on page, the component will be cached and hence there would be fewer HTTP requests the next time the page is loaded. Usually there is a ‘Expire Header’ assigned to images only because of their size but all other components like scripts, style sheets, etc. can be assigned  an ‘Expire Header’.