Monthly Archives: August 2009

Are Use Cases Harmful For Test Automation?

People who know me and my work probably know my emphasis on good test design for successful test automation. I have written about this in “Key Success Factors for Keyword Driven Testing“. In the Action Based Testing (ABT) method that I have pioneered over the years it is an essential element for success. However, agreeing with me in workshops and actually applying the principles in projects turn out quite often to be two different things. Apart from my own possible limitations as a teacher, I see at least one more reason: the way the testing is involved in the development projects.

A typical development project will start with a global understanding of what the system needs to do, which is then detailed out further, for example into use cases. These use cases have proven to be helpful in implementation and various testing efforts, but I’m getting more and more the impression that they may also pose a risk for good test design when they are the only source of information for the test team. There are two reasons:

1. they tend to have a high level of detail

2. they usually follow the end-user perspective

Re 1: the level of detail of use cases is primarily aimed at the developers, and the information they need to know. More often than not it is implicitly assumed that this is also a good level for information for testers, to develop test cases from (for sake of simplicity I will not discuss the usefulness of techniques like exploratory testing, I will just assume that test cases are made and their execution is automated).

Re 2: in many tests it matters how a system handles transactions, and provides the correct and complete follow up actions and information. The end-user interacting with the UI is then not always relevant, and I would not like to see it explicitly specified as part of test cases (in ABT the UI specifics would be hidden in the ‘actions’). However, having the UI oriented use cases as the primary source of information makes it hard to focus on the transaction handling and other aspects of the system.

My message would be this: don’t start creating test cases from use cases, or similar developer oriented sources of information, before you have had a chance to create a high level test design, in which you specify which test products you’re going to create and what level of detail they would need to have. In some of my articles I write more about test design. You can find them on LogiGear’s Hans Buwalda profile page.

August 2009 – Three Kinds of Measurement and Two Ways to Use Them

by Michael Bolton

This article was originally featured in the July/August 2009 issue of Better Software magazine. Read the entire issue or become a subscriber.

People often quote Lord Kelvin: “I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.”But, few note the sentence that precedes the passage: “In physical science the first essential step in the direction of learning any subject is to find principles of numerical reckoning and practicable methods for measuring some quality connected with it.” The missing sentence prompts some questions: Are software development and testing sciences subject to the same kind of numerical measurement that we use in physics? If not, what kinds of measurements should we use? How could we think more usefully about measurement?