Monthly Archives: December 2006

Manual Software Testing: Suggested Best Practices

By Hung Q. Nguyen and Rob Pirozzi, LogiGear Corporation

This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives.

Introduction

There are many potential pitfalls to manual software testing, including:

  1. Manual testing is slow and costly.
  2. Manual tests do not scale well.
  3. Manual testing is not consistent or repeatable.
  4. Lack of training.
  5. Testing is difficult to manage.

This article will cover five “best practices” recommendations to help avoid the pitfalls associated with manual software testing.

Types of Software Testing

By Hung Q. Nguyen and Rob Pirozzi, LogiGear Corporation

This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives.

Introduction

When thinking of the types of software testing, many mistakenly equate the mechanism by which the testing is performed with types of software testing. The mechanism simply refers to whether you are using manual or automated software testing. This article goes beyond that simple mechanism-based definition to define the intrinsic nature of the tests themselves.

Understanding Metrics in Software Testing

By Hung Q. Nguyen and Rob Pirozzi, LogiGear Corporation

This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives.

Introduction

Metrics are the means by which the software quality can be measured; they give you confidence in the product. You may consider these product management indicators, which can be either quantitative or qualitative. They are typically the providers of the visibility you need.

Metrics

Metrics usually fall into a few categories: project management (which includes process efficiency) and process improvement. People are often confused about what metrics they should be using. You may use different metrics for different purposes. For example, you may have a set of metrics that you use to evaluate the output of your test team. One such metric may be the project management measure of the number of bugs found. Others may be an efficiency measure of the number of test cases written, or the number of tests executed in a given period of time.


The goal is to choose metrics that will help you understand the state of your product.


Ultimately, when you consider the value of a metric, you need to ask if it provides visibility into the software product’s quality. Metrics are only useful if they help you to make sound business decisions in a timely manner. If the relevancy or integrity of a metric cannot be justified, don’t use it. Consider, for example, how management analysis and control makes use of financial reports such as profit/loss, cash flow, ratios, job costing, etc. These reports help you navigate your business in a timely manner. Engineering metrics are analogous, providing data to help perform analyses and control the development process. However, your engineers may not be the right people to give you the metrics you need to help in making business decisions, because they are not trained financial analysts. As an executive, you need to determine what metrics you want and tell your staff to provide them.

For example, coverage metrics are essential for your team. Coverage is the measure of some amount of testing. You could have requirements coverage metrics, platform coverage metrics, path coverage metrics, scenario coverage metrics, or even test plan coverage metrics, to name a few. Cem Kaner lists over 100 types of coverage measures in his paper “Negligence and Testing Coverage.” Before the project starts, it is important to come to agreement on how you will measure test coverage. Obviously, the more coverage of a certain type, the less risk associated with that type.

The goal is to choose metrics that will help you understand the state of your product. Wisely choose a handful of these metrics specific to your type of project and use them to give you visibility into how close the product is to release. The test group needs to be providing you plenty of useful information with these metrics.

Conclusion

The metrics provided by testing offer a major benefit to executives: visibility into the maturity and readiness for release or production, and visibility into the quality of the software product under development. This enables effective management of the software development process, by allowing clear measurement of the quality and completeness of the product.

Software Testing: Cost or Product?

By Hung Q. Nguyen and Rob Pirozzi, LogiGear Corporation

This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives.

Introduction

Many look upon software testing as a cost. While it is true that software testing does cost money, in many cases significant amounts of money, it is also an activity that helps an organization to avoid costly failures further on in the development process. Most understand this relationship; software testing is spending money to save money. What many do not also realize is that software testing also produces valuable assets for the organization. This article will discuss those assets of software testing.