Monthly Archives: August 2010

Professor Jeff Offutt provides some insight to LogiGear Magazine on how to become a software tester

Professor Jeff OffuttJeff Offutt – Professor of Software Engineering in the Volgenau School of Information Technology at George Mason University – homepage – and editor-in-chief of Wiley’s journal of Software Testing, Verification and Reliability,

LogiGear: How did you get into software testing? What do you find interesting about it?

Professor Offutt:
When I started college I didn’t know anything about computers. I was a math major and in my first semester, my adviser convinced me to take an introductory programming course that was being specialized for math majors. Programming was taught in the business department, so a math-focused class was quite different and the faculty wanted to make sure enough students took it.

I was immediately hooked. Programming was fun and it was easy! (Unlike calculus, which I didn’t like.) It took me a few semester to find out that the pre-engineering majors who thought calculus was easy and fun often found programming hard. I was shocked to find that companies actually paid good money to programmers!

But I was frustrated by how much effort it took to write really good software. Design was poorly done, languages and programming tools were terrible (I started with BASIC and COBOL on punched cards!), and debugging was horrible. So I wanted to do whatever I could to help build quality software. When I got to graduate school I met a professor who was working on testing and I quickly found I could apply my love for discrete math (not continuous math) to making better software. So I’ve worked on testing, and especially automated testing, ever since.

LogiGear: What kind of work are you doing? How did you pick those specific topics, anyway?

Professor Offutt: I’ve found the hardest problem in testing is the key issue of getting test values. Most other aspects are either easy, straightforward, mechanical, or automated. And I’ve had a passion for inventing criteria and algorithms that can automatically generate good tests since my PhD thesis work.

A few years ago I agreed to teach a class on designing and building Web applications. I immediately realized that deploying software on the Web affected every aspect of software engineering, including testing. So I developed a technique for modeling Web software component interactions that can be used for testing (and other activities). I also invented a black-box technique called bypass testing, and I am currently working on a mutation model to test the novel control and state interactions that Web applications use.

I pick problems based on what I have trouble with as a programmer or as a user. And it helps if the problems are interesting to students.

LogiGear: How can a college student prepare to go into software testing and become really good at it? What should he or she look for in teachers, courses, and methods?

Professor Offutt: Testing is truly entering a golden age. The need for software quality has been increasing dramatically, and new ideas like Agile processes put a heavy emphasis on testing. A tester should have two qualities: (1) an innate need to have high quality software, and (2) a very logical mind.

Give me someone who programs a little slowly, but who turns in programs without faults.

LogiGear: What sort of graduate programs should college graduates consider?

Professor Offutt: A testing researcher should be very good at discrete math. Logic and set theory, graphs, grammars and finite state machines — and abstract algebra if she can get it. Look for programs that are based on 21st-century concerns. Look for universities that have lots of software engineering classes instead of the standard one. Do they have an undergrad and graduate course in testing? Do they have more than one? How many faculty list software engineering and software testing as their FIRST research area? Many CS programs are teaching the same material that I learned as a student in the early 1980s. The industry has changed completely — how can all that material still be relevant? The answer is easy: Most of it is not.

LogiGear: Also, in your opinion, what are some of the more interesting research questions people are asking now and what do you think they’ll be researching in, say, 5 years?

Professor Offutt: New technologies (like the Web) are great sources for new software testing research problems. Emergent properties like security and usability are also major growth areas for the near future. Research is like science fiction — it takes 15 or 20 years to go from research ideas to practical use; so what will we need 15 years from now? The Web is a great example. All the ideas were laid out in PhD theses and conference papers in the 1970s and 1980s, then the Web was created from those ideas in about 1990, and finally Web applications were being developed a few years later.

LogiGear: Thank you, Professor Offutt.

BJ Rollison shares his thoughts on the testing industry in Viet Nam

BJ Rollison, Software Test Architect for Microsoft, is planning to speak at the Viet Nam International Software Testing Automation Conference 2010 (VISTACON 2010) hosted by LogiGear Corporation from 20 – 22 September 2010.

Mr. Rollison started working for Microsoft in 1994, becoming one of the leading experts of test architecture and execution at Microsoft. He also teaches software testing courses for the University of Washington, and sits on the advisory board for testing certification at the University of Washington, the University of California Extension Santa Cruz, and Lake Washington Technical College.

In his keynote address, he will be sharing ideas on some of the influences driving change in software testing, some common problems in our discipline, and suggestions on ways we can meet our future testing challenges.

We recently had the opportunity to interview him as he considered his upcoming visit. One of the more striking results of his responses was his optimistic tone about Viet Nam’s software-testing industry. Also, like many who are visiting here having only read or heard about it, he is excited to actually experience Viet Nam — the country that some, including analyst John Stepek of the UK’s Money Times, called “Asia’s New Tiger.”

What do you think about the development of the software testing industry in Viet Nam?

[BJ ] This is really a great question. But, I must confess that my knowledge of the software-testing industry in Viet Nam only comes from what I have read or been told second hand. I am really amazed at the rapid growth of the software industry in Viet Nam as a whole. The software industry in Viet Nam has been increasing by more than 20% per year over the past decade, and there is a lot of opportunity for continued success. As distributed development teams continue to flourish in our global economy I am very confident that Viet Nam will continue to have a bigger impact. I think Viet Nam will emerge as a desired business partner not just because of the lower costs as compared to India, but because of both the high quality of service and also the low turnover of employees — something that is very important for building stable relationships between companies. So, I am glad to have the opportunity to visit and learn more about the software-testing industry in Viet Nam.

How do you feel about your first time speaking at a conference in Viet Nam?

[BJ ] Honestly, I am a little nervous and very excited. I suspect that most people get nervous every time they have to give a speech. Since this is the first software-testing conference in Viet Nam I sincerely hope that my small contribution will add value to the attendees and help make the conference an over-whelming success. I am also excited to visit Viet Nam for the first time. Having studied Asian history I have great respect for the rich cultures around the world. I can’t wait to learn more about the country and the Vietnamese people in person rather than from a book.

What made you decided to participate in this conference?

[BJ ] My dear friends Michael Hackett and Hung Nguyen told me about their company’s progress in Viet Nam and I was very impressed. When they asked me to speak at this conference I felt deeply honored. Because Viet Nam’s software testing industry is emerging so rapidly, it is a privilege to speak at its first testing conference. I also value the opportunity to learn more about the software-testing industry there and to meet many of the software professionals in Viet Nam.

What are you going to present at the conference and what experiences are you going to share with Vietnamese engineers about software testing?

[BJ ] I am presenting a tutorial on combinatorial testing. Testing complex systems is a difficult challenge that many testers face. In this tutorial I hope to share how we train our testers at Microsoft to design a set of effective tests when faced with these challenges. I will also talk about a technique to design and build test-data generation tools to help testers generate test data. This is a significant innovation that can save companies a lot of time and money. I will also give a keynote address on the future of the software-testing industry based on my experiences and some of the strategic visions within Microsoft.

Viet Nam International Software Testing and Automation Conference 2010 (VISTACON 2010) is the first international software-testing conference in Viet Nam with the theme of “ADVANCING THE STATE-OF-THE PRACTICE OF SOFTWARE TESTING AND TEST AUTOMATION”. VISTACON brings leading thinkers from around the globe such as Microsoft, LogiGear Corporation, Florida Institute of Technology, Electronic Arts and McAfee.

The conference serves as the central resource for all software test professionals in the Asia-Pacific region to learn, share expertise, gather information and solve complex testing and test automation problems.

VISTACON 2010 will be held at the White Palace Convention Center, Ho Chi Minh City, Viet Nam 20-22 September 2010. For more information about VISTACON, please visit http://www.vistacon.vn

10 Essentials for Effective Test Automation

Test automation can provide great benefits to the software testing process and improve the quality of the results…. but its use must be justified and its methods effective.

The reasons to automate software testing lie in the pitfalls of manual software testing…

As we all know too well, the average manual software testing program:

Adapted from the LogiGear white paper

Offshore Software Test Automation: A Strategic Approach to Cost and Speed Effectiveness“.

- Is slow and costly
- Is difficult to manage
- Does not scale well
- Is not consistent and repeatable

Effective test automation resolves each of these issues, allowing management to:

  • Drive down costs
  • Bring software to market faster
  • Gain critical awareness into QA status

How, then, can test automation be made effective?

The most essential element of effective software test automation is a strong foundation in methodology. Methodology drives tool selection and the rest of the automation process. It also helps to drive the approach to offshoring the “appropriate” pieces of the testing process.

Here is the short list every manager needs to make methodology work for them:

10 Essentials for Effective Test Automation:

  1. Know the steps of the software development process and how they relate to each other.
  2. Have a solid understanding of the required planning.
  3. Understand that software testing is a strategic effort.
  4. Commit to giving software testing its own budget and funding.
  5. Use the Action Based Testing (ABT) methodology and choose the right enabling technologies that support it.
  6. Put in place the right people with the proper skills and training.
  7. Separate test design from test automation so that automation does not dominate test design.
  8. Lower costs by using less expensive labor than a local team.
  9. Integrate global resourcing strategies and best practices.
  10. Jumpstart the process with a pre-trained outsourcing partner.

Testing in Agile Part 2 – AGILE IS ABOUT PEOPLE

Michael Hackett, Senior Vice President, LogiGear Corporation

If your Agile implementation is not about people, you’ve missed the boat!  The most profound impact to becoming more Agile is happier teams!

Agile manifesto Value #1:

* Individuals and interactions over processes and tools

Words like these do not show up in Waterfall or RUP SDLC process descriptions.

Agile cannot get more basic than this: people, the team members, you, are more important than the process documents, best practices or any standard operating procedures. People sitting in a room together, talking, hashing out an issue, live, beats out any project management tool every time.

At the core of the Agile movement are a few ideals that often get clouded by structure, the need-for-speed or just taken for granted. First, technology people and knowledge workers have to be happier at their jobs for long term productivity and higher quality work output. The hangover and subsequent bust from the dot-com boom have done more to spread Agile ideas than the cost savings aspects. This has been a grass roots movement.

As Ron Jeffries says in “Extreme programming explained,” XP is:

  • An attempt to reconcile humanity and productivity
  • A mechanism for social change

These are not casual statements. They are not meant to be taken lightly.

Trust

It is difficult to find a valuable discussion of Agile, Scrum or XP that does not highlight trust, empowerment, courage, or respect.  In my opinion, a groundbreaking aspect of the Agile/XP/Scrum movement is Chickens and Pigs.

This is an oft repeated story attributed to Ken Schwaber concerning chickens and pigs. Here is goes:

A chicken and a pig are together when the chicken says, “Let’s start a restaurant!”
The pig thinks it over and says, “What would we call this restaurant?”
The chicken says, “Ham n’ Eggs!”
The pig says, “No thanks, I’d be committed, but you’d only be involved!”
SCRUMGUIDE — By Ken Schwaber, May, 2009

In terms of Scrum, the Team members, the doers are called “pigs.” They have skin in the game. Management, sales-people involved in product delivery but who do not produce it, are “chickens.”  In Scrum, chickens cannot tell “pigs” how to do their work.

In Agile, management and sales teams (chickens) “back off” from solely determining code delivery since they do not write the codes. The people developing the code (pigs) know how fast or slow ultimate software delivery wiill be – and they should be driving these schedules and milestones. The doers (pigs) and the business balance delivering value with working at a sustainable pace. The pigs are not dominated by chickens. For many years, technology teams have complained about this domination which destroys teamwork. This is where trust is proven or broken.

This turns out to be a major stumbling block for many companies attempting to say they are Agile. Often, management and sales still want to tell development teams how long their work should take, most often without understanding the complexity. This legacy notion of top down management direction is not sustainable.

This refocus of development teams (pigs) influencing schedule and level of effort stops Taylorism (production line management theory – where management dictates everything) dead in its tracks.  Agile implies that we hire smart, creative, effective people (pigs) – even experts!  Let the pigs tell the managers (chickens) how long it will take to complete their work. Let the doers be creative and learn about the best way to finish their tasks. Let the pigs find out what is a sustainable pace rather the chickens imposing one on them.

Working at a sustainable pace will eliminate or reduce one of the highest job dissatisfaction: overtime or even worse, weekend work!  Let us, the people actually doing the work, commit our feature development to delivery dates. Estimating, scoping, planning “poker” – this is all bottom up, not top-down.

It’s pigs and chickens working together- not one dominating the other! Agile empowers people!

Where is my desk?

With such a high focus on easy communication, Agile development teams need to sit together, preferably in a bullpen or at least be together to talk freely, every day to get the full benefits of Agile; this must include “the business” — a customer or customer representative.  And these “customers” need to participate in all facets of development; just participating in the daily scrum is not enough. It’s sitting together, working out problems together, talking to each other, asking each other questions with no delay or dead time waiting for email responses or the next weekly meeting. Communication as a constraint in today’s world is intolerable and frustrating.

Whatever your implementation of practices, keep in mind: communication, communication and more communication. Focus on resolving communication problems, easy communication, transparent communication, eliminate lack of response (close the loop) and feelings of being ignored or dominated. Teams needs to respect each others’ work and treat each other as humans. Simple as this seems, adversarial team members, people who never meet, do not know each other or finger-point and blame can not make good products! Team members getting to know each other, sitting together with open communication is a major step toward happier teams making better products. This sounds nice but in a world of remote teams, remote offices, home offices and too-busy multi-taskers, software development is often de-humanized. This needs to stop.

Most books you find on Agile development include a chapter on how you sit, where you sit — looking out a window or facing the center. Sitting together for pair programming, team building is very important, not only to just Agile: focusing on getting along better will make everyone in the team stronger! The bottom line: sitting together fosters feedback and open communication.

The work environment has always been a key ingredient in job satisfaction.

If the number of meetings you attend, the hours spent in meetings, has not massively decreased, you have missed the Agile boat. Death by meeting is SO last century. Enough said.

Teams full of happy people…

Empowerment — the idea of self-forming, self-directing teams — is the foundation of Agile software development. What works for one team may not work for another, even if the teams sit next to each other and work on the same product. Each team has its own individual team dynamic; often times, creativity and problem solving skills differ from group to group.  Teams build skills and when they find something that works, they can share it with other teams.

The extension of the Chickens and Pigs discussion is that teams commit to delivery. The whole team commits or no one commits. Everyone is responsible for quality – no finger pointing.

Multitasking is a failed notion – focus on one thing at a time! A quick web search on new research about how the brain works reveals multitasking diminishes capability and productivity! Agile focuses on a narrow scope which makes teams more productive and happier.

If continuous process improvements are not coming out of your retrospectives, you’re breaking one of the primary tenants of Agile. No one gets it right the first time – learn by doing! Make it better over time. As the team grows and skills grow – optimize and evolve your processes.

Key principles of Agile:

* Teams need to constantly reflect on how to improve their processes and methods.

Retrospectives not only demonstrate finished functionality, but your group must also talk about lessons learned, how to make work more efficient for future sprints, how to reduce stress, and what new tools might help future development.  Hopefully, you have put an end to useless postmortems.

Summary

We all want to be happy in our jobs. All managers want employee retention. You can’t get quality products from burned-out, stressed, distrustful, frustrated or angry people.  In software development, en masse, we finally realized this. Agile development concepts have been a significant step forward. For many organizations, Agile ideas and methods have been a complete remedy for companies who’ve struggled with bad team dynamics.

When thinking about Agile, remember…. Agile does not mean faster. Agile is about people.

Testing in Agile Part 1 – INTRODUCTION TO AGILE

Michael Hackett, Senior Vice President, LogiGear Corporation

Testing in Agile Part 1 – INTRODUCTION TO AGILE

In case you missed the first part of the series in our last magazine issue from Michael Hackett, Agile’s impact on software development teams is huge. For test teams it can be even more pronounced — and good, especially if your existing projects have been problematic.

There are many positive things about the move to be more Agile and responsive in software development, but like so many things the devil is in the details! What Agile actually means in your organization and what practices will be implemented vary dramatically from company to company. There is a rush to be Agile these days and I see it mis-implemented more than well implemented. I believe “Agile” itself needs to be re-understood, if not re-conceived.

Here is what I mean: Agile does not equal faster. It is not Scrum, it is not XP, nor is it Lean. Agile is not an SDLC or how to manage a project or how developers write code. Agile means too many different things to many people. This series of articles in not meant to be an Intro to Agile. The discussion here centers not on what Agile is or is not. It focuses only on those aspects which directly impact software testing and how traditional testers need to be prepared to be Agile.

Most of the white papers and speakers you find these days on Agile, Scrum and XP focus on its practices. There is more than enough written about things like test-driven development, sprint retrospectives, and estimating user stories. My focus is different.

First, I will focus on software testing. And by this I mean testing in the traditional sense, not in the unit testing/user story acceptance testing sense. Secondly, I will focus on the areas of Agile development that are most overlooked but quite important to successful implementation of Agile, such as team dynamics, self-forming or self-directing teams, and tools.

Let me layout some overview themes that guide my views on Agile and are critical points to be made about this movement.

1- These ideas are not new. In 1987, DeMarco and Lister famously wrote about software development teams in their groundbreaking book, Peopleware:
Things that have a strong positive effect:

  • Fellow team member interactions
  • Workplace physical layout
  • Protection from interruptions

Things that have a negative effect:

  • Overtime
  • Lack of closure

Agile addresses these issues like no other set of ideas in software development or project management before it. There is no Agile, whatever project management mechanism you choose, that is not based upon teams sitting together and focusing on one set of tasks with excellent, full-team, daily communication.

2- Agile is not about getting faster!
Agile is about people and interactions being more important than processes and tools, working at a sustainable pace, and self-directed teams!

People working on Agile projects should be more satisfied with their work, do it at a more sustainable pace and feel better about the quality of the released product. If not, your company has really missed the boat. If Agile at your company is about a tool, or set of tools implemented, something is very wrong.

3- Testers have recommended many Agile ideals and practices for a long time.

  • “Unit testing is really good- we should do more!” How long have testers been saying that?
  • “Accept the fact that we always accept late changes rather than pretend we don’t.”
  • “I [formerly known as QA] don’t own quality- we all do!”
  • “You are going to stop calling my team QA.”
  • “We need to automate more.”
  • “We need to design the code for testability.”

The fact is that test teams have known many good Agile practices for a long, long time. In some cases, using more Agile development practices may not change the job of a traditional tester; the focus of testing may not change very much. But how the projects are managed will change.

The following series of discussions on some aspects of Agile development that will explore how implementing Agile development practices and values can dramatically change your job satisfaction, level of respect, and work situation for the better — if practices are implemented and not just given lip service.

The series is entitled: “Testing in Agile” – Please read on to learn more about how Michael tries to help testers new to Agile Development

Professor Jeff Offutt provides some insight to LogiGear Magazine on how to become a software tester

Professor Jeff OffuttJeff Offutt – Professor of Software Engineering in the Volgenau School of Information Technology at George Mason University – homepage – and editor-in-chief of Wiley’s journal of Software Testing, Verification and Reliability,

LogiGear: How did you get into software testing? What do you find interesting about it?

Professor Offutt:
When I started college I didn’t know anything about computers. I was a math major and in my first semester, my adviser convinced me to take an introductory programming course that was being specialized for math majors. Programming was taught in the business department, so a math-focused class was quite different and the faculty wanted to make sure enough students took it.

I was immediately hooked. Programming was fun and it was easy! (Unlike calculus, which I didn’t like.) It took me a few semester to find out that the pre-engineering majors who thought calculus was easy and fun often found programming hard. I was shocked to find that companies actually paid good money to programmers!

But I was frustrated by how much effort it took to write really good software. Design was poorly done, languages and programming tools were terrible (I started with BASIC and COBOL on punched cards!), and debugging was horrible. So I wanted to do whatever I could to help build quality software. When I got to graduate school I met a professor who was working on testing and I quickly found I could apply my love for discrete math (not continuous math) to making better software. So I’ve worked on testing, and especially automated testing, ever since.

LogiGear: What kind of work are you doing? How did you pick those specific topics, anyway?

Professor Offutt: I’ve found the hardest problem in testing is the key issue of getting test values. Most other aspects are either easy, straightforward, mechanical, or automated. And I’ve had a passion for inventing criteria and algorithms that can automatically generate good tests since my PhD thesis work.

A few years ago I agreed to teach a class on designing and building Web applications. I immediately realized that deploying software on the Web affected every aspect of software engineering, including testing. So I developed a technique for modeling Web software component interactions that can be used for testing (and other activities). I also invented a black-box technique called bypass testing, and I am currently working on a mutation model to test the novel control and state interactions that Web applications use.

I pick problems based on what I have trouble with as a programmer or as a user. And it helps if the problems are interesting to students.

LogiGear: How can a college student prepare to go into software testing and become really good at it? What should he or she look for in teachers, courses, and methods?

Professor Offutt: Testing is truly entering a golden age. The need for software quality has been increasing dramatically, and new ideas like Agile processes put a heavy emphasis on testing. A tester should have two qualities: (1) an innate need to have high quality software, and (2) a very logical mind.

Give me someone who programs a little slowly, but who turns in programs without faults.

LogiGear: What sort of graduate programs should college graduates consider?

Professor Offutt: A testing researcher should be very good at discrete math. Logic and set theory, graphs, grammars and finite state machines — and abstract algebra if she can get it. Look for programs that are based on 21st-century concerns. Look for universities that have lots of software engineering classes instead of the standard one. Do they have an undergrad and graduate course in testing? Do they have more than one? How many faculty list software engineering and software testing as their FIRST research area? Many CS programs are teaching the same material that I learned as a student in the early 1980s. The industry has changed completely — how can all that material still be relevant? The answer is easy: Most of it is not.

LogiGear: Also, in your opinion, what are some of the more interesting research questions people are asking now and what do you think they’ll be researching in, say, 5 years?

Professor Offutt: New technologies (like the Web) are great sources for new software testing research problems. Emergent properties like security and usability are also major growth areas for the near future. Research is like science fiction — it takes 15 or 20 years to go from research ideas to practical use; so what will we need 15 years from now? The Web is a great example. All the ideas were laid out in PhD theses and conference papers in the 1970s and 1980s, then the Web was created from those ideas in about 1990, and finally Web applications were being developed a few years later.

LogiGear: Thank you, Professor Offutt.