Home » Testing » Test Cases Are Evil! Or are they?

Test Cases Are Evil! Or are they?

Ah test cases, the artifact everyone loves to hate. Why the hatred? Why no love? I am well aware of the testing communities dislike for them. But let me expound on some possible virtues of test cases.

Now before you (and the rest of the twitterverse) explode in animosity and rage over this, please sit down, grab a nice cold beverage, open your minds and free your thoughts. Be open-minded, non-judgmental and realize that everyone’s context is different and we do not all work in a utopian work environment. If you do, you can just stop reading and go back to playing fetch with your unicorn and experimenting with the varied usages of fairy dust.

Many of the downsides of test cases are well documented in blogs and articles.  They tend to be things like:

  1. The executor is just ‘checking’ and not ‘testing’
  2. Following a script may make you miss other important problems
  3. The executor tends to shut off their brain while executing
  4. You are only validating a small slice of functionality
  5. Anyone can execute them – they are not a replacement for skilled testers doing real testing
  6. If the series of steps in a test case are that important, then they should be automated
  7. A testers time is much more valuable doing other things then executing mundane, scripted test cases
  8. Test cases are frequently out of date and need to be constantly updated and revised

I could go on and on. While I do not disagree with most of these sentiments, what makes them downfalls can also make them valuable.

Huh?

How dare you!

The following are my thoughts on the benefits of test cases and how they can support your testing. As with everything in testing, they are based on context.

Ok, everyone take a deep breath, let’s continue…

Time

The bane of every tester. “I wish I had more time to test” is one phrase I’m sure we all utter from time to time. It is undoubtedly one of our biggest constraints (besides ‘resources’ that I will touch on next). For many of us who are constantly having to multi-task, context-switch, and be responsible for large numbers of products and features, test cases can be a useful resource to aid in our testing.  For little known or rarely used features, they can be helpful in doing simple checks (or smoke tests if you prefer). If release time is approaching and you have a large number of features to test, they can be useful if time is a concern. Using proper risk heuristics, to help determine what needs testing vs. checking, they can help you execute some simple scenarios and better yet, get others involved if need be.

Resources

Most organizations do not have enough testers and we sometimes need to bring in others to help. If you work somewhere that does have enough testers, and you are only responsible for a small portfolio of products and features, then good for you. For those that do not, resource management can be a problem. Where test cases can help, is by easily getting others involved in the testing.  This is important in regular regression checking and release testing. If time and resources are a problem, the ability to get developers, product owners, technical writers, UX people or anyone involved in testing is sometimes necessary. Test cases are written in a way that is transferable and easy to digest. The problem with checklists, mind maps and the like, are that they are easily understood by the author but not by anyone else. Also, when time and resources are a concern, the time spent hand-holding others and trying to explain artifacts and how to test something is not feasible either.

Reminders

When you own lots of features, there are times when you constantly ask yourself “What is the workflow again?” or “How does this work again?” or “How do I setup and configure this again?” or “What data do I need again?”  I think you get the point. Test cases can be a great reminder for testers as to how to configure, setup, and execute basic workflows. When time is of the essence, they can make a huge difference. A strong, professional tester should always veer off the path while executing a test case or use it as a base to guide them in their testing.

Domain Knowledge

For some that work in a “heavy” domain (think banking, aerospace, defense, medical, etc), the domain concepts can be a challenge. Who the personas are, how they work and how they will use and interact with your software can be challenging and difficult to wrap your head around sometimes. Having test cases to detail this information can be very important in giving you that context. Rather then just using a checklist of functionality to check, tying that into a persona and how they use that functionality, is important.

Product and Feature Knowledge

There are times when we need to test things that we do not know very well. When you are given a testing assignment that has a quick turnaround time, this is difficult. Having never worked with a feature, seen it in action or tested or experimented with it before is a tough challenge for a tester. The existence of test case(s) can enable quick ramp up times to understand basic workflows, personas, data, and basic items that may be important to check. Used wisely as a resource for a skilled tester, it can be greatly appreciated and useful.

Developing Shared Understanding

There are many standards used in organizations to help share information on how to do things. From a testing perspective, there is really only one that is universally understood by everyone – testers, developers, technical writers, product owners, product management, executives, outsource partners, support, services, and customers.  This is the old test case. Now whether or not it is the best way to test something is debatable, what is not debatable is the universal language it represents. This is where it ‘may’ have merit. This if for you and your organization to figure out.

Technical Debt

Most of us have a lot of technical debt. There are older legacy features that customers use that were either built in the early days with little to no automation, your team has built but did not automate or you inherited features that came with none. Finding the time to automate legacy products and features is tough. Your organization should help you prioritize this effort. In the meantime, the execution of legacy test cases may be necessary.

Better Than Nothing

I have often referred to the execution of a test case as ‘better than nothing’ testing – BTNT. While I do agree with a lot of the aforementioned downfalls of using test cases, in many of the examples I have described above, it certainly is better than nothing.

These are but a few examples of why I think test cases can have merit. So, before you dismiss someone’s use of them, think about their context, ask them why they use them, and have a discussion about the possible downfalls and benefits.

I’ve often espoused the benefits of doing exploratory testing over the use of executing test cases. But, I do think they ‘can’ have their place in some contexts.

And yes, if you can, automate the damn things as well!

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s