Writing tests based solely on the Functional Spec / Design Doc

Upon reading Paul Holland’s blog post entitled ‘Functional Specification Blinders’ (found here: http://testingthoughts.com/blog/185), I harkened back to all my past test experience and realized that this was the main source for all my test plans, test cases or test ideas.

The functional spec document is the ‘bible’ for the developers. This is everyone’s main focus of attention when developing a new feature. If your developer(s) are solid and the spec is well defined and complete, it kind of makes sense that the solution should be fairly solid based on what’s in the design document. Kind of makes you question why tests are based solely on these documents – is there real value in that?

If you think about it, the functional spec is constantly reviewed by multiple stakeholders and SMEs, reworked and re-written continually. The solution is demo’d and goes through many variations of testing (CAT, UAT, etc).

It does make sense to expand your test ideas and strategies beyond what’s solely in the design/functional spec document. As Paul suggested, it may be a good idea to come up with ideas prior to even reading the spec.

This is not to say the spec should be ignored. It is an invaluable tool in testing the feature, particularly if you are thinking of writing regression tests.

Just do not make it your sole focus…

How much test documentation do we need?

Ahh, the dreaded test documentation question…

Ask yourselves these questions:
1. Will anyone else really read this?
2. Does anyone care about what I’m writing (see also #1)
3. What level of detail do I need to go into?
4. Who is my audience?

There are many other questions you may ask yourself. I find I ask myself these 4 questions each time before I document something.

If the answer to #1 or #2 is ‘No’, question why you do it, and what is the purpose. After all, the more time you are spending documenting about what you are, or have been doing testing, the less time you are actually testing!

The answers to #3 and #4 will often vary on what the documentation is. Is it a test plan, test idea, test charter, test session, requirements test, regression test, bug report, etc? Try and think about who may/will read this and write them up accordingly.

Asking yourself these questions before you start typing away, it will help save you some time, clarify what you are trying to say, and better target your audience.