Just let me test the s**t out of it – my public rant!

With the advent of agile, lean, scrum/kanban and the like, the more I keep hearing things like: “We need to automate everything”, or “Anyone on the team can test”, “Unittests are good enough”, or my personal favorite “We don’t have time to manually test”. This reminds me of a phrase I once uttered in a meeting a long time ago, which i hope catches on:

“Just let me test the s**t out of it!”

So, what am I getting at here and why am I getting so hot and bothered?

With the proliferation of agile and it’s various derivations, seems to come this unhealthy view of testing. Anyone can do it, we need to automate everything, we are building so rapidly and iteratively that we can’t possibly manually regression test anymore, developer tests like unittests are good enough, and the like. So, where are these ideas coming from? These pervasive ideas are being touted online, in blogs/articles and in meeting rooms and boardrooms.

The idea that we can ‘automate everything’ is just plain silly. Automation is a great asset if it is used wisely. It should be designed and implemented to ‘aide’ in your testing by helping to alleviate pain points due to: the time it takes to execute reproducible scenarios, repetitive tasks, setting up and tearing down of test environments, creation of test data and any other regression checks that you may want in place. The idea that automation should be the only testing required or a replacement for actual testing, is ridiculous.

“Just let me test the s**t out of it!”

So, anyone can test eh? Anyone can do almost anything in life. It is how well you do it, and how long it would take you that matters. To use broad brush statements like that are harmful and just untrue. Hey look at me, I wrote an installation wiki page, so now I’m a Technical Writer. Hey look at me, I wrote a script to populate a table with records to use as test data, so now I’m a Developer. Hey look at me, I wrote a couple user stories, so now I’m a Product Owner. Hey look at me, I wrote out some requirements, so now I’m a Business Analyst. When I say examples like these, they seem silly right? Well so is the idea that anyone is a Software Tester. Do not diminish peoples skill sets, experience or training, and their dedication to the art and craft of testing.

“Just let me test the s**t out of it!”

Developer testing is good enough. I have heard this many times over my career and doing a quick Google search, you’ll see this all over the place. It is very important, there is no question (even more so than QA testing). The more we work hand in hand and alongside developers, the more quality is built into products. To say that dedicated testing is not required or needed, in some circumstances may actually be the case. But to say this all the time, is again not healthy. Dedicated testers provide inherent value: applying appropriate testing techniques, strategies, tools, training, the ability to use oracles/heuristics, the ability to adapt quickly to the context of what is being built, an unbiased view of the product/code, adopting a user mentality, the ability to divest themselves away from the technology, a second set of eyes, I could go and on. If you are in doubt of the value of your testers or test team, engage one of your senior testing leaders, talk to them, learn and most of all, listen!

“Just let me test the s**t out of it!”

We don’t have time to manually test or manually regression check anything. Say what? This is the most nonsensical thing I’ve heard in my many years of testing. However, we can devote endless hours of testers time to execute, analyze, debug, rework, and write automated checks (which provide suspect value in the first place in a lot of cases) but we have no time to manually test. As Tony Soprano would have said “Get the f**k outta here!’  Exploratory Testing is a way better alternative if you only have limited time. I highly encourage learning more about ET and its value. There is also the debate of testing vs. checking. As James Bach and Michael Bolton espouse all the time, there is a significant difference between checking vs. testing. For more information on this, a good resource is James Bach’s blog, an example blog post is: http://www.satisfice.com/blog/archives/856.

Scripted testing (which can be automated or manual), the use of automated checks and exploratory testing all have it’s place but should never be relied on solely – if at all possible.

Give me some time, armed with my own test strategy/plan, some test oracles/heuristics, my knowledge of implicit and explicit requirements, test ideas and checklists, and I’ll show you the value of manual testing and it’s necessity!

“Just let me test the s**t out of it!”

Ok, enough of this rant. Hopefully after reading this blog post, you take something away from it… otherwise my rant was ineffective and darn it I just lost valuable manual testing time.