Since my transformation to agile, continuous delivery and the concept of attempting to deliver value to customers quicker than ever before, I have often struggled with the concept of an Alpha, Beta or MVP (Minimum Viable Product). What makes it an alpha/beta? Viable to whom? What makes it minimum? How is it possibly a product in this early, unfinished state?
I recently read a very good article by Henrik Kniberg (@henrikkniberg) where he tries to explain the concept of MVP as well: http://blog.crisp.se/2016/01/25/henrikkniberg/making-sense-of-mvp. I personally love his analogies to ‘Earliest Testable Product’, ‘Earliest Usable Product’ and best of all ‘Earliest Lovable Product’. Although articles like this help in the understanding, it really does not help testers understand how to test them.
One of the major stumbling blocks for me as a tester is determining what to test, what not to test, and what to not worry about for now. This often generates statements from the business like: “It’s good enough for now”, “We’ll cross that bridge when we come to it”, “Let’s just wait until we get feedback”, “Um yeah that’s a problem but…”.
When my company switched to agile teams, my team was one of the first to build a separate component feature that was going to get into the hands of the customer early and often for feedback. There were many challenges with this obviously. The toughest part for me (and my team) was understanding how to test this thing and what to care about from a quality perspective. The first time our PO said that we were going to ship the product to them at the end of the next sprint, I went into typical tester panic-mode! Drop everything else, try and test everything, report everything I find, document everything, etc. To encounter blase feedback to issues I was finding was disheartening and challenging to understand. How can we deliver a product with all these incomplete pieces, with all these known issues (many not even raised) but more importantly – how can I help the team make informed decisions about the level of quality and what is important for the team to know about prior to shipping?
Examples of many of the things I found were: design wire frames did not match what we had, use cases were incomplete and workflows were not executable, usability issues, functionality bugs, performance and reliability issues, data validation and error handling not done and the list goes on and on.
I tried my best to model my testing to represent the product in its current state of development, so I always knew what it was supposed to look like and behave like at any given point in time. I was doing this well. But, how do I handle all the issues I was finding and how do I wrap my head around the concept and idea that the product is ‘good enough for now?’ As a tester, I found this very difficult.
I still do not have all the answers and always welcome feedback and ideas! Below are some items that I do that help:
- Document known issues (not necessarily as raised bugs) and keep them around for discussion purposes with the team.
- Continually provide feedback to the team as to your own perceived level of quality – find ways of doing this effectively.
- Get the whole team involved in traditional testing activities (including the PO). You would be surprised how quickly issues can get fixed when others see how annoying they are too!
- Get the UX/UI designers involved in the testing of design and usability issues. This can help in making the call that ‘this is good enough for now’. Pairing is an awesome way to do this.
- Functional workflows – inform your team about the inability of users to perform specific actions in your product. It may not quite be as MVP as they originally thought.
- Create a mechanism to ‘talk testing’ and not just about bugs. Often it is hard to talk about testing, the challenges you encounter, the risks, and the health of your product.
- Disassociate yourself from the notion of trying to test everything! It cannot be done nor does it need to be done. Continually have discussions with your team about what are the most important things to test today, tomorrow and as time goes on.
- Instead of simply reporting problems, inform the team about the impacts these problems have. These are much better discussions to have.
Most importantly, don’t sweat it so much! It doesn’t have to be perfect. It is early, expectations are lower, and there should be an understanding amongst the team, the customer and the business that this is an iterative approach to development and delivery. Your team will get there. The customer is getting involved earlier in the process and incomplete or not, this is a way better way of delivering value to your customers.