Test Automation – my definition

This post is an explanation how I see Test Automation. There is nothing new in here, and you probably have read that all somewhere else. I just want to create a reference in case I need to explain myself again.

Over the past few days I wrote some blog post on the test automation pyramid, or rather the band-pass filter. And I wrote about the trust in testing. This lead to some discussion and misunderstanding, hence I decided to write this post.

This post is about my opinion and my take on test automation. Your opinion and experience might differ, and I’m okay with that, because – you know – context matters. You might also prefer different names for the same thing, I still prefer Test Automation.

Disclamier and Differentiation: This article is about test automation, which means tests that have been automated/scripted/programmed to run by themselves without further user/tester interaction until they detect change (see below for more details on this aspect). Using tools, scripts, automation or whatever that assist your testing, go in my understanding under the term that BossBoss Richard Bradshaw coined as “Automation in Testing” or basically as “testing”. This is not part of this article!

In my humble opinion good test automation is providing the following aspects:

Protection of value

Automated tests are implemented to check methods, components, workflows, business processes, etc. for expected outcomes that we see as correct and relevant to our product. This reaches from functionality, design/layout, security, performance or many other quality aspects that we want to protect. We want to protect our product from losing these properties, that’s why we invest in test automation so that we can quickly confirm that the value is still there.

Documentation

Methods or components and especially the code should speak for itself, but sometimes / often it doesn’t. We can’t grasp every aspect that the method should be capable of doing, just by looking at the code. We can help with that, by manifesting abilities of a method or component in automated tests.

Change detection

Test automation is there to detect change. When the code of the product has been touched and an automated test fails, it has detected change. It has detected change of a protected value. This could be a change in functionality, flow, layout, or performance. If that change is okay (desired change) or not (potential issue) has to be decided by a human. Was there a good reason for the change, or do we need to touch the code again to restore the value of our product.

Safety net

The automated tests provide a safety net. You want to be able to change and extend your product, you want to safely refactor code and you want to be sure that you don’t lose existing value. Your automated tests are the safety net to move fast with lower risk. (I don’t say that there is no risk, I just say the risks are reduced with a safety net in place.)
Also maintenance routines like upgrading 3rd party dependencies are protected by the safety net. Because changing a small digit in a dependency file can result in severe changes of code that you rely on.

And this is also where the topic of trust comes back to the table. You want to trust your test automation to protect you, at least to a certain degree!, so that you have a good feeling when touching existing code.

What Test Automation is NOT

Test automation is not there to find new bugs in existing and untouched code. Test automation is not there to uncover risks, evaluate the quality of your product, tell you if you have implemented the right thing, or decide if it can be deployed or delivered.

Test automation cannot decide on usability, or proper design or layout. Automation doesn’t do anything by itself. What it does is enable you to re-execute the implemented tests as often as you want, on whatever environment you want it to run, with as many data as you want. As long as you implemented the tests properly (different story!)

Test automation, when done properly and right, is there to tell you, that all aspects, that you have decided to protect, did not change to the degree of protection you invested in.

In the process of creating automated tests, many more things can happen and can be uncovered. I prefer an exploratory approach to test automation, but that is subject to maybe another blog post. But once the test is in place and ready for regular re-execution, it is there to protect whatever you decided is worth to protect.

Test Automation can never be your only testing strategy. Test Automation is one part of a bigger puzzle.

Advertisement