Reinventing Testers and Testing to prepare for the Future

When I want to speak about the future of testers and testing, I first have to think about the situation we have now, and why I want to change anything about that.

In the past decade or so there were multiple threats to testing and testers. “Test is dead”, “Testers need to learn to code”, of course the ongoing discussion (especially by tool vendors) “all testing can be automated”, and some more.

Then there was a workshop on Orcas Island mid of May 2016, organized by James and Jon Bach. The topic was “Reinvent Testers” and I don’t want to go into any details about the workshop since I was not even there and don’t have more information than what is available on the website. It was not the workshop that created some disturbance in the equilibrium of the community, it was just one slide – out of context – that was spread via Twitter.

The statement that got my attention was “there is an ongoing and longstanding attack on the testing role“. I don’t want to focus on any of the points on the slide, because I don’t think they support the statement in the header in any way. And I don’t want to say anything against the chosen hashtag, which was a typo (testing instead of tester) as far as I know, combined with a side blow on some minor political campaign currently ongoing in the U.S.

Perze Ababa was so kind and shared the slide after that one with some people on a slack channel. And that slide was reflecting in my opinion the actual problem way better. I don’t want to share the slide here, since I was not part of the workshop and I did not ask James and Jon for approval of sharing it here. But I want to quote some statements from the slide that better describe the current situation of how the tester role and testing as a phase are often seen by other participants of the software development life cycle.
“We must eliminate the need for judgement among testers, so that there is no controversy about what is a bug!”, “Nobody smart wants to be a tester!”, “Testers need to assure quality!”, “Testing is too unpredictable, you should instead make a Definition of Done and use Acceptance-Test Based Development to know when you get there!” and “You should write down all your ‘test cases’, tracing them to requirements, and track metrics against them!”.
This is only a small part of the statements on the slide, but I guess most of my readers get the idea of the tone I’m referring to and were themselves more than once in similar confrontations.

How I see the current situation of testers and testing as a phase

When I look at the current situation of testers and the standing of the testing phase in projects in general, I can understand why there seems to be a problem.

The testing phase was born in waterfall-like approaches many decades ago, and testing usually comes in the end, when everything was sort of stable. And that is where many people (testers and non-testers alike) still see the testing phase. Talk to the average factory-tester and he or she would advise to have a stable piece of software available for the testing phase. When you have to run all tests in the end and want to make a statement if all tests passed, that actually makes some sense in that context to not have a moving target and execute your test scripts with 10 or more different versions of the software. Reality might look different.
Being the phase in the end also gives you the disadvantage of biasing opinions that in case of a delay it’s your fault.

Two decades ago development approaches started changing away from waterfall. XP, SCRUM, Kanban, Agile, Lean, whatever the buzzword you prefer, it’s all about getting code faster into production or the hand of the customer. Faster feedback is the driver to develop the right thing and not waste much money on walking into the wrong direction.

The testing phase has a problem in those environments. It did not evolve together with the development approaches. Maybe developers sometimes don’t appreciate the feedback they get from a late testing phase, enforcing them to redo some of their work or getting told that they implemented the wrong thing. From a developer perspective that I experienced and also from a management perspective, testers should only verify that specifications have been implemented correctly. And most of the new development approaches found ways to get exactly that feedback faster without the need of testers. It seems nobody was missing additional feedback and information coming from the testers. Who knows why?

Testers who stayed in such projects simply forgot to adapt and tried to “only” attach a testing phase at the end of whatever development approach was chosen. There was also not much support from the testing industry. iSTQB that initially was taught in waterfall context only came up with a new certificate, the Certified Agile Tester. Not much of a help. ISO29119 is a documentation-heavy framework that hasn’t found an answer for Agile approaches yet. In the beginning of the millennium niches like the context-driven testing community, with a couple of thousand testers world-wide, were simply not big enough to make a difference in an industry with over a million people.

Nowadays testing phase testers have the problem that development approaches have evolved so far, that they cannot keep up any more.

Testers in general have a problem they didn’t cause. Testing was seen as simple task on one hand and was tried to be kept cheap on the other, since it added no value to the product, so many cheap people from various backgrounds were hired into testing jobs. The career opportunities in most companies are usually to step up some ladder and move out of testing positions. So the best testers were simply lost to other disciplines over time, because of the lack of opportunities in their profession. And I also claim that a good portion of people who stay in testing more than 5 years lack motivation to improve or don’t lack laziness. They just stay there because that’s all they think they can do and they have a safe job.

Management is also treating testing still like 2-3 decades ago, because back then it was easy to manage, so why change that. No motivation from that side either.

How Agile & Co. cope with testing

Projects, products and companies that work Agile have dealt with the situation and try to include testing earlier and let the developers do the job. Lots of checks that come up from testing are getting automated to speed up regression testing / checking. They try to keep testing/hardening sprints short and everybody helps there when not busy with fixing bugs. Approaches like BDD, TDD or ATDD support the cooperation between business analysts and developers, and help developers produce more fitting solutions.
Some companies even take it to the extreme and let developers accompany their code into production, see there if it works, and then leave it alone. No testers involved.

Do you think those companies, at least where it works, are crying for a testing phase or more testers on the project? I don’t think so.

In the AB Testing podcast, Alan Page and Brent Jensen regularly share a view on how Microsoft tackles software development processes, and it seems they are doing not too bad.

When claims come up that 90% of the people working as testers should get a new job, James Bach was citing such a request in a podcast I stumbled across last year, I say, may it be only 70% but it’s true. If more and more companies successfully change their development approaches, we will need less testers. And of course I want to keep only the top 30%, not any 30% and not the cheapest 30%.

How the role of testers will evolve

To survive in the world out there, testers have to speed up their game. Testers cannot waste a big chunk of their time with documentation. My motto since being team lead for a small team is, keep people testing. Time spent knee-deep in the system is usually more valuable than writing pages of test cases.

Testing in my opinion is all about providing necessary information for stakeholders to make informed decisions about the project. And those information are collected by actually spending time with the people and the system, not by preparing, updating and grooming long lists of test cases and other useless pieces of documentation.

Testing should be shifted left as far as possible. Regression testing should be reduced to the absolute necessary minimum. As Michael Bolton stated in a discussion yesterday, when you have to do lots of regression testing because the developers don’t understand how changes to the code affect the product, the developers have a problem, not the testers. Those problems need to be addressed differently. Not by blaming that the regression test phase is too expensive (I have a friend, who knows someone, where that might be the case).

Having testers in your team should not be a surrogate to writing and maintaining automated unit and integration tests, because developers are too lazy to write them, or the architecture is not favoring writing them. Then you have a different problem. Throwing testers at the problem won’t help solve it. But it provides someone else to blame.

Testers should take care of the bigger picture, things that are hard to automate, cope with complex problems. Testers should help developers to create good automated scripts, help them to understand and test the application and train them to do the simple tests earlier and faster and provide feedback faster. And yes that needs skills, competence and behaviors of a systems thinker, of a detective, of an explorer, and of a coach. That’s nothing you can buy cheap on the next corner. Those people are rare.

The role of testing needs to be changed?

Some people participating in the discussion call out to testers to take different roles in the project to still provide value. I see that as a wrong approach. As a tester in the role as tester, used in the right situations, I can provide value to the project only I as tester can provide. And I don’t want to give up my role as a tester. I want to continue asking questions, experimenting with the system, analyzing strange problems, I don’t want that to go away.
If my job as tester on my project is not filling out my whole day, or I as an individual decide that I can be more useful and contribute to the project in different other roles that may even improve my (part-time) job as a tester, that is a great opportunity. And I strongly advise you to take that chance. But your role as tester doesn’t change.

I don’t think that the role of testing should be changed by taking on new tasks. Those are simply other roles that can be filled by the same person (specializing generalist comes to mind). I recommend to make better use of the role testing and focus instead. Tasks that are not testing and produce more waste than value should be taken away from the role of testers. Enable testers to act faster!

What testers need to do is evolve or reinvent the way they work, not by taking additional jobs in the project, but finding a way how to produce the necessary information faster, earlier and more efficient. And testers need to dare to hand over simpler testing tasks to developers to stages or phases where it makes more sense.

The testing phase should be changed. If you need an unnecessary long regression test phase, you better rethink your development approach. If you run tests and checks in that phase that take days and weeks, you might want to rethink that approach. If you regularly run into problems in the testing phase you have a problem.

And when Brent Jensen says that data scientists and telemetry are attacking testing (episode 39 of AB Testing podcast), I can only hope that testers position themselves accordingly, stack up their arsenal and be part of those approaches. For me that describes excellence in testing. Monitor and gather feedback from production and use scientific methods to use that data to inform further actions. Fantastic! Where can I sign up? I am a tester, and I want to master those tools.

Responsibility for the overall decrease of quality

Software developing companies have a common problem in the last few years, starting maybe over a decade ago: Product quality gets worse and worse. The possibilities of faster cycles to production makes many companies negligent about actual quality, because it’s easy to provide a hotfix. Only the rules of the market help to filter out the worst of them and also to prevent the ones with good quality from the start to get a chance in the market, because they tend to be too slower.

I would see that partially as a problem of neglecting testing phases, doing the wrong testing, doing testing wrong, or hiring the wrong people to do the testing.

Testers need to fight and find their way back into those projects and provide value to help the team raise the level of quality again.

[Update: The claims offered are my personal views, and I go into more details here.]

What companies need to do

Companies need to decide whether they need or want to stay in waterfall-like approaches, having that big testing phase at the end, and taking all that comes with it into account. Waterfall is not bad, and Agile is not for everybody. And always remember: A good waterfall is much better than a bad Agile-ish attempt.
Some may even have to bite the bullet and outsource their testing to some expensive, documentation-overload-producing consulting company. There might be good reasons for that.

One hint: You might want to rethink your approach if you are in regulated environments and take it as excuse to say you need a documentation-heavy testing process. I have heard from multiple people that this is just a myth.

Invest in the future: Prepare your people to fill that high sophisticated role as tester. Find those who are willing to invest in their craft and profession. Spend time and money on training and conferences, let them interact with like-minded people, get them engaged, get them motivated, and pay them a decent salary to have them focused on the job and not on how to pay the bills next month.
And if you ask what happens if you train the people and they leave, better ask what happens when you don’t train them and they stay!

What testers need to do

Managers also need to change their view about testing and testers. But it’s the task of the testers to build trust in the information they provide and demonstrate the value they can add beyond test scripts and developers testing themselves. Then managers will hopefully change their opinion that testers only need to provide a count of test cases to show their value.

Do everything you need to get better in your craft. Learn tools, methods, approaches. Get better with soft skills like communication, critical and creative thinking, problem solving. Stay up-to-date. Interact with your peers in- and outside your company. Go to local meetups. Read books, blogs. Go on Twitter and engage with other testers. Learn from them where to find more sources for information.

If you want to be in that 10-30% that should stay in the profession, you better start now.

And my last tip: Be proud to be a tester!



15 thoughts on “Reinventing Testers and Testing to prepare for the Future”

  1. Great post, I wholeheartedly agree. As testers we need to build trust, invest in ourselves and demonstrate the enormous value that we can bring in many different ways to a project. We are lucky to have fantastic people in the testing community sharing ideas and inspiring others to do so. This can only assist in our quest to adapt and evolve the testing role.

  2. Do I need to read all this.? Something kept me going and I am happy to go through the whole post a couple of times. Many valid points and insights.
    Good article .

  3. I disagree when you say that the points on that slide don’t match the header\title. However since that’s not really the main point of your post, I’ll move on.

    Instead I want to ask about this perceived future where only 10-30% of testers remain in the field. Not that I’d mind particularly if that happens.. Let those who have allowed themselves to become replaceable with tools be replaced by tools. However I still see increasing demand for non-thinking testers, and I’ve been waiting for that to change for many years now. Please, give me hope, where are you seeing a decline in demand for this kind of awful “testing”?

    1. Hi Kim,
      thanks for your comment.
      On the one hand my hope is coming from the idea that when development is more and more “converting” to modern approaches, where developers handle much of the early testing themselves, where systems are designed in a way that regression is more unlikely or easier to discover by automation. When development teams move to faster approaches to deliver software, better and faster testers are needed and not as many as before, in phases that don’t come in the end. My hope is that the demand will be reduced.

      On the other hand that statement also implies that 70-90% are unfit for the profession and don’t do something against it. My claim implies that people are trying to fill a position they shouldn’t have from the beginning. I agree partially your claim that more testers are needed, because those we have don’t do the job right.

      If you say there is a demand for more “non-thinking” testers, I’d assume that this demand comes from parts of the testing industry that still didn’t get the memo that the world is changing and moving in a different direction. And I hope (clearly only hope, I see no evidence or anything that it actually happens) that the force of the market will proof those concepts wrong.
      It might even end in a split of testing society where one group will move on and rename their role to not be brought into connection with the other.

      Predictions are tough, especially for the future.

  4. Thank you Patrick for writing this! I will share it on various channels and I hope that many people will read this not only testers!
    Have a great day!

  5. As a remotely located test manager, I have daily video meetings with our testers and developers, most of these meetings are about understanding the progress of the software development. With this blog you made me realize that I have to focus much more on the progress of my testers personal development. I usually focus on these things when I’m on site, but as of today, I will be asking the additional questions: “What have you learned yesterday?” “What do you plan on learning today?”

  6. Hey Patrick,
    Thanks for this really good post once again.
    Even though I don’t agree with the statement of software quality has decreased over the last years. If so why are we using more and more of it in our everyday lives?
    Anyway I think your point about having less testers is quite important and my take on it is that we must communication better what we are doing and being able to do and what not. Personally I feel this QA title is preventing people from understanding it better.
    This misconception of what our work is really about creates a lot of bad testers. I guess the same is a bit true for PMs.
    What we do is unfortunately not always to easy to explain but therefore very exciting 🙂

  7. Thanks for a thought-provoking post. I need more time to re-read and digest it.

    But I do want to say, your description of what happened to testers and testing in agile teams is certainly true in a lot of situations, but by no means universal. I joined my first XP team in 2000 because Kent Beck’s _XP Explained_ book is ALL about quality and testing – even though he and the other XP proponents at the time didn’t think they needed professional testers.

    At the same time that Ron Jeffries was telling me that everyone on an XP team ought to be writing production code, he and Ward Cunningham, Bob Martin and Kent Beck were all having wonderful conversations with me and others about how to address various testing problems in XP teams. How to write effective acceptance tests? How can the dev team collaborate better with the customer team? Yes, I had to use my elbows sometime, but I was warmly welcomed into that XP community. All those people encouraged me to write my first book (with Tip House), _Testing XP_. That was published in 2001.

    So I feel sad when a bleak picture is painted. I’ve gone to agile conferences every year for 15 years, and I talk to a lot of people on teams around the world. Janet and I have 70 stories from 40 different testing practitioners on agile teams around the world in our latest book and had more in our first book. Over the years agile teams (including my own) have woken up to the fact that they need exploratory testing, they need to consider all quality attributes including performance, security, usability etc. They have embraced testers and testing. And what’s especially cool is that I know so many test-obsessed programmers, sys admins, data experts, designers, product owners.

    To give one more example, Elisabeth Hendrickson’s book _Explore It!_ is popular among many agile teams I know, and more and more of them understand the value of exploratory testing. The devs on my own team are always eager to improve their ET skills. I know of many other teams where that is true.

    I agree that we all need to keep learning, we all need to keep leading – show the value of what we contribute. But, please don’t tar all agile teams with the brush of not valuing testers and testing. Just keep helping the ones who haven’t discovered that value yet.

  8. Have you thought about mixed roles? For me it’s a myth that e.g. regression testing and other not so nice things can be removed. Sure, it all depends. If you can patch a web app in minutes, you might be OK with more regression bugs. But that doesn’t work many business models. And it’s a myth that regression bugs go down to an acceptable rate, if the devs just think harder.
    I think having testers in early stages is nice, providing holistic feedback is nice, but in the end testing is about finding bugs.
    If you think that testers are bored after some years and quit, I don’t think you should broaden the term of testing, but let team members work in different areas.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: