State of Testing Survey 2017

It’s this time of the year again.
PractiTest and TeaTime with Testers are again collecting information about the current state of the testing industry and the trends going on and coming up. They are doing this now for the fourth time.

The survey doesn’t take long, and they are looking for feedback from all levels of the industry. From newbie to senior. And of course from all corners of the Earth.

The result is a well-crafted analysis comparing the current state with the past years to see trends in testing evolve. (see last years report) Last year the survey had over 1000 participants, and we need more to get an even better picture.

So please, take some minutes of your valuable time and participate in the State of Testing survey 2017. And if you can, please also share the link with your colleagues and friends in the industry.

Take the survey here!

Thanks for your help


Network Console for your test scripts

As exploratory tester I love my browsers’ dev tools. Especially since I’m currently working on a CMS project, analyzing the available elements and their styles is what I’m doing every day.

When it comes to automating, Selenium Webdriver is capable of locating the elements of the DOM tree, interacting with them and even reading out there styles for verification and many other actions. But there is a tool in the toolbox that can also come in handy in automation that Selenium doesn’t cover (afaik): the network console! Selenium itself cannot capture what is going on when the browser captures its often 100+ files for one site from the web. The Webdriver “only” deals with the results.

For writing a simple check script I looked into this “problem” and found a very easy solution for Python that I want to show you. It’s by using the Browsermob Proxy. Just download and unzip the proxy to a folder reachable by your automation script and start coding.

Firing up the proxy:
# Start Proxy Server
from browsermobproxy import Server

server = Server("../browsermob-proxy-2.1.2/bin/browsermob-proxy")
proxy = server.create_proxy()

print("Proxy-Port: {}".format(proxy.port))

# Start Webdriver
from selenium import webdriver
co = webdriver.ChromeOptions()
co.add_argument('--proxy-server={host}:{port}'.format(host='localhost', port=proxy.port))

driver = webdriver.Chrome(executable_path = "../drivers/chromedriver", chrome_options=co)

Now you have a running Webdriver that can collect information in HAR format. Let’s call a website and record the network traffic.

# Create HAR and get website

Now you have all the information that usually your browser’s network console provides in this HAR entity. So let’s find out, which files get loaded, how fast that was, and how big the files are.

# Analyze traffic by e.g. URL, time and size
for ent in proxy.har['log']['entries']:
    print("{} ms".format(ent['time']))
    print("{} kB".format(round((ent['response']['bodySize'] + ent['response']['headersSize'])/1024, 2)))

Don’t forget to clean up after you.
# Shut down Proxy and Webdriver

The output is now a long list of entries looking something like this:
1103 ms
129.46 kB

Now let your imagination run and think of what you could track and analyze for your project with that simple tool? Maybe a basic performance monitoring?

If you come up with something cool, let us all know by adding it to the comments here. Thanks!

The code used here is using Python 3.5 and the latest Chrome webdriver as of Dec. 30, 2016.

2016 – What a year…

2016 was a year with many highlights for me. And for those who know me a bit longer, the years before had way more lowlights than highlights, so 2016 really was a change for the better.

It started with the intention to finally look for a new job. And in March I had the first contact with a small test consulting company called QualityMinds in Munich, with a lot more to follow.

End of March I was speaking for the first time at a conference. And what a conference it was. TestBash Brighton! In front of nearly 300 people in the Corn Exchange was a fantastic experience.
Great thing was also that my wife and daughter joined me afterwards and we enjoyed a wonderful week in England.

In May I held my first conference workshop at Let’s Test in Runö, Sweden. Also it was the first time I volunteered as facilitator, which was a good experience as well. And like the year before it was a place where I had the chance to meet some of the folks I met online for the first time.

The day before I went to Sweden I handed in my notice. I decided I had suffered enough.

On the off-testing side of my life my artisan work in 2016 was nearly non-existent as my main focus was on re-building the interior of the attic. The final touches were made end of August, just before starting at my new place. On my birthday in September my daughter started school, which started a whole new chapter for the family as well.

Back to testing. On September 1st I started at my new company, QualityMinds, a place that hired me for being me, as I found out only later. I’m part of a great team of engaged people; to be honest, for the first time in my work life.
After 6 days I also started my first project, and in a real SCRUM environment. To the day I’m still shell-shocked that SCRUM can actually work. After experiencing lots of suffering with trying to introduce a bit more agile ways of working at my last place, I actually saw SCRUM in action. What a wonderful way of working, and finally I understood lots of blogs and tweets from people working in such an environment. It is possible. But my experience also told me that not every place is ready for such a way of working. But I love it.

Thanks to fabulous Danny Dainton I was back at a TestBash in October, this time in Manchester and as an attendee, enjoying the show for 2.5 days. That’s also where we (Kristine and me, together with our team of Vera, Marcel and Daniel (in absence)) finally laid the foundation for TestBash Germany, which was publicly announced in December!

In November I was at my first client workshop with my boss, which was an interesting experience. And I’m looking forward to more experiences of that kind.

In 2016 so much happened, and it feels so much longer than one year. By now the first 8 months of the year at my last place are nearly forgotten and so much great happened that I can honestly say, 2016 was a really good year for me!

What will 2017 bring.

Well, my crystal ball is still in the repair shop, so I assume that 2017 will be busy. As my project contract has been extended, means I stay in this project for at least a few more months.

The year starts with the Dutch double-feature TestBash NL and DEWT #7 end of January. In March I’ll be at TestBash Brighton to learn more about the organization of a TestBash hands-on.

My QualityMinds team has set a couple of interesting goals, that will keep us also pretty busy besides our project work. And thanks to our wonderful learning coach Vera I’ll be busy on the QualityLearning side as well. Both helping colleagues to improve and learning lots myself.

The most part of the rest of the year will be busy with preparing, organizing and advertising the first ever TestBash Germany in my hometown Munich on October 6th, 2017.

I also want to work on a few topics to submit for conferences later in 2017 or early 2018.

In case I have some time to spare, I also volunteered for now three friends from the testing community to help review their books in progress, which I’m very much looking forward. And I wish all three of you the best of success in your writing efforts.

And then let’s see what else might come my way. In the end I am called Agile Tester, so let’s be agile!

Shift X – Why are testers not happy where they are (put)?

My statement: “Testers don’t seem to be happy where they are (put) in the software development life cycle (SDLC).”

After TestBash Manchester there was an Open Space on October, 22nd. The topic I choose to put on the schedule was “Shift X – Going up- or downstream?” My motivation to choose that topic was that in two talks during the conference we heard – again – about testers on the move.

There is (sadly still) the widely spread opinion and teaching that the testing performed by testers happens when development is done to ensure conformance to requirements before software goes to the customer for acceptance or into production. Testing as an activity is too often underestimated and hard-wired to performing test cases on a finished piece of code. And testers world-wide are trying to leave their profession behind to advance in other roles in the SDLC.

In Manchester I was able to discuss this topic with Duncan Nisbet, Ash Winter, Del Dewar, and Matthew Bretten.

My short summary of the discussion is, “we want to provide more value!”

We split the topic in three areas “Shift Left”, “Shift Right”, and “Where are we now”.

“Where are we now?”

As testers, waiting for devs to finish their work, we might have been able to review or at least read the requirements and prepare our test cases to cover the requirements’ functionality, reporting our job done with pass/fail rates. Do we have influence on the requirements? We can be lucky if we are co-located with the team and not sitting in our silo all by ourselves, not speaking of being in a different country and time-zone. Often we have minimal feedback from production and with that no information about the added value of the feature we just tested. Though we more often than not get the bug reports from production. But does the fact of no bug reports from production for a feature automatically explain that the feature was adding value?

For us it it was relatively clear, that in the position the classical tester is set, the amount of value they can provide is not too much, at least not enough. And what happens when mankind is not happy where they are? They go on the move.

Shift Left

Like the salmons in Alaska, as Duncan showed us the day before, there is a lot testers can provide value with when moving upstream in the SDLC. Our points were to help get a shared understanding of the upcoming stories and start the learning journey earlier. The earlier we provide valuable input for others, the bigger the impact. One rather explicit point was to also start tasks like performance testing earlier to uncover trends in performance before it’s too expensive to change.
There is an urge to show that testing is a valued function and help the team to perform it in the most helpful way.

Shift Right

In approaches to deliver stories faster we enter the DevOps world, where everything is optimized to fast and safe deliveries, where information from production is highly valued. Testers can help to analyze log files, signals and trends on production, to understand better how the product is actually used and deliver these information to decision makers throughout the team.

Shift Out

This approach we didn’t discuss in Manchester, but it’s what we often see in our industry. Testers are looking for ways to provide more value in software development. All too often they move to positions like project management, business analysis, development, and so on. Too often we (the testing profession) loose a good tester that way, sometimes we get a good project manager, analyst or developer in exchange who knows about the benefit of good testing.


There are many places in the SDLC where good testing can increase value, benefit and quality of information. We, who know that, need to raise the awareness of that fact with testers and all other project participants alike. We need to show testers how they can deliver better value, and get more influence in the development of a project or product. This can happen all the way from an idea (or “desirement” as Duncan called it) to observing in production. And we need to show business analysts, architects, developers, project managers, operations folks, and management, that testers can have the necessary abilities to support all throughout the SDLC, not only in that one spot. That way companies and projects can also provide valuable positions for testers to grow and advance their skills, providing value, being accepted as partners by all, and not seeing the move to other positions as only way to advance.

The five of us, who sat together in Manchester, are in projects and positions where the benefit of testers is seen and accepted. We know that it’s possible! But we also know many examples from our own experience where testers are locked up in the SDLC at the place where we didn’t want to stay.

Other sources on the topic:

Jesper Ottosen wrote only lately about the ways to shift:

TestBash – more than a conference

If you don’t live under a rock, you know it was TestBash time again. For the first time in Manchester, Richard‘s home town. And north-west England was hungry for a conference like this. The event sold out way in advance and the majority of participants was from the area.

What brought me to Manchester was the fact that the amazing Danny Dainton asked me if I wanted his ticket, as he found out that by the time of the conference he will be father of a little daughter and wanted stay around his family. Danny I promise you that this will be the last time, but THANK YOU my friend! And that gesture already describes a big component of TestBash: sharing, caring, helping, and many more, I assume you know what I mean.

TestBash is also a relaxing conference. For speakers and attendees likewise. You don’t have to worry about missing something, it’s single track. And it is always a track of really cool topics and speakers. Speakers don’t have to worry about “how many people will show up” and nobody needs to run around, searching for the next room. If you meet someone in the breaks, just ask him about the last talk, you can be sure they saw the same talk.

There we come to the next thing, TestBash is about conferring and meeting people. For me it was again a mixture of meeting old friends, meeting folks I interact with via social media and finally meet in person, and meeting new people. I love that mixture. If my introvert side becomes stronger I can mingle around old friends, if my extrovert side reigns I will meet new people. Alongside the conference there are pre- and post-TestBash-meetups where people can spend more time with each other. Because time is crucial. There are more people too meet than time available, as always.
TestBash attendees are from different backgrounds, specializations, experience, age and everything. You get a lot of fresh views when talking to people there.

And there are the 99 second talks. The chance for every attendee to go on stage and talk about a topic of their choice for 99 seconds.

If I have to summarize TestBash Manchester in a few words. It was about the importance of learning, listening, being positive, yet still critical and ways to provide more value. The videos will come soon to the Ministry of Testing Dojo. It’s worth watching! You find some photos beneath the post.

The open space on Saturday presented a forum to talk about everyone’s topics and discuss it with others to get new insights, answers, questions, directions or guidance. And we had lots of good topics and a wide mixture of people to interact with. The office of was a fantastic location high above the city (11th and 12th floor) and provided lots of openness and room to discuss, mingle, and confer. And Rhian from was a fantastic and caring hostess. Thank you!

TestBash, it was fantastic to be part of you again this year. Rosie and Richard, great job! You are amazing.

Master of Ceremony: Vernon!

James Bach on “Managing Critical and Social Distance”

Iain Bright on the “Psychology of Asking Questions”

Kim Knup “On Positivity: Turning the frown upside down”

Stephen Mounsey on “Listening: an essential skill for software testers”

Duncan Nisbet on “Be More Salmon!”

Helena Jeret-Mäe and Joep Schuurkes on “The 4-hour Tester Experiment”

Mark Winteringham on “The Deadly Sins of Acceptance Scenarios”

Huib Schoots on “A Road to Awesomeness”

High Energy: Gwen Diagram on “Is Test Causing Your Live Problems?”

Beren Van Daele on “Getting The Message Across”

Open Space

The Agenda for Open Space

The add-on agenda


My community – quo vadis?

The basic intention behind this article was slumbering in my head for a while, I use the current emotions, to finally write it.

I am following the context-driven testing community now for about 4 years. Still I am not sure if I am part of the community itself, or as the question was changed during the last year, do I want to be a member of that community?

When I first found the context-driven testing community, I got answers to so many questions I collected during my first 10 years in the testing industry. I finally felt understood, or better I found people who favored similar approaches and were even able to express them in an understandable way. I found lots and lots of like-minded people, many of them I even consider as friends now, a handful or two even as good friends.
I began reading blogs and magazine articles and books recommended by the community in amounts never experienced before in my life. I would assume that I read more words in the last 4 years than in the 35 years before that. The community is a fantastic source for discussions, sharing information, insights and wisdom among all who are interested.

Through the CDT community I found my way into testing conferences and got the courage to stand up and speak. And from what I saw so far from conferences driven by CDT people and a few other testing conferences I visited or followed, there is a difference. The conferences closer to CDT excel in their range of topics by people sharing thoughts and ideas and personal stories. The “other” conferences often share success stories or marketing talks. There are multiple reasons for the existence of both. For me personally, the talks about thoughts and ideas help me more than the stories of what worked for others. And the personal stories tell me more about the people I interact with.

As many might already know, I found my way into the community after a EuroSTAR webinar by Michael Bolton in September 2012. And shortly after that I discovered James Bach. And those two became a main source for information, insights, approaches and points of view on so many topics, especially in the beginning when I was seeking for answers. But then I discovered so many others who were willing to share their knowledge and the portion of Michael’s and James’ influence on me became smaller and smaller.

I want to introduce a SCRUM analogy here. In general we are all part of a big team of equals with no hierarchy. But as in every team of equals you sometimes, maybe in tough situations, seek guidance through a leader. In my opinion, this is the reason for Michael and James standing out over the equals. They are two people who did so many good things and were willing to share so much of their time to help others. So if members of the community seek for guidance that’s of course one way to look.

From afar some seem to  assume that the CDT community is a group of disciples of James and Michael. Maybe even a sort of religion or a cult around them. And the community is often treated as such from “outsiders”. Maybe some members of CDT even see themselves as such, that’s their own thing. And, if something new is shared by James and Michael or approved by them, people read the words and spread the news. Challenges rarely happen. And I wonder why, because that is one thing I got taught by them. Challenge things! That’s one of the key drivers behind good testing. We accept too much as is every day. The tasks of a good tester is to challenge things and that also applies to things told about testing. (And yes, you can challenge the thought if challenge is a key driver of testing! As Michael often did with me, I invite you to do so.)

But then there are challenges from people, from outside the community and even inside. Which in general is good. And as James and Michael often emphasize, debate brings topics forward. Only, the tone often got extremely rough this year. There were several incidents around James and Michael, that I don’t want to list here, as most people will be aware of them, and also from others, where basic rules of interacting with people were ignored. There were also several harsh attacks from outside the community on the community itself, some on James and Michael directly. People were requested to take a side or stand against someone. Anyone arguing in favor was discredited at once. It seemed more like personal warfare.

The other day I woke up to a community that was asking for a code of conduct for conference speakers. Really? Why is that even necessary. Yes, I see, there is James who sometimes makes such a set of rules necessary. But do we really need a written set of rules for behaving like grown-ups and interact in a polite and civilized manner? Come on! There should hopefully be better ways to deal with that topic.

I don’t want to defend James behavior in any way, I want to distance myself from it. It was unacceptable. Still I highly value his addition to the body of knowledge to the community. But if he can’t behave on stage, maybe he should not be invited on stage by organizers. As it should happen for any other person who misbehaves on stage.

I spend much time every day with testing, thinking about testing, reading and sometimes even writing about testing. I don’t want to waste that time with bullying and rudeness, sadly the internet is already full of it. And until last year I thought that my community is free from such bullshit. I hope that people will stand up for their standards of politeness in interacting with people. In the end, a community is a group of like-minded people, and in the case of CDT, a group of people who believe in testing and want to bring that profession forward.

And while we are at it, people who call themselves context-driven should be aware of what that actually means, because often I observe people who argue with techniques and approaches generally accepted in the CDT world as best or even only choice to get the work done. That is context-imperial and is no quota better than consultants or companies who argue in favor of techniques or approaches as the only weapon of choice that are often chosen as bad examples in CDT discussions. If you are truly context-driven, don’t take it as a label to use approach X in every context, understand your context, know as many approaches out there, understand your stakeholders, your limitations and so on (context, eh), and then apply what you think fits best in your context. And if, for example, the context seems to favor test cases, and you hate test cases, you have three choices. Either leave, or find a method that really works better, or live with it and use test cases and do the best possible to get your work done professionally and help the project.

I for now will continue to build my own community, my collection of people I trust and like, my sources of information, recommendations, insights, wisdom and friendship. I will continue to share my thoughts with anyone who is willing to read it, I will take my time to discuss with anyone, who wants to question my statements. I will continue to reach out into other communities, as many useful things can be found there as well. I want to get shit done, and I want to be part of a testing profession that can help to build the future.
And I hope that I find the courage to stand up against bullying.

[Update] I want my community to be a safe place for women. I have learned so many things from the amazing women of the testing community in my career. A lot of my biggest break-throughs in my thoughts were triggered, motivated, intrigued by fantastic women. And I was always happy to be in a community with big female involvement, at least compared to some other technical disciplines. And by the way, the same is true for any other group. I don’t care about gender, sexuality, religion, color of skin, culture, age, or the style of music you listen to. We share a common ground that is in my eyes bigger than all possible gaps out there. We share the interest in testing, bringing forward the profession, learning about it, getting better at it. The interest in sharing and gaining knowledge. Testing has such a big human element, and from every influence we can only learn more and more, and get better and better. So I want my community to be a safe place. Challenges, debates, and discussions are welcome. You don’t have to agree on everything. But harassment, embarrassment, blame and alike have no place here.

Thanks to Lisa and Lanette for reminding me of this important fact that I have forgotten in the initial write-up. [/Update]

As my good friend Damian wrote, “labels can be helpful, and labels can be harmful.” (paraphrased) In general I don’t want to be labeled and put into a box. If there is one label/box I proudly accept, it’s “passionate”.

Thank you for your attention!


One very personal thing, I want to add here in that context:

Yes, I admit, I have a favorite person whom I was sort of attacking a few times over the past years, Rex Black. In general I have two arguments with him, that come up every now and then. 1) I don’t like the way the iSTQB certificate exam works. Partially driven by my disliking of the iSTQB syllabus, that many find useful, but I see some parts of it more harmful than useful for the wider testing community. 2) Yes, that’s a CDT topic, I don’t like the term “best practice”. Rex does. So we often argue about a freaking label, that for me is the personification of imprecision in management bullshit bingo lists. I know what he wants to say in favor of it and partially even understand it, but it usually only hits a nerve with me.
After some reflections last year, I stopped attacking Rex, because I found my own behavior was more than wrong. I even tried to interact on a normal level a few times with him, even if he tried stabbing again (might have been a reflex and I don’t take offence on it, based on our personal history). And it made me very happy when he answered to my first day of school tweet.

Rex, I hope we meet one day in person and can discuss about the topics face to face over a beer.

TestPappy on “tacit knowledge”

TL;DR: This article only reflects the highlights of a discussion on tacit knowledge, it will not describe what tacit knowledge is. My key takeaway was, based on the exchanges in this twitter conversation, there didn’t appear to be a shared understanding of ‘tacit knowledge’ by all participants of the discussion.

The concept of tacit and explicit knowledge is an important one to software testing. Testing usually is a highly brain engaging discipline. And making things explicit is part of our every day. Writing test scripts, writing documentation to share knowledge, reporting on our testing, etc. But what about the tacit part, that often stays tacit?

In some discussion happening lately on Twitter there was a disagreement on the nature of tacit knowledge. I thought I have a certain understanding of what tacit knowledge is, but a personal discussion with a good friend left me not so sure anymore. There is explicit knowledge, knowledge that has been spoken, written, or somehow made available for others to consume without interacting with the initial knowledge owner. Then there is knowledge that is not explicit. You might call it “tacit” knowledge. So, a question that came up was, is tacit knowledge “only” very hard to describe or “simply” impossible to describe? Or is it “just” something that was not made explicit yet? That’s what I was trying to learn.

I posted a badly written question with a survey on Twitter the next morning:

And it triggered some interesting discussions. I will only quote few examples from the threads out of context, so you rather check them on Twitter yourself, if you are curious.
This will not be my definition of tacit knowledge, I will only summarize some insights and highlights I got from the discussions.

First of all I quickly realized that my question was badly explained and too ambiguous. I got the impression that several people thought that I don’t understand the principle of tacit knowledge. But that helped to trigger some further interesting thoughts.

Even if Wikipedia partially describes tacit knowledge as impossible to describe, alternating with “just” very hard to describe, most people participating in the survey agreed rather on “very hard to describe”.

The first (and longest) example used was learning how to ride a bicycle. Actually most of the examples were very technically oriented. The discussion also changed between “teaching” a robot to ride a bike and teaching a person to ride a bike.

For Stephen Blower it seemed to be very important that the learning from explicit knowledge is successful on the 1st try.
When I remember how I learned how to ride a bike, I remember a lot of failed attempts to do so. So is explicit knowledge sometimes held to different standards than tacit knowledge?
But even if it is on the 10th try means that it is possible, and maybe only hard to describe or wasn’t described yet.

And there was no agreement on the bicycle example being really a good example. Which for me showed an important thing. Several people that I all highly respect, especially for their deep understanding of many test related topics, were not able to agree on a “simple” example of tacit knowledge.

Several times I got the tip to read Harry Collins’ book “Explicit and tacit knowledge“. But is that just an excuse, because people are not able to explain it in their own words?
For me that raised also the question, is that the only truth about tacit knowledge or are there other explanations existing as well? Maybe contradicting Collins?

I was not happy to get answers with a reference to a book.

A new example came on the scene, using cooking, as in following a recipe (that in my opinion presumes a lot tacit knowledge) or “simply” cooking based on what you know (your tacit knowledge).

Now that is a position that was completely new to me. “An experience long forgotten.”

But Mark Federman also came up with alternative reading material on the topic. The concept of “Ba” is very interesting on how knowledge can be managed within organizations and companies. It also describes the spiraling lifecycle how tacit knowledge becomes explicit, to then become tacit knowledge again for other people. It is though not talking about if tacit knowledge is hard or impossible to make explicit. It simply skips that part and assumes that knowledge is transferable from individuals to groups and organizations.

One aspect that makes a good example for tacit knowledge in my opinion are emotions. Mohinder mentioned them shortly, but nobody picked it up from there.

Emotions are actually very hard, if not impossible to describe, if you can’t rely on comparison or the other person having had the very same experience. But is that even possible? Is naming the emotion just shallow agreement? Do both really feel the same?
Are emotions an example of something that is impossible to describe in a sufficient way? Or do we tacitly agree on a more or less shallow understanding to make life not too complicated?

In the learning to ride a bike thread, John Stevenson tried to shift the aspect away from the actual riding a bike, but to ride a bike in cultural context.

One aspect that was important for me, some things are maybe possible to make explicit, but they are not worth it.

My summary: Tacit knowledge seems to be of tacit nature by itself. Very hard to describe, though some, e.g. Collins, have done it. At least to a degree that satisfied many readers. What I can observe on Twitter or in blog posts is, that people boldly use the term tacit knowledge as if it has a well-known meaning. But my impression is rather that it’s shallow agreement. Most people I know are aware of the concept of tacit and explicit knowledge, some read Collins, some read summaries, some read other interpretations, some learned from someone. But my short, and malformed, question and the following discussion showed me clearly, that it seems not be that clear of what it actually is.

I, at least, have learned a lot about tacit knowledge in the course of the discussion. And I want to thank everyone involved for their input.