Month: June 2016

TestPappy on “tacit knowledge”

TL;DR: This article only reflects the highlights of a discussion on tacit knowledge, it will not describe what tacit knowledge is. My key takeaway was, based on the exchanges in this twitter conversation, there didn’t appear to be a shared understanding of ‘tacit knowledge’ by all participants of the discussion.

The concept of tacit and explicit knowledge is an important one to software testing. Testing usually is a highly brain engaging discipline. And making things explicit is part of our every day. Writing test scripts, writing documentation to share knowledge, reporting on our testing, etc. But what about the tacit part, that often stays tacit?

In some discussion happening lately on Twitter there was a disagreement on the nature of tacit knowledge. I thought I have a certain understanding of what tacit knowledge is, but a personal discussion with a good friend left me not so sure anymore. There is explicit knowledge, knowledge that has been spoken, written, or somehow made available for others to consume without interacting with the initial knowledge owner. Then there is knowledge that is not explicit. You might call it “tacit” knowledge. So, a question that came up was, is tacit knowledge “only” very hard to describe or “simply” impossible to describe? Or is it “just” something that was not made explicit yet? That’s what I was trying to learn.

I posted a badly written question with a survey on Twitter the next morning:

And it triggered some interesting discussions. I will only quote few examples from the threads out of context, so you rather check them on Twitter yourself, if you are curious.
This will not be my definition of tacit knowledge, I will only summarize some insights and highlights I got from the discussions.

First of all I quickly realized that my question was badly explained and too ambiguous. I got the impression that several people thought that I don’t understand the principle of tacit knowledge. But that helped to trigger some further interesting thoughts.

Even if Wikipedia partially describes tacit knowledge as impossible to describe, alternating with “just” very hard to describe, most people participating in the survey agreed rather on “very hard to describe”.

The first (and longest) example used was learning how to ride a bicycle. Actually most of the examples were very technically oriented. The discussion also changed between “teaching” a robot to ride a bike and teaching a person to ride a bike.

For Stephen Blower it seemed to be very important that the learning from explicit knowledge is successful on the 1st try.
When I remember how I learned how to ride a bike, I remember a lot of failed attempts to do so. So is explicit knowledge sometimes held to different standards than tacit knowledge?
But even if it is on the 10th try means that it is possible, and maybe only hard to describe or wasn’t described yet.

And there was no agreement on the bicycle example being really a good example. Which for me showed an important thing. Several people that I all highly respect, especially for their deep understanding of many test related topics, were not able to agree on a “simple” example of tacit knowledge.

Several times I got the tip to read Harry Collins’ book “Explicit and tacit knowledge“. But is that just an excuse, because people are not able to explain it in their own words?
For me that raised also the question, is that the only truth about tacit knowledge or are there other explanations existing as well? Maybe contradicting Collins?

I was not happy to get answers with a reference to a book.

A new example came on the scene, using cooking, as in following a recipe (that in my opinion presumes a lot tacit knowledge) or “simply” cooking based on what you know (your tacit knowledge).

Now that is a position that was completely new to me. “An experience long forgotten.”

But Mark Federman also came up with alternative reading material on the topic. The concept of “Ba” is very interesting on how knowledge can be managed within organizations and companies. It also describes the spiraling lifecycle how tacit knowledge becomes explicit, to then become tacit knowledge again for other people. It is though not talking about if tacit knowledge is hard or impossible to make explicit. It simply skips that part and assumes that knowledge is transferable from individuals to groups and organizations.

One aspect that makes a good example for tacit knowledge in my opinion are emotions. Mohinder mentioned them shortly, but nobody picked it up from there.

Emotions are actually very hard, if not impossible to describe, if you can’t rely on comparison or the other person having had the very same experience. But is that even possible? Is naming the emotion just shallow agreement? Do both really feel the same?
Are emotions an example of something that is impossible to describe in a sufficient way? Or do we tacitly agree on a more or less shallow understanding to make life not too complicated?

In the learning to ride a bike thread, John Stevenson tried to shift the aspect away from the actual riding a bike, but to ride a bike in cultural context.

One aspect that was important for me, some things are maybe possible to make explicit, but they are not worth it.

My summary: Tacit knowledge seems to be of tacit nature by itself. Very hard to describe, though some, e.g. Collins, have done it. At least to a degree that satisfied many readers. What I can observe on Twitter or in blog posts is, that people boldly use the term tacit knowledge as if it has a well-known meaning. But my impression is rather that it’s shallow agreement. Most people I know are aware of the concept of tacit and explicit knowledge, some read Collins, some read summaries, some read other interpretations, some learned from someone. But my short, and malformed, question and the following discussion showed me clearly, that it seems not be that clear of what it actually is.

I, at least, have learned a lot about tacit knowledge in the course of the discussion. And I want to thank everyone involved for their input.

 

Advertisement

Is software quality really getting worse?

I made a big fault in my last blog post. I claimed that software quality is getting worse in the past decade or two without giving any facts to undermine that claim. And I’m very sorry for posting a personal perception like that.

I still think that software quality is not getting better. And I want to give you the points why I think that is.

What is quality?

Quality is value to someone (who matters).
– Jerry Weinberg & add on by James Bach

Quality as I would describe it better consists of a bunch of ilities (like Capability, Reliability, Usability, etc). ISO 25010 counts about 30, the famous poster from Rikard Edgren & Co. contains over 50. There are many factors that add to the perceived value of a product. Maybe mine differ so much from yours that your opinion on the same fact is completely opposite to mine.

Quality of people producing software

I came into professional IT in the year 2000. In the years of the boom. CS in universities was the subject to take. People sat on the floor or had to stay outside in crowded class rooms. There was a huge demand in IT people and universities were not fast enough in producing new talents. So companies tried to attract people from other areas into IT. I remember starting in my big test project in 2003 with over 70 people in my team. There were only two people with formal CS degrees and me with vocational training as developer. The rest of the team came from various backgrounds. I don’t want to say that those folks were bad, not at all. Some of the best testers I worked with so far were among them, having diplomas in philosophy, meteorology or similar. But definitely not all were good or should have pursued a career in IT. But we needed the people, so they stayed.

With my current job I got the responsibility of hiring people for my team myself. My amount of experience is not big at all, and maybe other reasons influence the perception. In the last three years I have seen about 150 CVs and interviewed 40+ people for 2 job openings in the past years. And the ~150 were only those HR sent through. And the success in finding people with the right skills who fit into the team was very low.

A question I tend to ask is how the candidates came into software testing. Most given answer was something in the sense of, “friends told me that there is an opportunity for me to get into IT”. My colleagues searching for developers and business analysts have similar time consuming experiences with only slightly higher success rates.
When speaking with friends in the industry asking them about their hiring experience, all share the same tenor. It gets harder and harder to find good people.

Software companies and products popping up everywhere

Success stories like Facebook and Instagram, collecting billions of dollars for their ideas and the users they have, local success stories getting still some millions from bigger companies who want to have the product. These are the factors that inspires people to come up with their own ideas to make some big bucks. Thanks to the internet its easier than ever to publish software and draw some attention to it. If your software goes viral (enough) you have made it.

There are lots of companies who try to emulate success stories and come up with a similar product. Once a product gets some fame, you can be sure that there will be dozens of clones available in no time, trying to get a piece from the cake. Thanks to the law of the market those products quickly reach their end of life, some keep dwelling in the shadows.

Frequency of updates

The internet and modern software development approaches like Agile and DevOps make it possible to update software quickly. There are companies using DevOps approaches bragging with daily or even more frequent updates. Time to market is one of the driving factors. Software companies don’t have months and years to come up with a product anymore. If you want to earn money and stay ahead of your opponents, you have to move quickly. Agile and the approach to publish MVPs (minimum viable products) is getting strong to produce the right thing. And it works e.g. great with websites and other centrally hosted applications. But with that approach, e.g. as seen in the app market for smartphones, the industry also produces thousands of apps that start with some small feature and then being either forgotten or growing from there. Often the aspect of viability is not very distinct.
Two decades ago I first heard the name “banana projects”, “ripes at the customer”. Back then it was a dispraise to get that title. With MVP that approach was made the weapon of choice. Don’t waste time on producing a product, fail fast and learn. That approach has two sides. Software companies save money and get a chance to produce something with potential, on the other hand users waste time to evaluate dozens of products to find a solution that fits. Win-win situation? I’m not so sure.

Big companies like Apple, Google and Microsoft have problems with prestige projects, affecting millions of customers. There are fixed dates to hold, promoted by marketing, without listening to project stakeholders. There is a keynote, they need to publish on a given date. Thanks to some late changes we see problems over and over again coming in the initial versions asking for patch releases soon after. Android has a different problem with device fragmentation growing by the hour. Teams concentrate on the majority of the user, evaluating if problems reported by users from the minority are worth fixing at all.

Mobile apps are the example I see that phenomenon every day. I have about 100 apps on my smartphone. Every work day I get at least 1 update for one of the apps. Sometimes up to 8. Shortly before and after releases of new versions of the operating system that number gets double-digit. On all my devices I spend a fair amount of time when using it with maintenance and updating. Something I don’t want to waste my time with, to be honest.

Performance

Hardware is getting faster and faster, and developers simply don’t have to care much about optimizing their code. Just add some cores or some GB of RAM, and it works faster again. This has of course always been a phenomenon. Developers were simply not trained to optimize the last bit of their code. Hardware gets replaced even faster these days, so why worry about slow devices? And so goes the spiral.

A long time ago when most things already got delivered in CD-ROM, there was an initiative that tried to fit awesome stuff on floppy disks (you might remember those things that look like a 3D print of the save symbol), taking max. 1.4MB of disk space and running performant on older CPUs as well. Sadly I couldn’t find any links.

Then there was Fli4l, a project that produced Linux distributions that fit on floppy disks to run some valuable software on old machines like firewalls, proxy servers and web servers booting from floppy disks.

You say in times of 128GB USB sticks that’s no longer necessary. Well, exactly there lies the problem in my eyes. People (developers and users alike) don’t care, because they don’t have to.

Times change

Times change and so do development approaches, ways of distributing software, and ways of using software. Is the demand for software higher than 10-20 years ago or is that demand artificially induced?

With the few topics I tried to explain here I still say from my point of view and in my opinion that software quality in many cases is decreasing. Some companies make it right and use the available approaches and technologies to improve their software. But how many are out there that are doing it wrong or simply deliver bad software faster. If you don’t come across those pieces of software in your life, good for you, and I hope it stays that way for a long time.

 

Reinventing Testers and Testing to prepare for the Future

When I want to speak about the future of testers and testing, I first have to think about the situation we have now, and why I want to change anything about that.

In the past decade or so there were multiple threats to testing and testers. “Test is dead”, “Testers need to learn to code”, of course the ongoing discussion (especially by tool vendors) “all testing can be automated”, and some more.

Then there was a workshop on Orcas Island mid of May 2016, organized by James and Jon Bach. The topic was “Reinvent Testers” and I don’t want to go into any details about the workshop since I was not even there and don’t have more information than what is available on the website. It was not the workshop that created some disturbance in the equilibrium of the community, it was just one slide – out of context – that was spread via Twitter.

The statement that got my attention was “there is an ongoing and longstanding attack on the testing role“. I don’t want to focus on any of the points on the slide, because I don’t think they support the statement in the header in any way. And I don’t want to say anything against the chosen hashtag, which was a typo (testing instead of tester) as far as I know, combined with a side blow on some minor political campaign currently ongoing in the U.S.

Perze Ababa was so kind and shared the slide after that one with some people on a slack channel. And that slide was reflecting in my opinion the actual problem way better. I don’t want to share the slide here, since I was not part of the workshop and I did not ask James and Jon for approval of sharing it here. But I want to quote some statements from the slide that better describe the current situation of how the tester role and testing as a phase are often seen by other participants of the software development life cycle.
“We must eliminate the need for judgement among testers, so that there is no controversy about what is a bug!”, “Nobody smart wants to be a tester!”, “Testers need to assure quality!”, “Testing is too unpredictable, you should instead make a Definition of Done and use Acceptance-Test Based Development to know when you get there!” and “You should write down all your ‘test cases’, tracing them to requirements, and track metrics against them!”.
This is only a small part of the statements on the slide, but I guess most of my readers get the idea of the tone I’m referring to and were themselves more than once in similar confrontations.

How I see the current situation of testers and testing as a phase

When I look at the current situation of testers and the standing of the testing phase in projects in general, I can understand why there seems to be a problem.

The testing phase was born in waterfall-like approaches many decades ago, and testing usually comes in the end, when everything was sort of stable. And that is where many people (testers and non-testers alike) still see the testing phase. Talk to the average factory-tester and he or she would advise to have a stable piece of software available for the testing phase. When you have to run all tests in the end and want to make a statement if all tests passed, that actually makes some sense in that context to not have a moving target and execute your test scripts with 10 or more different versions of the software. Reality might look different.
Being the phase in the end also gives you the disadvantage of biasing opinions that in case of a delay it’s your fault.

Two decades ago development approaches started changing away from waterfall. XP, SCRUM, Kanban, Agile, Lean, whatever the buzzword you prefer, it’s all about getting code faster into production or the hand of the customer. Faster feedback is the driver to develop the right thing and not waste much money on walking into the wrong direction.

The testing phase has a problem in those environments. It did not evolve together with the development approaches. Maybe developers sometimes don’t appreciate the feedback they get from a late testing phase, enforcing them to redo some of their work or getting told that they implemented the wrong thing. From a developer perspective that I experienced and also from a management perspective, testers should only verify that specifications have been implemented correctly. And most of the new development approaches found ways to get exactly that feedback faster without the need of testers. It seems nobody was missing additional feedback and information coming from the testers. Who knows why?

Testers who stayed in such projects simply forgot to adapt and tried to “only” attach a testing phase at the end of whatever development approach was chosen. There was also not much support from the testing industry. iSTQB that initially was taught in waterfall context only came up with a new certificate, the Certified Agile Tester. Not much of a help. ISO29119 is a documentation-heavy framework that hasn’t found an answer for Agile approaches yet. In the beginning of the millennium niches like the context-driven testing community, with a couple of thousand testers world-wide, were simply not big enough to make a difference in an industry with over a million people.

Nowadays testing phase testers have the problem that development approaches have evolved so far, that they cannot keep up any more.

Testers in general have a problem they didn’t cause. Testing was seen as simple task on one hand and was tried to be kept cheap on the other, since it added no value to the product, so many cheap people from various backgrounds were hired into testing jobs. The career opportunities in most companies are usually to step up some ladder and move out of testing positions. So the best testers were simply lost to other disciplines over time, because of the lack of opportunities in their profession. And I also claim that a good portion of people who stay in testing more than 5 years lack motivation to improve or don’t lack laziness. They just stay there because that’s all they think they can do and they have a safe job.

Management is also treating testing still like 2-3 decades ago, because back then it was easy to manage, so why change that. No motivation from that side either.

How Agile & Co. cope with testing

Projects, products and companies that work Agile have dealt with the situation and try to include testing earlier and let the developers do the job. Lots of checks that come up from testing are getting automated to speed up regression testing / checking. They try to keep testing/hardening sprints short and everybody helps there when not busy with fixing bugs. Approaches like BDD, TDD or ATDD support the cooperation between business analysts and developers, and help developers produce more fitting solutions.
Some companies even take it to the extreme and let developers accompany their code into production, see there if it works, and then leave it alone. No testers involved.

Do you think those companies, at least where it works, are crying for a testing phase or more testers on the project? I don’t think so.

In the AB Testing podcast, Alan Page and Brent Jensen regularly share a view on how Microsoft tackles software development processes, and it seems they are doing not too bad.

When claims come up that 90% of the people working as testers should get a new job, James Bach was citing such a request in a podcast I stumbled across last year, I say, may it be only 70% but it’s true. If more and more companies successfully change their development approaches, we will need less testers. And of course I want to keep only the top 30%, not any 30% and not the cheapest 30%.

How the role of testers will evolve

To survive in the world out there, testers have to speed up their game. Testers cannot waste a big chunk of their time with documentation. My motto since being team lead for a small team is, keep people testing. Time spent knee-deep in the system is usually more valuable than writing pages of test cases.

Testing in my opinion is all about providing necessary information for stakeholders to make informed decisions about the project. And those information are collected by actually spending time with the people and the system, not by preparing, updating and grooming long lists of test cases and other useless pieces of documentation.

Testing should be shifted left as far as possible. Regression testing should be reduced to the absolute necessary minimum. As Michael Bolton stated in a discussion yesterday, when you have to do lots of regression testing because the developers don’t understand how changes to the code affect the product, the developers have a problem, not the testers. Those problems need to be addressed differently. Not by blaming that the regression test phase is too expensive (I have a friend, who knows someone, where that might be the case).

Having testers in your team should not be a surrogate to writing and maintaining automated unit and integration tests, because developers are too lazy to write them, or the architecture is not favoring writing them. Then you have a different problem. Throwing testers at the problem won’t help solve it. But it provides someone else to blame.

Testers should take care of the bigger picture, things that are hard to automate, cope with complex problems. Testers should help developers to create good automated scripts, help them to understand and test the application and train them to do the simple tests earlier and faster and provide feedback faster. And yes that needs skills, competence and behaviors of a systems thinker, of a detective, of an explorer, and of a coach. That’s nothing you can buy cheap on the next corner. Those people are rare.

The role of testing needs to be changed?

Some people participating in the discussion call out to testers to take different roles in the project to still provide value. I see that as a wrong approach. As a tester in the role as tester, used in the right situations, I can provide value to the project only I as tester can provide. And I don’t want to give up my role as a tester. I want to continue asking questions, experimenting with the system, analyzing strange problems, I don’t want that to go away.
If my job as tester on my project is not filling out my whole day, or I as an individual decide that I can be more useful and contribute to the project in different other roles that may even improve my (part-time) job as a tester, that is a great opportunity. And I strongly advise you to take that chance. But your role as tester doesn’t change.

I don’t think that the role of testing should be changed by taking on new tasks. Those are simply other roles that can be filled by the same person (specializing generalist comes to mind). I recommend to make better use of the role testing and focus instead. Tasks that are not testing and produce more waste than value should be taken away from the role of testers. Enable testers to act faster!

What testers need to do is evolve or reinvent the way they work, not by taking additional jobs in the project, but finding a way how to produce the necessary information faster, earlier and more efficient. And testers need to dare to hand over simpler testing tasks to developers to stages or phases where it makes more sense.

The testing phase should be changed. If you need an unnecessary long regression test phase, you better rethink your development approach. If you run tests and checks in that phase that take days and weeks, you might want to rethink that approach. If you regularly run into problems in the testing phase you have a problem.

And when Brent Jensen says that data scientists and telemetry are attacking testing (episode 39 of AB Testing podcast), I can only hope that testers position themselves accordingly, stack up their arsenal and be part of those approaches. For me that describes excellence in testing. Monitor and gather feedback from production and use scientific methods to use that data to inform further actions. Fantastic! Where can I sign up? I am a tester, and I want to master those tools.

Responsibility for the overall decrease of quality

Software developing companies have a common problem in the last few years, starting maybe over a decade ago: Product quality gets worse and worse. The possibilities of faster cycles to production makes many companies negligent about actual quality, because it’s easy to provide a hotfix. Only the rules of the market help to filter out the worst of them and also to prevent the ones with good quality from the start to get a chance in the market, because they tend to be too slower.

I would see that partially as a problem of neglecting testing phases, doing the wrong testing, doing testing wrong, or hiring the wrong people to do the testing.

Testers need to fight and find their way back into those projects and provide value to help the team raise the level of quality again.

[Update: The claims offered are my personal views, and I go into more details here.]

What companies need to do

Companies need to decide whether they need or want to stay in waterfall-like approaches, having that big testing phase at the end, and taking all that comes with it into account. Waterfall is not bad, and Agile is not for everybody. And always remember: A good waterfall is much better than a bad Agile-ish attempt.
Some may even have to bite the bullet and outsource their testing to some expensive, documentation-overload-producing consulting company. There might be good reasons for that.

One hint: You might want to rethink your approach if you are in regulated environments and take it as excuse to say you need a documentation-heavy testing process. I have heard from multiple people that this is just a myth.

Invest in the future: Prepare your people to fill that high sophisticated role as tester. Find those who are willing to invest in their craft and profession. Spend time and money on training and conferences, let them interact with like-minded people, get them engaged, get them motivated, and pay them a decent salary to have them focused on the job and not on how to pay the bills next month.
And if you ask what happens if you train the people and they leave, better ask what happens when you don’t train them and they stay!

What testers need to do

Managers also need to change their view about testing and testers. But it’s the task of the testers to build trust in the information they provide and demonstrate the value they can add beyond test scripts and developers testing themselves. Then managers will hopefully change their opinion that testers only need to provide a count of test cases to show their value.

Do everything you need to get better in your craft. Learn tools, methods, approaches. Get better with soft skills like communication, critical and creative thinking, problem solving. Stay up-to-date. Interact with your peers in- and outside your company. Go to local meetups. Read books, blogs. Go on Twitter and engage with other testers. Learn from them where to find more sources for information.

If you want to be in that 10-30% that should stay in the profession, you better start now.

And my last tip: Be proud to be a tester!