I want to offer you a hypothesis, a proposition that I don’t have proof for. But I believe I’m on the right way.
“Your personal definition of quality influences the way you test.”
Quality is a seven-letter word. I think that’s the only statement that we can commonly agree on. Quality is a very complex matter and I don’t want to go too deep into detail on that this time.
To make my point I’ll base this post on three common defintions of quality, that your personal understanding might be more or less based on.
Q1: “Quality is conformance to requirements.” – Philip B. Crosby
Q2: “Quality is value to someone who matters at some point in time.” – Jerry Weinberg, extended by James Bach, Michael Bolton and Anne-Marie Charett
Q3: “Quality is fitness for use.” – Joseph M. Juran
I want to describe the type of testers that I see behind those definitions. This is based on my mind model, my experience of 16 years and the people I work with and talk to. I know that this number is way too small to be representative. But maybe it is helpful for some, at least it helps me to understand people and their motivations, and their way of working, how projects are set up and so on.
Type “Quality is conformance to requirements.”
Frameworks like iSTQB and PMBoK are based on variations of Q1. And this totally makes sense from their point of view. That way testing is plannable and controllable, and you might even come up with metrics to make quality measurable, based on that definition. It’s a good way to define price tags for testing, which enables schemes like outsourcing testing.
The other definitions would not be able to serve that purpose in a similar way.
Testers with an understanding of quality like Q1 might tend towards test cases and test coverage metrics based on requirements. Waterfall-like approaches (including those covered as Agile) make them feel comfortable and standard test case deduction methods are their daily tools.
Projects and general product development with very specific lists of requirements, standards to adhere to and processes to follow would need a quality understanding like this.
Also people with a background in model-based testing might feel comfortable with this definition.
For concrete implementation projects they need to rely on customers that are able to express their requirements.
Type “Quality is value to someone who matters at some point in time.”
Proper Exploratory Testing in my opinion tends to be more based on definitions similar to Q2. They explore the system under test from different angles, and exercise it based on the findings that they evaluate most important. Test reports should inform decisions and rather tell a story than produce numbers. They are aware that they are not the customer or end user of the system, but they try to resemble them as good as possible.
I could imagine that people who see themself as context-driven testers might have a quality definition based on Q2.
They understand that context matters most and the usefulness can change over time. This type of tester in my opinion is more aware of potential risks and trying to detect potential risks is more important than covering every edge case possible. They also understand that quality is different for different stakeholders and users.
Type “Quality is fitness for use.”
Approaches like observation, monitoring, testing in production, data analytics and alike belong more to quality definitions like Q3. They want to see that the implementation works in the field. Testing before releasing to production is mostly used to minimize risks of massive failure. Carefully releasing software into the wild and rolling back in case of failure is their preferred way of checking code changes.
I’d assume a trend towards incorporating parts of definition Q2 in their understanding of quality.
Background story
I came to this hypothesis recently when I had to change teams in the same context and room from a customizing implementation team, to the product development team. I did not feel too well in the beginning after the change and I wanted to understand why. It’s not the people or the domain. It’s the way of working or rather the definition of quality you need to apply that defines the context. In my case I had to switch from a Q2-context to a Q1-context. Guess, what I prefer.
Summary
I believe that your personal definition of quality is a fundamental piece of the puzzle how you subconsciously work, how you test, how you design test strategies, how you’d set up a testing project and so forth.
Of course people can adapt to their current context and fulfill the requirements of the job, and do it good. But I assume they won’t feel as comfortable as they could. At least I do.
I need your help!
You made it this far, thanks for staying. I need your help! I would like to know if my hypothesis is worth following up on.
If you have a personal definition of quality, maybe it fits roughly to one of the three examples provided. And maybe you are aware of what kind of context you mostly enjoy working in.
Please let me know, if my generalized description above fits to your personal situation or not.
Thank you!
Test Pappy 
A very thoughtful post!
My favorite part: “This is based on my mind model, my experience of 16 years and the people I work with and talk to. I know that this number is way too small to be representative. But maybe it is helpful for some, at least it helps me to understand people and their motivations, and their way of working, how projects are set up and so on.”
Also, I agree that “Your personal definition of quality influences the way you test.” and I think that your post helps support that claim.
Of the 3 definitions provided, I most closely align with Q2. However, I can see how I might agree with the other definitions, depending…
Q1: “Conformance to requirements” could be restated “compliance (acting in accordance with some wish or command) with wants or needs” and might imply “of someone at sometime”. And if so, then I agree – that’s quality!
Q3: “Fitness for use” might mean “something that is suitable to fulfill some particular role/task” and imply “of someone at sometime”). And if so, then I agree – that’s quality!
And, since you asked: Yes, I think you should ruminate more and provide a follow-up if/when you’re ready! I’d love to see where this goes…
A lot of good testers, are biased against the Q1 folk. I agree with that. However, there is a positive bias towards Q3. I don’t agree with that. The people who follow Q3 do not have an understanding of testing. You don’t have to choose between Q3 and Q2.
The reality is that you can (you shouldn’t) ignore Q2 – customers compensate for your shortcomings. Support takes care of issues.
In the software industry ET/Q2 does not stick. You cannot rationalize that. I think the Stanford grad whose idea of testing is test automation is the same as the ISTQB certified engineer writing testcases, or worse.
Of course, if you understand ET, there is no turning back. it can be difficult to get others to see your point of view.
TL;DR: Fitness for use is not either-or with Value to some person. (Q2 is not either-or with Q3)
Let’s not unnecessarily vilify the Q1 folk.
This is good! I will quietly hold my hand up as a Q1 tester, HOWEVER, a thing that is under-appreciated is that Q2 and Q3 can and should feed back into Q1! If you are getting product returns or tech support calls, or whatever the consequences are for your product that you missed the mark, you have to then update the requirements. Complaints from the field can drive changes to the current product, and that’s good. But they should also drive changes to your next product’s test plan, and that’s even better!
I’m a hybrid… mainly Q2 with some Q3, about 60%/40% respectively. I really do not enjoy the Q1 approach and unfortunately a lot of companies expect it (maybe because they don’t have a comprehensive understanding of the role).
I enjoyed your article because it reinforces in my mind what I’ve been thinking for a number of years. I’ve always wondered if my personal approach needs to be changed, given the expectation can be so vastly different depending on the team you’re in. It’s nice to know that there are just different approaches, simple as that. We just have to be versatile enough to apply our skills to the environment that requires them.
In my experience so far, described Qs pretty much describe what I have witnessed. Over the years I found myself in all Qs described in the post multiple times. Personally, I feel most comfortable in the Q2. However, I believe that we should be ready for any of these contexts when need arises to switch between them.
What helped me during these switches is to understand why it is important in this particular situation to apply certain context/mindset. It helps me going through some work I might not prefer if I understand why it is important to do it.
And yes, I’d like a follow up in the future, please. I believe this topic really deserves further exploration 🙂
Like your classification, but do not necessarily like the subtone of “Q1 is not good”. At the moment you automate something, you are in this area. I would say all three have their right time. My definition of quality pretty much tends towards Q3: the user should get the value expected. But you need Q1 and Q2 to get there…..
Thank you for your comment, Ursula.
I agree that my subtone in regards to Q1 is probably not neutral.
When automating certain behaviour of an application, I’d still rather go with Q2 or Q3, as we tend to reduce the requirements that we want to observe with automation to those that are most useful to us, selected from those that have been explicitly written down for implementation. Or we add scenarios that we came up with during design and implementation.
“Fitness for use” and “Value to someone” also include that the stakeholders have requirements that need to be fulfilled, but are more customer-oriented.
Maybe my negative tone against Crosby comes from his general theory of “only zero defects mean perfection” and additional statements like that.
Most probably I misunderstand some of the theories, but my understanding / interpretation of Q1 is that Quality happens before production, and Q2 and Q3 very much relate the product with its users.