The words “Testing” and “QA”

Introduction

I’m not one to harp on about semantics, I mean, I care that we all understand each other, but I’m very conscious of how irritating it can be to constantly point out “you’re using that word wrong”. However, I do find it frustrating when people assume that language used in the community or in documentation is commonly agreed, understood and given the same meaning by everyone in the same way. Very frequently, this is not the case and the words “testing” and “QA” seem to be an example of this. As a tester, keeping in mind these semantics can be very useful in resolving misunderstandings that may have been missed.

Why do I find it frustrating?

Ever since I was a newbie to testing and the tech industry, I’ve been trying to understand. I hear phrases or words and try to find out what they mean. Someone may teach me the word, I may read it in documentation or hear it explained in talks. Most of the time I learn words through the context they are used, which is probably the most natural way we all learn language.

So I learn words or phrases and believe I understand what they mean and hence believe that I would be understood when I reuse them the same way. However, because we are creating new words and phrases all of the time, some of them don’t have a “standard” meaning. An obvious example of this I can think of is regional dialects or slang, and in the UK we have many, many words to describe a bread roll.

So if I asked for a “Bacon Stotty” in London I would get a puzzled look and if I asked for a “Muffin” I might get one of these:

I personally see this as simply a natural development of language, we invent new words all of the time to describe things that are new to us, especially if we don’t have an appropriate word to use already. However, due to distance and culture as humans we may come up with different words to describe the same thing or the same word describing two different things.

The frustration for me is when I encounter people who don’t seem to accept this. The phrase “QA” is widely used in the tech world in many different contexts and doesn’t appear to have much common agreement on its definition. For this reason, I don’t like using the phrase because I don’t believe I would be well understood most of the time. However, I haven’t felt the need to talk about this until now because I haven’t come across an example that justified my feeling on it.

Enough accuracy for enough understanding

So I recently interviewed someone who was looking to start as a tester, they didn’t have any experience of testing and were keen to understand as much as they could about the role. I will add they were impressive for someone with no experience! During the interview one of my colleagues explained the gist of how the development teams were structured and what they worked on. They described a tester on one team as performing “performance testing” and then another one on another team as performing “QA”. I wryly smiled to myself about the use of the word “QA” but I didn’t say anything to it because it was a reasonable, high-level description of how we might work in a very generalised sense. While I knew some of the words were misleading, it wasn’t the time or place to pick it apart because:

1. I didn’t want to embarrass my colleague and I didn’t want to give the candidate a bad impression of our relationship.
2. I didn’t want to spend the limited time we had in the interview explaining why those words weren’t right.
3. The gist that was given felt enough to me for the candidate to understand how we worked, at least for now.

Recognising misunderstanding and addressing it

The candidate was happy with this explanation and I was happy that it was enough to at least give them some context to ask any further questions. The interview continued and at the end the candidate expressed they were really happy we had an “Internal QA” role because they knew of a similar role in their current company which involved checking products were meeting guidelines and standards set out by governing bodies. They felt that this was a role they would like to start with because it would give them an easier step into testing and more technical roles (they openly admitted they didn’t have much technical knowledge of experience but wanted to learn).

Now it had become apparent the gist we had given was clearly not appropriate because the candidate had understood the phrase “QA” differently to how my colleague had meant. The candidate now clearly expressed a desire for such a role, particularly because they were keen to have some guidance. They liked the sound of QA because it sounded more scripted, more guided and therefore an easier leap into the tech world. I immediately explained to the candidate that in the context of this business, we don’t have a role like that and really the roles are far more exploratory in nature. I then had to go into explaining what I defined as “QA” and “testing” what the difference was and why my colleague had interchangeably used them.

This real example hit home for me a justification that the phrase “QA” is misused and misunderstood. It can have multiple interpretations and in this case it could have led an interview candidate into accepting a job completely different to what they had envisioned.

What are my definitions of “QA” and “Testing”?

My definitions are:

• “QA” or “Quality Assurance” involves checking that a piece of software conforms to a predetermined set of qualities. This can come from legal requirements, industry standards or certification. QA is generally scripted in nature.
• “Testing” involves exploring a piece of software to discover information about it. While it may include the use of checking predetermined sets of qualities it focuses on the unknown rather than the known. Testing is generally exploratory in nature.

I do not dare suggest these are commonly agreed definitions and I definitely do not go around correcting people on this. I have learned it’s pretty irritating and counterproductive to do this. However, I use these definitions to help me identify when I have misunderstood or someone else misunderstood these phrases. In other words, I’m aware that these words aren’t commonly understood the same way and I choose to clarify my understanding this way when the discussion comes up.

No one is at fault here

I want to emphasise that in the interview situation I refer to, no one was at fault. I do not expect my colleague to understand all of the semantics of testing. I do not expect a newbie tester to be aware of them either. I’m simply observing that it is useful to keep in mind these semantics and how people may be silently misunderstanding each other and not realising it.

I don’t think we can really do much about preventing this divergence of meanings of words and phrases, I personally feel they are a natural flow of language. The English language is full of words that have many contextual meanings as it is. Nor can we “know” all of these meanings. All we can do is share our different meanings of words and raise each others awareness to these different meanings and perhaps come to some greater consensus on definitions.

Providing value beyond bugs

Introduction

I’ve had several experiences where I’ve worked on projects where a manager has proclaimed “we don’t need testers for this project, we’re not bothered about bugs”. When I started testing 6 years ago, I would have felt that this was wrong simply because “bugs” can take many forms. However nowadays I’ve come to realise that testers provide much more than “bugs”. I still find it incredibly difficult to explain this to people (particularly managers) though and it has only been through bad experiences that I’ve felt justified in arguing the case.

Why people might think they don’t need testing

I have now worked with several projects where I’ve been told that testing wasn’t required, the reasons have varied:

• “This is an internal project and we’re not bothered about embarrassing bugs”
• “This is a quick prototype and we want you to focus on more important projects”
• “This isn’t the software you need to test, it’s just a library/tool/software we’ve bought so we don’t need you to look at it”
• “This project is in its early stages so there is nothing for you to test yet”

It seems that people still see testing as simply being the bugs we report, and not only that, but sometimes people are only thinking of bugs in the popular sense - visually obvious and embarrassing bugs. They also seem to be making decisions regarding risk and priority with the information they have at the time.

Why did I think these projects would benefit from a tester?

Well I think the main reason is because I see testing differently, I don’t see my job as simply reporting bugs, but about telling the truth about software, about observing the people and processes that produce the software and trying to help those people to produce better software. In other words, I see testers as people dedicated to creative and critical thinking on a project.

• Internal projects still need to ‘work’ right? Not to mention, do we really understand all of the kinds of bugs or risks up front about the project? Maybe the project is for internal use, but does it interact with external systems? What about the people and processes, do we not want to help them? Do we not want to track the progress of the project? Even internal projects have costs and implications if they are late or don’t fit the requirements.
• Frequently quick prototypes prove to be quite useful, this is generally their purpose - to rapidly learn what is useful or valuable. Testers are great at rapid learning and especially learning what customers or end users find valuable or useful, so such information is invaluable when moving on to design the eventual fully-fledged product. Why not boost the success of your prototype by involving a tester to help focus the product on the learning as well as bring their skills in analysing the truth of what makes the prototype a success. Not to mention that prototypes tend to very quickly become the “finished product” without any re-design or re-development, it’s easy for people to assume it’s ready to change the usage from “concept” to “ready for mass use”. Involving a tester might help avoid this easy slide and highlight the risks.
• Also, what if the projects we are testing affect your prototype? We won’t know that if we aren’t aware of your prototype, which means there is a risk that we could break your prototype if it depends on other projects.
• Assuming other people’s software is well tested and perfect is an all too easy assumption. Then there is the question about whether it’s even compatible with your own software and the assumption that you understand what it does or how it exactly works.
• Many bugs are caused by assumptions made at the design stage either because of ambiguous language, cognitive bias’ towards information or simply because we cannot think of everything. Does it not make sense to more cheaply catch and resolve these bugs at the design stage rather than finding them after we have spent time building a product?

I believe in all these cases and in any project, you can benefit greatly from involving a human being who dedicates their focus to critically analysing each piece of information and offering feedback. It’s not just about banging on keys or finding “bugs” in the language of computers, you can discover plenty of “bugs” in the language of humans.

Changing the tone from an argument to an invitation

Its draining to constantly have to argue to be included in meetings and projects and this can heavily affect my motivation at times. It’s much easier when I’m invited and do not need to convince people of my value. How to do this then? I think I’m still learning how to become better at this, but one obvious way is simply by becoming extremely knowledgeable about a project, its technology and the end users. By simply being able to answer many questions and provide this knowledge, you naturally become an oracle people refer to, which means you become invited to more meetings.

Take advantage of your opportunities to learn

What do I mean by opportunities? Well firstly, opportunities to become very knowledgeable can take many forms - for example, investigating bugs typically shows you the guts of a system and also provides information of why a system was built a particular way (otherwise how do you know its a “bug”?). Try to view everything as an opportunity to learn more and you may pick up and remember a lot more.

Now you’re in a design meeting, how do you prove your worth?

I also mean opportunities in terms of demonstrating your value in providing critical and creative feedback. It can be very easy to squander these, I’ve repeatedly been too critical, asking the wrong question at the wrong time and earning the ire of my colleagues for wasting time or dragging out discussions. This can lead to being left out of discussions for being disruptive and not constructive.

Another area I’m trying to learn to improve is catching myself when I react emotionally to something, I tend to blurt out a critical question when I see something wrong. Sometimes I should consider how to word my question better to come across as more inquisitive rather than criticising. Sometimes I should really ask the question later and not put people on the spot. However, as ever, such feelings can be heuristics and sometimes I am right to ask the question there and then, sometimes the emotion draws attention to important details. The main point here though is that you shouldn’t always let your emotions guide you.

Trust

Asking too many questions tends to make people feel they are being interrogated, and perhaps not feel trusted to do their jobs. For example, several times I’ve found myself invited to meetings about new projects that I’m not completely up to speed with and a discussion start with the participants assuming I know some things already. It’s too easy in these situations to criticise and say that they’re making a lot of assumptions, when really they established details in another meeting and you’re simply not aware. While this is something that needs to be resolved, by aggressively criticising them for this, they will get defensive and you potentially derail the meeting. It’s being aware of these situations and being critical at the right times that allows you to provide your value, being critical all the time feels counter-productive to me.

Asking questions rather than making declarations

I’ve caught myself declaring things in the past, which comes off as criticism of an individual or a statement of objective fact to argue about. I try to instead ask questions rather than make emotional statements, emphasising that I’m not criticising an individual or think something is bad. I guess in a way, this is a kind of “safety language”, to use the RST (Rapid Software Testing) phrase. For example;

Instead of:

“This isn’t a good idea, I don’t know why you’re suggesting that?”

I might try:

“I don’t mean to criticise you, but are we sure that X is a good idea?”

Instead of:

“These requirements are really ambiguous and unclear”

I might try:

“Apologies if this has already been covered, but could we clarify the requirements here? X and Y could be misinterpreted?”

Conclusion

I think to change the view on whether testers contribute more than just bugs, we have to show value in other ways. One way to do this is becoming an oracle of knowledge about a product, business or end users in general. This can then lead to further opportunities where we can provide value through our critical analysis skills. However, if we are not careful, we can deal more damage to this image than good if we are too disruptive.

I care a lot that my contribution to projects is valued and that I’m of service. I don’t want to waste people’s time and I want people to invite me because they appreciate my skills and knowledge. However, the nature of testing tends to be viewed as negative and sometimes I myself contribute to this negative view when I disrupt meetings with heavy handed criticism. Life is all about carefully balancing my critical feedback with self-criticism and also recognising opportunities and carefully not squandering them.