I love testing and this is why!

Introduction

Over the last year I’ve been on a little bit of a journey of discovery, for 5 years I had been working as a tester but never really seeing it as a career I wanted to keep. In this 6th year of testing, I finally realised I loved it after discovering the friendly and passionate testing community. I’ve learned to appreciate my own work, justify it and learn how to become better and better at it. Suddenly what felt like a job that didn’t have any further growth exploded into a fascinating and immersive world where I could maximise my skills and help others to realise theirs too. I’m taking this moment to write down why I love it so much as a little preparation so that I’ve thought about what I want to say whenever I talk about this again in future.

Testing is endless learning

I’ve spent most of my life (up to this point) in education, I went to nursery at the age of 3 and graduated with my degree at the age of 21. On reflection I feel I’ve always liked having that next target, be it grades for next year, progressing to university or securing full time work. I’ve come to realise one of the things I relish about testing is always having something to learn. I now effectively describe my testing as learning, especially thanks to attending and studying Rapid Software Testing with Michael Bolton which really hit home this point. As an example, before I started my current job, I knew very little about performance, security, API and automation testing. I can now confidently talk about each of these subjects and know where I need to improve.

When someone presents me something to test, I’m trying to learn as much as possible as soon as possible and I love doing this, I love building on each bit of knowledge and using it to ask more and more questions. Let’s say for example I’m presented with an accounting service that integrates with other systems, I start reading the documentation or asking questions and learn that one of these integrations accesses through an externally-facing API. So then I’m asking questions about what is this integration? What does it do? Who is it for? Why have we built it? Can I test it? What does it expect as a response? Is the API secure? What happens when I try to access it without authentication? What happens when an error occurs? What is the desired performance end-to-end with this integration?

Testing is about facts, though opinions are valued too

It’s sometimes easy for people to perceive testers as picky or pedantic people, always criticising or bringing bad news. If we are not careful with our expression, we can over-state a personal opinion or fail to justify why something is a serious problem. I personally enjoy trying to constantly improve my communication, maintain a high standard and keep learning how to interact better with so many different kinds of people. All the while trying to keep in check my own emotions while at the same time knowing when to listen to them.

Testing is about being involved everywhere

As a tester I find myself working at every point from start to finish of a software project. From trying to learn as much about our customers and the business to learning all about the behaviour of a particular line of code. I get to work with so many other roles such as development, business analysis, project management, systems administration, technical support, sales and marketing and all the wonderful and horrible ways people might like to use the software. Testers have moved on to become business analysts or project managers and I think it’s a very natural career step for some.

Testing is about being persistent

Like the world’s best detective, as a tester I try to leave no stone un-turned and I try to look beyond the obvious explanations. I actually find this one of the harder parts of testing, it’s easy to accept an explanation that you don’t fully understand, because sometimes you think you do but don’t really. Sometimes it’s also difficult to keep persisting, especially when a problem resolves itself. Maybe you don’t always have time to keep searching. However, the satisfaction in finally discovering the root of a challenging problem is so high that it makes it worth it. I try to remind myself of the detective analogy because it helps re-energise me (it feels more exciting!) when dealing with tricky problems.

Testing is about directly and indirectly helping people

In providing quality information, I’m helping people all of the time. Helping developers learn more about what code actually does, helping product owners track the progress of a project or helping everyone become more careful with their language and aware of assumptions. I’m also indirectly helping end users of software, trying to understand their needs and understand how these match against the software. I feel caring about people is so very, very central to testing.

Testing is about balance

I like to think of myself as pragmatic and realistic, I care about being critical of my own views with respect to considering someone else’s point of view. So I relish the challenge of balance, I see so many things in life, even life itself being one of balance. There are always pros and cons, always another perspective, another time when things might make more or less sense. The most common challenge that crops up in testing is balancing the desire to learn as much as possible with being quick. Why is time important? Because to learn everything you need all of the time in the world, which you never really have. So you’re always prioritising and trying to identify where the most important pieces of information may be hiding so you can find them as soon as possible. Everything costs time and so we are always asking ourselves whether it’s worth spending the time or if we are spending it in the right places.

Testing is a field ripe for discovery

While many people have discussed, experimented and written about testing for decades, it still feels like many of us are only just beginning to really understand it better as a field. The spread of knowledge and encouragement to discuss how to become better at it seems to be getting closer to other fields like programming. To some extent, it feels like testing is still lagging behind those other fields, we are only just starting to see more conferences and meetups. As a relatively new tester, I can’t speak for what it was like before, but I went a whole 5 years without even knowing there was a community who talked about it, let alone had ideas to improve it. I like this because it feels like an exciting place to be, with so many topics to explore, so much to possibly contribute to. There feels like a lot of work still left to be done to reach out to more people, both new testers and those who may become a tester in future. Not to mention that as we educate ourselves about what testing really means, we also become better at encouraging those other fields to understand it better too and interact with them better.

Testing is great for drawing on diversity

I make use of several different skills and bits of knowledge all of the time in my testing. An obvious example would be making use of my programming background to build useful tools or understand and read code. But I’ve also made use of my education to help me write reports, produce presentations and I’d even credit my studies of history (such as understanding how to analyse sources) with how I approach analysis. I’ve heard of examples of testers using their background in journalism to ask better questions and gather better answers or testers with a keen ear for audio being able to more accurately describe an audio problem. I love that testing so visibly benefits from diversity.

Testing is very psychological

I love psychology, it’s fascinating to me and so much of my work is affected by my psychological state. Topics such as confirmation bias and social interaction come to mind as the most obvious examples. Understanding these areas helps to stay alert to some of the most easily missed issues. For example, if you consider Usability Testing and how you analyse the information you gather from it. You don’t just ask users a list of questions and take away their answers. You are looking out for so many psychological factors too - how are they interpreting the question? Why did they give that answer? What were they thinking when they performed that action? When they say that they want X do they really need Y?

Testing is creative

In our pursuit of learning, it helps to be creative, because it allows us to explore much more than if we stick to the path laid out for us. In testing, being creative is highly rewarding as you are better able to not only discover issues that no one had thought of but also workaround issues or make an innocent-looking problem worse.

Testing is about being a team player

Testing cannot be separated from the work done to produce software. I’m not a rock star, walking onto projects and “assuring quality” and telling people what they’ve done wrong. I’m here to support others in producing software, to work together and learn together. By working more closely with people and learning to become better at working with people, my testing becomes more effective, efficient and supportive. I personally hate working alone and love working in a team. I love “team spirit” and pushing each other to become better.

Summary

This all I can think of right now on why I love testing, I’m sure I have more that will crop up another time. Many thanks to specifically James Bach, Michael Bolton, Jerry Weinberg and Karen Johnson for unlocking this love and appreciation for testing and the ideas and analogies I use in this article. Particular thanks to Michael for this blog post which inspired me to write this and also Andy Tinkham on this podcast for reminding me (through the idea of writing down what I think testing is) to write this.

Games Testers should be proud of their work, not ashamed

Introduction

A few months ago I interviewed someone for a testing role and the question of past experience came up, to which the interviewee said something along the lines of ‘well this might be a bit sad, but I’ve done a bit of video game beta testing’. Then a week later I noticed a discussion on the Ministry of Testing’s slack team where people were discussing how they had got into testing. A couple of people mentioned how they found it useful to start in games testing but felt games testers get a lot of stick for what they do. This prompted me to jump in and talk about my story as I’m quite passionate about this subject! In fact this very sentiment is where my career pretty much started after university!

Pizza, pop and candy

I will never forget this line. I had just finished university and was applying for every programming job I could find. There was a wild week (they seem to happen every year or two) where I had accrued three interviews to take place in one week, just prior to one of my brothers getting married. I had two interviews for programming roles and one for a games testing job.

The first interview I didn’t pass, the second one I seemingly impressed and got offered the job to test games and the third one? Well, bearing in mind I wanted to get a programming job to make the most of my degree, I went into it feeling glad to have the games testing job in my pocket but wanting the programming job. However, the third interview went something like this:

“So we see from your CV that you’re interested in working in the games industry?”

“That’s right, but I’m keen to learn whatever I can from any programming role I can find!”

“Pizza, pop and candy”

“Excuse me?”

“That’s what you ‘gamers’ like isn’t it?”

“Erm…”

“Yes, we hired a programmer from that games company nearby, what was the name?”

“Traveller’s Tales?”

“Yes, that’s it, but they didn’t last long. Great programmer, but not a good fit for our organisation...”

Needless to say, that was the most awkward, aggressive and stressful interview I’ve ever experienced. I honestly don’t know why they decided to waste their and my time in such an interview. But it made me suddenly feel very, very comfortable to be accepting a job in games testing and definitely removed any second thoughts! My testing career effectively started from a statement of disgust about ‘gamers’.

Learning to test

So here I was in a job testing, having barely heard the word ‘testing’ before (it seems crazy to think now that you can complete programming degrees without ever hearing the phrase ‘unit test’!). I’ve loved video games all of my life and I had seen my fair share of bugs with them, so I knew pretty well what ‘good’ looked like and what ‘bad’ looked like. I also play a lot of different types of game at varying difficulties, so I had a lot of knowledge to draw on - not to mention a degree in games design & programming. I understood what people wanted from games, how they are made and what could go wrong. I hadn’t worked a proper full time job before but I was pretty confident I could do a good job.

I learnt a huge deal games testing, especially regarding the sheer variety of forms a ‘bug’ or ‘problem’ could take. Not only were there the obvious bugs that a lot of people might recognise like crashes, hangs and freezes (yes, they had technical differences) and graphical glitches, but also problems you don’t necessarily see visually. The bulk of the testing was exploratory in nature, usually with a task being to focus on a particular area - be it a collision detection test or a playthrough of the game just to make sure it could be completed.

To give you some idea of the variety of problems we would look out for:

• Graphical - the most obvious and visual kinds of bugs that affected the look and feel of the game.
• Functional - quite simply, bugs to do with what the game should ‘do’. This was very contextual as every game is designed differently, but there are some fairly typical consistencies such as being able to steer a car in a racing game..
• Difficulty - this was pretty subjective, but we had to judge whether ‘easy’ was consistent and didn’t suddenly jump in difficulty on a particular level.
• Audio - maybe the voices in the game don’t synch up with the animations of the characters or the quality of the audio was poor (such as once hearing the ‘beeps’ of the recordings being started and ended for the voice actors!).
• Legal - sometimes games would include trademarked or copyrighted material which they hadn’t credited. Or sometimes they could accidentally feature close similarities to real world products.
• Health - we were trained on photosensitivity and how to identify bugs that would trigger epilepsy.
• Technology - we were sometimes testing for bugs with particular technologies such as 3D, motion control, force feedback devices, augmented reality, virtual reality headsets and more! All of these technologies feature their own particular bugs and problems.
• Compliance - we checked that games were compliant with the standards set out by the corporation. Games that didn’t comply with these standards would be rejected so we tested to make sure they were compliant.
• Localisation - although we didn’t translate games nor could necessarily read all languages, we did check for bugs related to the subtle differences in localisation. An innocent missing language file would be enough to make a game unplayable or crash. You don’t need to be able to read German to notice the text doesn’t fit inside buttons!
• Multiplayer/Performance - we tested for load, stress and also for exploits. Some games need to be able to handle large amounts of players and exploits can ruin people’s enjoyment of their online gaming experience.
• Compatibility - there were some cases where games were ported to new hardware or developed to communicate with older systems. We tested to find problems with how these games were compatible.
• Transactions - we tested to make sure real world payments could be completed and that people couldn’t exploit these systems.

There are probably a few I’ve forgotten, but hopefully that gives you some idea of the wide range of issues we would learn about and find!

Along with learning about the wide variety of problems, I also learned about testing techniques and strategies. Unfortunately the company was heavily using test cases at the time. There were differing views on these and I learned to find and report problems from my exploratory sessions more than fill out the boring and repetitive test cases. There were also the beginnings of automation testing which was being used to help carry out load and stress tests in multiplayer games along with experimented use in running regression test cases.

I learned two major lessons about testing in this job:

1. I hated test cases and found them fairly useless. Occasionally they were useful to find test ideas or guide exploring a product, but they felt awkward to write and use and just generally wasteful of time. I found most of my bugs when I was performing exploratory testing.
2. Automating tests on a product that was constantly changing, especially in a graphical sense was a waste of time. By the time we had written automated tests of the product, it had changed again and we had to re-write them. Quite a lot of game projects didn’t require testing after release, so the value of the automation tests was rarely seen. It only seemed to return some value when running large scale performance tests.

What did I pick up that could apply to other testing jobs?

There were things that I picked up in my first job that I hadn’t formed thoughts around or hadn’t found the language to describe effectively but were important to becoming a better tester. While I gained better understanding and language later in my career, my first job gave me awareness of these issues:

• Managing relations with developers - I quickly learned that diplomacy was important when raising bugs with developers. I picked up an analogy that I re-used in future testing interviews - “imagine an artist has created a magnificent piece of art and you come along and say ‘you missed a bit there’ - you can imagine that’s quite irritating so I look for ways to soften that blow”.
• Costs of automation - I didn’t have the language to describe it, but I learned the hidden costs of automation and the temptation to try and find ways to use it. I was already wary about the value and how easily people could misunderstand that automation is some kind of replacement for ‘all’ testing.
• Knowing when to stop testing - I had started to get experience on when and where I might want to stop testing. While we were generally time-boxed at in my first role, I definitely saw the diminishing returns and learned which risky areas I wanted to focus my testing on.
• Knowing when to repeat tests - I also started to form ideas on how to judge regression testing although I didn’t know at the time how else it could be done other than re-running test cases.
• Justifying my testing - I had started to build up confidence through my experience of being able to justify my testing and write effective bug reports. While I didn’t have the best language to explain myself, I at least had experience to refer to that gave me a determination to learn better language!
• Determination to understand more about the product, earlier - Typically in my first job, we didn’t have very good documentation to refer to (sometimes none at all!). When we did have documentation, it was much easier to quickly provide feedback on the product. So in future roles, I had a strong determination to get involved earlier in projects so I could understand more about them, much more quickly. I knew documentation was expensive but I felt that trying to access sources of information was crucial to getting my testing really started.
• Thinking ‘outside of the box’ - I quickly learned that the ‘best bugs’ were the ones that people hadn’t even thought of. Sometimes these would be fairly expensive bugs to fix and it was valuable to find them as soon as possible.
• Being organised, fast and clear - I took pride in my work being organised but fast and effective. I quickly understood the need to setup my tests quickly and maximise the amount of time I had available. I created my own techniques for being able to do this, while still providing clear feedback in my reports, which have served me well to this very day!
• Being adaptable and willing to learn - My programming background helped me pick up technical or complex products very quickly and I learned that this can be very valuable. Again, it was useful being able to maximise the time I had available to test and it meant I could provide quick and clear feedback. I then looked for opportunities to learn more skills that could complement my testing efforts.
• Understanding and empathising with the end user - If you’re in a games job, you’ve probably played a lot of video games and so you probably have a pretty good idea of what end users do, what they want and what they like. So it was easier to think like an end-user and therefore I really appreciated the value this brings when coming up with test ideas, justifying your testing and justifying your bug reports. This experience still motivates my desire to always understand more about the psychology of the end users as I highly value it in my testing.

Summary

I was a games tester for 3 years in two different companies and I will never talk about it as a negative experience. That’s not to say that everything was perfect, the games industry has a lot of problems that still affect it today. I left the industry partly because testers are treated as an end-of-cycle role and because I was keen to build my career in testing. However, I still managed to leave and secure a job I would describe as being ‘professional testing’ - I wouldn’t have been able to do this without the experience I gained from games testing. If you are currently testing games and you feel like you’re not a professional tester and have to prove something - you don’t! You’re already a professional tester!

My attempt at the 30 days of testing challenge

Introduction

So just over a month ago, the Ministry of Testing started a challenge called “30 days of testing” and I decided to have a go at tackling it. I had intended to actually achieve it and post all of the progress I made here but I didn’t make much progress in the end.

However, I’ve decided to post the progress anyway because I really liked this initiative and the behaviour it encouraged in the community. Even if I didn’t manage to achieve over half of the challenge, I’d like to encourage others to have a go and share the experiences anyway! If you only achieved one of the challenges, that’s still a good thing, especially if it was helping others to test.

Why didn’t I complete it?

Lame excuse time but basically it was because work and life in general got pretty busy in the second half of last month. I also made the mistake of trying to bundle some of the challenges together - so I was thinking of completing the “mind map”, “5 bugs from a mobile app”, “accessibility bug” and “user experience bug” into one but then spent ages trying to find a mobile app I’d like to explore. I should have just dived into any old app but ended up postponing it and then becoming too busy with work and life in general.

I was also keen on getting a photo of my team but due to illness or holidays I couldn’t get a good time to take a picture of everyone.

But basically I just ran out of time.

What did I achieve?

Day 1 was easy, I already had in mind some books I had been meaning to buy, so this was just an easy motivator to go buy them! I started Dear Evil Tester but I didn’t finish reading it by day 30, so I didn’t successfully complete this one, but at least I got my hands on these books! I’ve enjoyed Dear Evil Tester so far and I’m keen to get started with Explore It too! I definitely recommend both!

Day 2 needs some further explanation (which is why I wanted to write this blog post!) - one of the things I’m working on at the moment is writing some simple scripts to help speed up some aspects of our testing at work. With a shift to a microservices architecture, this provides more opportunities to test much faster at a service level, rather than always at a GUI level. So I’ve been keen to help the testers in my team get on board with this by providing some scripts to quickly generate test data and therefore allow more time to focus on testing a particular feature rather than spending time setting up data or environments in general. Funnily enough, a month after posting this tweet I am automating a bunch more stuff to try and free up more of our time away from the crazy-busy project we are on at the moment.

Day 3 was one that I was looking forward to because I had never considered listening to a podcast before. Well, I had listened to testing podcasts before, but not on any kind of regular basis. The Let’s Talk About Tests Baby! podcast caught my eye as the topic of telling stories in testing resonated with me as I’ve always felt communication is critical in testing, starting from learning a project to reporting a bug. I find myself studying the language, social interaction and psychology aspects of testing more and more lately. I liked that the podcast itself was short but thought provoking - definitely worth a listen if you’ve got a spare 15 or so minutes!

Day 4 I decided to trade blogs with a great developer at work, I didn’t like the idea of just telling someone to read a blog, I felt I should reciprocate and I like the idea of understanding what other people read too! So I shared Michael Bolton’s blog, particularly this post on what is a tester because I feel it sums up how I view my own testing. In return I got Joel Spolsky’s blog which was quite an illuminating view on how perhaps many developers feel about their work too.

Day 5 I basically thanked Rosie Hamilton for all of her hard work on the testing survey she has conducted the last few months. I think we all make a lot of generalisations and assumptions on what the state of the wider testing community and I really value any work done to try and provide some data to consider. The more that people do this kind of thing the better, because it means we can hopefully start to measure the difference we could make by constantly improving the testing community. I’d like to see the results of improving come out in data like this. Or if not, at least we can try and learn and understand why not. I also like the theme of talking about our backgrounds in testing because I think we all bring a variety of different backgrounds and skills that we use in our day-to-day testing!

Day 6 I really struggled with because I had no idea what could be described as a “crazy” test (I generally take the word ‘crazy’ to mean something non-sensical or illogical). I decided it would be “crazy” to test weather forecasting due to how unpredictable it can be, or so I thought! I made a record of what weather accuweather predicted for particular hours in 2 days time and checked if it was correct. Turns out it was pretty accurate, it had predicted cloudy weather for most of the day, rain showers at 11am, 1pm and 3pm and it did indeed rain at those specific moments. Or maybe it was just lucky that I live in Manchester and that is pretty regular weather around here haha.

Day 7 I did a little research and found this neat little tool for tweaking the colours on websites to account for colour blindness. I found a couple of places on this blog that needed tweaking (specifically the main one was the shades of blues for the link text have been ever so slightly changed so they are easier to read on a white background).

I also managed to complete days 10 and 16 - the former I heard about Ministry of Testing’s “Masters of the Ministry” and I’ve booked myself into Richard Bradshaw’s awesome-looking week of workshops. The latter I attended a non-testing event in the shape of Manchester Tech Nights, partly because I’ve attended before and was curious to see what the numbers of testers were like and also because the head of HR at ResponseTap, Rebecca Taylor, was speaking.

What I almost achieved or technically did achieve

• Number 14 “Step out of your comfort zone” - I’m hoping to do this soon by appearing on the Let’s Talk About Tests, Baby! Podcast! I hate listening to myself so definitely not comfortable with it, but looking forward to giving it a good go anyway! I think I might enjoy it though!
• Number 19 “Find and use a new tool” - I think I technically achieved this with the colour blindness tool, but I wanted to really use this challenge to get my teeth into one of the many tools I keep seeing mentioned from time to time around the testing community.
• Number 21 “Pair test with someone” I probably achieved as well because in my current role I try to do this as much as possible anyway. I did pair test with a developer last week in fact.
• Number 26 “Invite a non-tester to a testing event” I regularly keep plugging TestBash at every meetup I attend so I’ve invited a lot of them technically!

No more excuses, here’s some I can at least achieve right here

• Number 17 “Find and share a quote that inspires you” - “At the simplest level, people are doing testing all the time. You often run and analyze tests without even knowing it.” - Gerald Weinberg Perfect Software and other illusions about testing, 2008.
• Number 22 “Share your favourite testing tool” - well the tool I use the most is a pen and notepad, but my favourite testing tool is probably the programming language Python. I write a lot of tools for my own testing with it and I love how versatile it is thanks to its popularity. I don’t think I like any tools that describe themselves as testing tools, I’ve not used one that I’ve liked - for example TestLink or DevTrack.
• Number 27 - “Say something nice about the thing you just tested”. Ok so I’ve recently been testing a particular new application at work and I love how easy it is to test it in isolation. It’s a service that consumes messages from other applications so its reliant on these messages in order to test its logic. However, the developers helpfully included some API endpoints to simulate the messages so where necessary it was possible to test the logic of the service in isolation quite easily. It has meant we can test some things a little earlier and quicker than waiting for a fully integrated environment (although obviously we still need to test that too).

It was fun!

I’m a little annoyed that I didn’t complete the challenge but I can of course still do some of them later. I definitely will be trying to keep up some of the good habits from this challenge like commenting on blogs, listening to podcasts and occasionally testing other people’s stuff and sending them helpful bug reports.