Best of the BSides - A friendly security conference in Manchester

Introduction

Today I attended a great little conference in Manchester called BSides Manchester. This was a free conference about security ran by members of the security community in a similar way to TestBash. In fact the whole event was a bit of a “SecurityBash” in so many respects, which is awesome and I recognised many familiar topics, concerns and ideas. Whether you're experienced with security or a newbie, I highly recommend this conference. I went along with no expectations, just hoping to learn as much as I could, expose my brain to new ideas and even if I didn’t pick it all up immediately, it could give my brain a place to start. Not only did I actually learn quite a bit but I also noticed that there was a great deal of similarities to testing so I thought I’d talk about the conference from that angle.

The similarities and parallels to testing

In no particular order:

• The security community seem to be very keen to promote leaner and more effective ways of improving security such as getting involved earlier and trying to be involved in discussions about new projects or approaches. This is exactly the same as with testing in general and both are frustrated when they are only asked for their opinion very late in projects. Perhaps this is the biggest area we share in common and maybe we could share our experiences and lessons with each other. Perhaps we can also be allies on this, for example where a tester has managed to get involved in the project early, we could be advocates for involving security professionals earlier too and vice versa.
• Carolyn Yates gave a great talk on the bowtie method which is very applicable to testing too and reminds me of how we look to use examples like mind-maps to effectively visualise our work. She also made the point that “not all tools need be programs, sometimes they can be visual aids” which I think as testers we can certainly appreciate too.
• There was a great talk by Collette Weston about echo chambers - in particular the difficulty for women and other industry minorities to break into the InfoSec industry and community and what can be done about it. I think we can all agree this is an issue across the software industry as a whole too and while I feel testing is a little bit better in this regard, it’s definitely not as good as it could be. This talk also prompted a great discussion about how some companies had started trying to diversify their security personnel (including hiring people with biology degrees) and I know in testing its well appreciated that we benefit greatly for our diverse backgrounds.
• In two separate talks by Ian Trump and Charl Van Der Walt there were discussions of what the future might hold and how artificial intelligence and the advance of technology will shape the industry and the work of security professionals. It seems obvious but I found it quite re-assuring to know that it’s not just testers who are wondering how these advances will affect their jobs. There was also discussion of the effects of automation and whether people were really considering these effects on the loss of jobs and how humans interact and use the automation. This echoes the concerns I’ve heard many testers raise and reminds me of my old blog post on this subject.
• Naturally there were several more technical talks focusing on particular types of hacks, attacks and penetration tests. This included discussions of how to defend against these attacks too. The mindsets and techniques that security professionals use to find and report these exploits is exactly the same as how testers find and report bugs. I think we have a lot in common on this subject (as, well, it is a form of testing) and I think we could do more to engage with the security community and share our experience - just as much as we can learn a lot from them too! All the things we talk about in testing were present here - such as trying to turn exploits into the most damaging problem they could find to justify and explain to companies why they need to fix it. I believe as testers we can also become more effective at general testing by learning about these exploits too - both in helping raise security issues earlier but also giving us more ideas for other kinds of testing. Perhaps we could share our knowledge, approaches and experience of exploratory testing with them.
• Another common theme of the conference was that security was not really a technological problem, but a people problem. This is of course not a new revelation, there are many historical quotes and philosophical discussions, for example, “a bad workman blames his tools”, “pick the right tool for the job” and so on. However, as humans we clearly find it difficult to keep these lessons in mind and it is easy with bias to miss that we are making assumptions about our problems. As testers I feel we should be very aware of this too and typically many challenges we face are nothing to do with the particular technologies involved. Many software bugs are caused by humans and machines are simply doing as they are told, the same applies for security exploits.

The differences

Of course, for all our similarities, there are also differences:

• As part of the discussion about diversity in the industry from Collette’s talk, there was also discussion about autism and how there was a general belief that many “black hats” may have struggled at school, dropped out and only picked up hacking because they had no other options. It was pointed out that because many companies require specific levels of education (such as GCSEs), it meant there was no way for these individuals to become security professionals. Why is this different to testing? Well in the testing industry I don’t feel we have such a specific concern with autism (though it will definitely also affect the testing industry and community too!), I feel our concerns are more about increasing awareness of testing as a possible career in the first place!
• I think this one is probably obvious but the security community is more naturally technically focused and capable, in tandem with the above point, most people seem to join the industry because of their interest in it and interest in technology. As such, while there is diversity, I get the general impression the diversity of backgrounds is a lot more acute as opposed to the very broad backgrounds of testers. As a result I feel testers tend to be less technically focused and more a balanced spread of soft skills to go with the technical subjects. That said, the conference did feature plenty of talks that were more about the soft skills, although probably a different balance compared to some testing conferences.
• I feel that the security community is even more aware of justifying their testing and explaining the effects of the exploits they find than the average tester because of both the ethical act of the testing and it’s very technical nature. Not only must they be very careful in not breaking laws or damaging a company but they also have to be very good at explaining why they think something is a significant problem and helping the company fix it. I think as testers we have a lot we can learn from this, not because we don’t do a good job of this, but our testing is a lot safer and doesn’t always require as much explanation. However, I think this will change over time if we get more involved with DevOps, challenging requirements and testing in production.

You should go to!

All in all, it was a great conference, I took a lot away and enjoyed myself. It was very reassuring to see so many similarities to testing and seeing ways in which we could work together. I hope to go to some other conferences in other areas around software development like Programming, Project Management, UX, Business Analysis, Operations and Systems Administration and continue learning from them. Maybe even to begin talking about testing at their meetups and conferences and see more sharing across our disciplines.

Mob Programming with Woody Zuill

Introduction

Today I attended a 1-day workshop organised and run by Woody Zuill who, along with his colleagues at Hunter Industries, originally developed the idea of ‘mob programming’. This workshop was part of a major conference being held in Manchester called Agile Manchester and I heard about it after attending a meetup last month called Manchester Tech Nights. I had been invited to that meetup to represent testing with a 5 minute lightning talk. It was a great experience and even better that I found out about this workshop and conference from talking to people there! After finding out, I just had to book my ticket and attend, it seemed like a great opportunity to really understand what mob programming was all about without having to travel very far.

What on earth is ‘mob programming’?

Basically, it’s the idea of taking pair programming and applying it to the whole team. So the whole team works on a single piece of work at a time, with a driver and navigator rotating at timed intervals. The proposed benefits of this idea is that when we work separately, we deliver both the best and worst of our work whereas with the whole team present, we support each other and produce the best of our work at all times. You can read a much better written description by Woody himself here.

Why was I interested in this workshop?

Barely a year ago I only just started to explore the wider testing community and start reading and listening to different experiences and ideas on testing. Through that research I came across Maaret Pyhäjärvi’s blog and her talks from previous TestBash conferences on the Ministry of Testing’s Dojo. Some of the particular topics that caught my interest were on her recent experiences of working as the sole tester within a large group of developers and the challenges in managing this situation. Through her talks and blogs she mentioned the idea of mob programming (or mob testing) as a means to both learn very quickly but also cultivate understanding and respect across a cross-functional team. My curiosity was really piqued when I had an all too brief taste of pairing with Llewelyn Falco as well hearing his lightning talk on the benefits of mobbing at TestBash Brighton 2016.

I’ve started to experience the need to adapt testing to become more integrated in development processes in agile environments and mob programming struck me as a potentially useful way to encourage the spirit of not only developers and testers working together, but also the wider team, be that product owners, designers or scrum masters. The workshop presented a great (and well timed!) opportunity to observe how it might be done and get a better idea of why exactly Woody and his team back at Hunter Industries even came up with this approach.

The workshop and what I learned

The workshop itself was the kind of friendly, fun and relaxed workshop I’ve come to expect after the high standards set for me back at TestBash! Woody was very honest and open in explaining that he was only here to explain what mob programming is and how it has worked for him in the past. He didn’t claim that this approach would work for everybody. The workshop was split into two parts - the first part was where Woody explained the context of how he and his developed this idea and we took part in exercises to observe the benefits of working together as a team. The second part was where we actually wrote code as a mob!

The main reasons that led to mob programming as an approach being developed were:

• Originally Woody wanted to encourage his developers to recognise bad code. He wanted to achieve this by allowing people to figure things out and practice their craft together regularly - in the same way as any sports team.
• He also wanted to overcome the challenge of getting access to project managers to answer questions that blocked work - involving them in the mob helped provide immediate feedback to any questions the developers had.
• However, to achieve this, he didn’t force people to participate, everyone was invited but attending was not compulsory. This allowed people to get comfortable with the idea before they attended.
• Hence, the only reason the approach of mobbing worked was because the team decided to do it. It was not forced and was developed simply by trying ideas and trying to solve problems when they arose.

I really liked how open and honest this explanation was of the context and how this approach was developed. It was exactly what I was looking to learn by attending this workshop and considering how it might be useful (or might not be useful!) back at my workplace or in future.

Coding again!

Although I’ve been writing a fair bit of Python the last 3 years I haven’t really touched any more advanced programming than that for about 6 years (since my degree basically!). I was relishing attending this workshop and throwing myself in with a group of experienced programmers and trying to keep up. Having read and heard Maaret’s experiences with mob programming as a tester and how it was a great way to learn, I really wanted to experience this and see this benefit for myself. Not to mention, to hopefully contribute something with my skills as a tester. Having said that, I was reasonably confident that I could somewhat keep up due to my background in my degree and my recent Python experience.

It was a lot of fun! I didn’t feel too ashamed by my rustiness with Java and I started getting back into it pretty quickly. It was awesome to see how quickly we started interacting as teams, at first I was concerned that having less experienced programmers (or people who didn’t know any code at all!) would drag the team down. But I actually felt the less experienced programmers got up to speed remarkably quickly! It’s really quite impressive (and not at all surprising when you think about it!) how quickly people can pick things up when they’re thrown in the deep end but given the support to perform.

Woody was a great facilitator and I learned a fair few tips on how to conduct these sessions successfully - the main points being:

• The hardest part is to shut up and let everyone have an equal say! Treating everyone with consideration and respect was key, so that the whole team is on the same page.
• The key part that I think testers (and non-technical people) really bring a lot of value to mobs is focusing the team on considering each task in the language of plain english rather than in ‘coding language’. There was an emphasis on navigators giving instructions in english rather than specifying code. Although it may be necessary to support non-technical people to guide them through typing specific characters, it means as a tester you can easily navigate as you can focus on the objective at hand rather than technically how to solve a problem.

What does this all have to do with testing?

I believe that in agile environments and with ideas like continuous development it is generally no longer very effective to conduct all (note - perhaps some is still necessary) of your testing at the end of a development cycle or separately to development. Not only does this quickly lead to ‘testing bottlenecks’ but it just doesn’t make a lot of sense to be discovering basic flaws in logic or implementation only after development has finished. We could be bringing our skills as testers much earlier in the development cycle and discovering these flaws earlier and more cheaply. If we can challenge assumptions made before the code is even written, or even as the code is written, we will surely prevent some of the major re-factoring or re-writes that sometimes occur.

I personally see mob programming as a potential tool for encouraging much better collaboration between developers and testers as well as product owners and designers. The idea that ‘everyone is a developer’ who simply bring different skills is something that makes a lot of sense to me and I believe our testing can be become much more effective when we understand more about the code we test.

However, as always, maintaining critical distance and trying to always be an advocate for the customer is key to our work. But I strongly believe we can bring those values as well as learn and contribute more from a technical perspective without compromising those values.

What next?

Now that I’m armed with some better knowledge of the context of how mob programming came about and some experience of what it’s like to be a part of and facilitate, I’ll carefully consider it as a tool to suggest in future. I can see that if a team is already collaborating and working well together, pairing and mobbing in an informal sense, that this approach may not be necessary. However, I’m looking to build a healthy stock of ideas and tools to try when the time arises and if nothing else it’s always useful to try these ideas and see if there is anything we like about them.

In the scenario of rapidly sharing knowledge about a particular technology or language, I think this formal approach of mob programming could be very very effective and I will be looking out for opportunities to do this.

In summary, I highly recommend this workshop if you’re interested in the subject! As I’ve said a lot in this blog post, Woody is great and I got everything out of the workshop that I wanted to.

The role of a tester in backlog grooming and planning

Backlog grooming

As an on-going activity, the product owner and the scrum team should be actively reviewing their backlog and ensuring the work is appropriately prioritised and contains clear information. This allows each piece of work to be more easily planned into a sprint as it can be more accurately estimated.

As a tester I actively try to be involved in this process as it is my first opportunity to assess the requirements and the information provided. It also allows me a chance to gather information required for testing, which allows me to provide more reliable estimates.

The objective is to be in a position where for any piece of work presented in planning you know exactly what the work requires. You should then have a good idea of what you will test and therefore provide reliable estimates. If this is not the case, then the work cannot be effectively planned into the sprint.

Planning and estimation

At the start of each sprint, you have a sprint planning meeting. In this meeting the team collectively decide what work they can commit to being done by the end of the sprint. This may or may not have an estimation process. This is your last opportunity as a tester to ensure that you have all of the information you need before work is started. If the appropriate backlog grooming has been done, then this should be a relatively straight forward process, however inevitably there will be pieces of work that need clarification or may have missed something.

Some example typical thoughts I have in a planning meeting for a piece of work are as follows:

• Do I understand the requirements?
• Does everyone else understand the requirements? (and does their understanding match mine?)
• What requirements have not been written down?
• Are there any external factors such as legal requirements or third parties?
• How can I test this work? Is it even testable?
• Do I need any additional tools to test this work?
• Do I have any dependencies in order to test this work, such as needing live data or having to work directly with a customer or third party?
• Does this work conflict with anything else in this sprint?
• Does this work conflict with work being done by other teams?
• Are there any repetitive checks that I could automate for this work?
• Do I need to consider other forms of testing such as security or performance testing?
• Is the balance of development workload to testing workload viable?
  
  

Hopefully I should have asked most of these questions during backlog grooming, and I wouldn’t always ask all of these questions, it very much depends on the context of the work. But hopefully this demonstrates that you can easily think of a lot of questions and it is important to ask these before and during planning.

Once you have the answers to all of these questions, you should have a good idea of what you are going to test. From this you can provide a more reliable estimate of how much testing you would like to do. Not only should you have a good idea of how long it would take, but also you should be better equipped to analyse risk.

Summary

• Planning a sprint is easier with clearly defined work and when the team has prepared for the planning meeting.
• To achieve this, backlog grooming should be used to ensure tickets are prioritised appropriately and contain enough information.
• Backlog grooming should also be used to prepare for planning such as considering if you need testing tools or environments.

Testers in development teams

Introduction

For this blog post I’m going to talk about my experiences as a tester in a development team working to “scrum” which is an agile development methodology. I write this as someone who has worked to more traditional waterfall style processes prior to working in scrum and as someone who has worked for several years within scrum. I’ve also worked to a “kanban” model too but I won’t cover the pros and cons of the other methodologies here. The goal is to hopefully help someone in a similar situation to me have a clearer idea of how they can fit as a tester into scrum.

I’m going to assume you know the basics of how scrum works but if you are new or unsure about it, please have a read of the wikipedia article first:

https://en.wikipedia.org/wiki/Scrum_(software_development)

I would also add that my thoughts on this topic are from the view of an agile project, where speed is absolutely key. Most of these thoughts would possibly not apply to a project where speed is not desired, though I feel the role of a tester and the skills they use are the same no matter the situation.

Should a development team include a tester?

I’m going to start by tackling this first. You may notice in the link above that there is no distinction between team members in a scrum development team. A common view that I have come across before is that testing should be a process external to development. There are some that feel that testers shouldn’t become too close technically with the system they are testing because they may make some conscious or subconscious assumptions, because of their heightened insight. There is a fear that testers would lose their ability to think like an end user and would think too much like a developer.

For me, this is nonsense, I consider it my job to be aware of end users and to think beyond what I know or see. I have found it to be faster and more efficient in general to work as closely as I can with developers and I have built a much better rapport with them. This has meant that the levels of trust and understanding are much higher and developers see the value in testing. As soon as testers and testing are perceived to be a separate process, I have found the two communicate less and become more frustrated with each other.

In scrum this is crucial as you don’t have the time to waste with the traditional methods. This means that a tester cannot sit outside of development and only test pieces of work when they are complete. This would lead to development work being completed in a sprint, defects being found after a sprint and then the development work having to be revisited in a new sprint. This is obviously not an efficient way of working.

So for me, a tester must sit within a sprint team and be considered part of the development team. While their skills and role is different to a developer, they are an important contributor to considering a piece of work “done”. It is also important to realise that developers are capable of contributing to the test effort. While automation tests do not replace manual exploratory testing, they are very useful for taking the heavy load of more focused tests. Developers can be an alternative resource for constructing automation scripts, expanding upon their unit and integration tests with more advanced Selenium UI tests for example.  Testers should view themselves as being responsible for improving the quality of the work and not as an executor of tests. If tests can be easily automated by developers, then it doesn’t always have to be the tester who creates and executes the automation.

Developers can also provide assistance and help with setting up more intricate or complex tests, reproducing problems and also providing more technical explanations about how the system functions. This can be an invaluable source of information if you have little documentation or wider business knowledge about the system (though it should not be relied upon on its own).

At the same time, a tester can do more than simply test work as it is completed. In some cases, it is possible and very useful to test pieces of work while they are still being developed. Working in close collaboration with the developer, a tester can provide quick and effective feedback on the work and really focus the developer on producing quality work. The tester can also continue to perform preparation activities such as gathering any additional information, ensuring they have the appropriate tools and environments or planning their testing.

I personally feel a tester doesn’t just bring an ability to exploratory test, but also to gather knowledge, critically challenge designs and re-focus a team on what is important to deliver. These skills are best put to practice before the sprint even begins through to finishing the work, rather than after the work has been completed - where it is too late to really reap the benefits of these skills.

Summary

• In Scrum, testers are most effective when they are part of a team of developers.
• A collaborating team is far more productive than separated focused individuals in short time periods.
• Developers can contribute to developing and supporting automation testing.
• Testers should actively ensure they are not becoming too close to a system and are aware of external areas to the team - such as end users, other systems, business concerns, legal requirements, etc.