Microservices discussion by XP Manchester

Introduction

A couple of weeks ago, I was invited by some fellow programmers to attend an event on microservices organised by XP Manchester. Microservices are a hot topic right now in software development so I wanted to go along partly from my own interest in the subject area but mainly to think about how testing may be impacted or what considerations there may need to be for testing. The event was focused on programming and software architecture but it's discussional format allowed for questions so a variety of different points were talked about through the evening.

What is a “microservice”?

The conclusion from the evening was there is no agreed definition! However, I think we can summarise that microservices as an architecture is the process of breaking down your system into smaller parts. The main reason you want to do this is for scalability, but it is an expensive process and the conclusion for the evening was “Microservices are very hard!”.

What does testing have to do with this?

I had initially gone to the event hoping to understand better some of the implications on testing. But I actually found myself taking a step back and observing the meet up and the discussion from a philosophical view. I actually found a lot of parallels to the debates on automation testing in the testing world. So while the content of the discussion had little to do directly with testing, I think there are some lessons that apply to automation testing and many other “hot topics”.

The temptation to try something new - the fallacy of “best practice”

One of the points raised on microservices was that as a concept it had been around for several decades, but only very recently has it become a popular subject. This is mainly down to the famous use cases of Netflix and Spotify. It seems it is very easy for people to see companies such as these and want to copy their models. The problem is, solutions like microservices are very expensive and complex, they are solutions to very particular problems - they are too expensive as a solution to be used at all times. It is tempting for people to consider them a “best practice”, which is totally inappropriate. I see the same kind of attitudes on automation testing too - that large companies talk about automation testing and everyone else decides to follow it as a best practice. Automation testing is also very expensive and is not a best practice, it is a solution to a particular problem. I also see automation testing get discussed as the “best thing to do” in a similar vein to microservices.

At the event, someone mentioned a great analogy - you wouldn’t use the same design and manufacturing methods to build a model plane as you would a real Jumbo Jet. Just because famous or bigger companies use particular solutions, doesn’t mean these solutions are appropriate for your situation.

Only looking at the benefits, without considering the cost

Another point that I could relate to is the belief from some people that microservices is easier and simpler - that by breaking down your monolithic code base, you’re breaking down the complexity. This is false, the complexity is always still there, it’s just been spread around - which makes it both easier and harder to manage in different respects. While a particular area of code is easier to manage in isolation, the overall integration or full system is much harder to manage in terms of infrastructure, deployment and debugging.

I see the same problem in automation testing - a common view I’ve come across is that automation testing is always quicker than a human manually typing on a keyboard. Just like with microservices, people are ignoring the bigger picture here - focusing on the speed of executing a test, rather than considering what you gain and lose in the wider process. Automation testing is more than just executing a test, there is a lot of work to do before and after the execution! The cost of automation testing is the time it takes to write the test up, analyse its results every time its run and the maintenance cost of the test. With a human manual tester, the cost of writing the test is massively reduced because you are not writing code - in some cases perhaps nothing needs to be written! Analysing the results can be much quicker for a human to do - they are both able to run the test themselves, notice irregularities and analyse all at the same time - something a computer cannot do. Maintenance is also a lot less for manual testing because a human can adapt to a new situation easily.

Because microservices and automation testing are very expensive, the cost must be weighed up against the benefits. Only if the benefits outweigh this cost does it make sense. Typically, the value in automation testing comes from repeatable activities - and over time this will overcome the high costs. But for anything that isn’t repeatable, it’s difficult to really justify automation over simply carrying out the testing manually.

Additional thoughts

On a different note, I’d also like to talk a little about how the event was organised by XP Manchester as I felt it was a very successful format that I hadn’t experienced before. The format was one where everyone was asked to sit in a circle with 5 chairs in the middle of the circle. 4 people would sit on the chairs, leaving one vacant and discuss the topic (with the discussion being guided by a moderator). If someone wanted to join the discussion, they would sit on the vacant 5th chair and someone else from the discussion must leave. Meanwhile, the rest of us in the circle had to remain silent and listen. I felt this format was fantastic for keeping a focused discussion while allowing 30 people to be involved or listen to it. It was a refreshing change from traditional lecturing approaches or just a chaotic discussion with 30 people talking to each other at once. In some respects it was the best of both worlds. Credit to the guys at XP Manchester for running a great little event that produced some useful intellectual discussion!

Summary

• There are definitely a lot of relatable problems when it comes to decision making for programmers, as there are for testers.
• Don’t be tempted to follow a “best practice” or “industry standard” without considering whether it is right for you.
• Always consider the costs of your decisions, always treat so called “silver bullet solutions” with suspicion - is this solution really the best? Is it really as easy as people are suggesting?
• For groups of 30ish people, if you want to generate a focused, intellectual discussion for people to listen and learn from but don’t want to use a lecture/seminar format - then consider the format described above!

Sec-1 Penetration Testing Seminar

Introduction

Recently I was invited to a seminar by Sec-1 on Penetration Testing. It was a great introduction to the discipline and I took away quite a few really great points. I’ve never really performed Penetration Testing myself and I only have a general knowledge of it - enough to be able to identify and understand some basic problems. I’m looking to expand my knowledge of this type of testing so that I can bring much more value to my functional testing. In this blog post I will talk about the main points that I took away from the seminar.

You are only as secure as your weakest link

You may spend a lot of time securing one particular application server but if you have just one older or less secure server on the same network, your security is only as strong as that one weak sever. There may be an old server that is used for your printer in the office and it may not be considered as a security concern, but it can be used by hackers to gain access and compromise your network.

Whitelist, don’t blacklist

If you write rules that attempt to prevent specific malicious interactions, you end up with a maintenance nightmare by continually having to maintain and update these rules for each new threat. It is much more effective to instead write your rules to expect the correct interaction.

Keep up to date as much as possible

It can be a pain to keep libraries, servers and software up to date because the updates can break existing application functionality. But where possible it should be encouraged because these updates may contain important security updates. However, you cannot rely on the change logs for the update to tell you about any important security fixes because typically they are described as “minor bug fixes”. Companies do this because it can be considered bad publicity to admit they are regularly fixing security holes.

Keeping up to date will save time in the long run as your systems will become more secure without you needing to create your own security fixes later for systems you have no updated.

Only a small amount of exploits are needed to cause major vulnerabilities

At the seminar they demonstrated a variety of techniques that could be used to get just enough information to open larger vulnerabilities. Through SQL injection, a poorly secured form could provide full access to your database - allowing hackers to potentially dump your entire database which they can then use to gain further access to your network or sell on to other malicious entities.

Attacks do not have to be direct

Even if your own system is highly secure, hackers can target totally independent systems such as targeting websites your employees visit and gathering password data. Typically a lot of people still re-use passwords and this can be an alternative way into your system. In the same vein, you are open to attack through integration partners if their own systems are not as secure as yours. Again, you are only as strong as your weakest link.

Summary

• I found the seminar useful and I certainly learnt a lot more. I can certainly recommend Sec-1’s seminars to anyone who only has a general knowledge of penetration testing and wants to understand more.
• Keeping software and hardware up to date has more benefits to security than it may first appear because security fixes are not always made public knowledge.
• Penetration testing requires specialist skills and knowledge. However, I feel there is still some worth to having a better understanding as a functional tester. It allows me to pick up on potential areas that may concern security and helps me to drive quality in terms of security by challenging lax attitudes on these “minor issues”.

Important defects or significant information?

Introduction

As a tester I feel I am a provider of information, information that allows others to best judge quality and risk. If this is correct, should I merely report all information with no care for judging importance or priority? If I filter the information to what I think is important, surely I'm influencing the decision process of others? I feel this is, as ever, a murky gray area that has no easy answers.

Who decides importance?

Project Managers, Product Owners, Business Analysts, stakeholders - whoever determines what is worked on - decides importance. There is absolutely no question of that. As a tester I am not the one who decides what is worked on. I am not usually in a position where I am in conversation with the entire business or have sufficient knowledge of ‘the big picture’ to make these decisions and it isn't the job I'm hired to do. Of course, there may be circumstances where these jobs are blurred (there are Project Managers who test). Still, in a typical company set up, I'm rarely hired as a tester to manage projects.

However, that doesn't mean as a tester I can’t have some knowledge of the wider project or business concerns. It vastly improves my testing ability to have this knowledge! So, I do have the information to assist others in measuring or deciding importance. I am a gatherer of information, and it is how I communicate this which is all important. In doing this, I will need to use careful language to ensure that I am not under-emphasising or over-emphasising particular parts of that information.

For example, a product owner is gathering metrics on how a system is used. They measure how often particular features are used by customers and decide how important bugs or problems affecting those features are, based on this metric. If I have found a critical bug with a feature I have to be very careful to highlight and justify the critical nature of the bug. If I only described the bug as “There is a problem with this feature”, the product owner may choose to dismiss the bug if it’s a relatively low-use feature. But what if the bug corrupts the database? This surely has to be fixed? This is why careful use of language is important. A tester needs to understand the significance of the information they have gathered; and convey this in a balanced way

Informing not blocking

So if a tester helps decide significance, how far should they go in justifying significance? This is the tricky part - you must ensure you are not blocking the business, or even more importantly, you must ensure you are not seen as blocking the business. You need to balance providing useful information, which allows others to make decisions, with the language you use in describing that information. For example, if I find a defect which I believe is very significant and I feel I need to highlight, I could use the following kind of language:

“There are lots of important defects, affecting all sorts of areas. We must fix them immediately and we cannot release until they are fixed!”

This is very poor use of language and isn't providing any useful information. Firstly, I'm declaring the importance with finality - I am not the one to judge the importance of the defects. Secondly, I'm telling people what to do and then failing to provide any justification. This also gives an impression that I am demanding the defect is fixed.

Now consider if I worded it like this:

“There are two defects that are significant. The web application server fails to start because it is missing a configuration file and the database updates have deleted the accounts table. I also have a further list of defects that I think are significant but these two I think need attention first.”

Here I am highlighting what I know are significant defects and providing information about them so that people can make their own conclusions. By highlighting the significance, I focus people’s attention on those defects. By providing summaries of the defects, I allow people to make their own judgement of whether the defects are really important. So here I am not declaring importance, but I'm suggesting a course of action backed up by the information I have. This helps promote an image of my testing as being informative, rather than demanding.

And the blocking? Surely I can’t let a defect go live?!

Its also important to therefore have an attitude which allows you to accept and understand the decision that is made. Even when you have presented this information, the business may decide to accept the defect and not fix it. At this point you have done your job and should not feel responsible for the decision. Testers are not the gatekeepers for defects going live. Testers are more like spies - gathering intelligence to inform decisions made from a strategic level. Just like a spy, you may commit significant time and energy to delivering information you may have felt is significant - sometimes beyond what you were asked to do. But the spy doesn't act on the information, they merely deliver it. 007 is not a good spy in this respect, he’s trying to defeat the defect all by himself and it may only be another henchman. We want to be a team player, provide information to others and help attack the boss pulling the strings!

So I just let someone else decide, so I shouldn't care?

No you absolutely should care! You should be passionate about quality and take pleasure in delivering clear and accurate information to the relevant people. Recognising someone else takes the decision is not a sign that your information, your work, doesn't matter. Its a sign that there is more to consider than one viewpoint - the business or organisation cares about the wider view. But it can only make the best decisions if the smaller views that are communicated up are given care and attention. The value of any group must surely be the sum of its parts, so by caring you are implicitly helping the wider group care.

Summary

• Testers don’t decide importance, however, they can influence importance by providing information on significance.
• The language you use shapes how others perceive significance, so your words must be carefully chosen - they should be objective, not subjective.
• Testers should help the wider business to make informed decisions, not become the gatekeepers for defects.