Introduction
Last week I ran my first ever workshop at a conference for TestBash Manchester! It was an awesome experience, totally different to the talks I’ve done before at meetups and smaller conferences. The workshop was all about how to get started with web API testing and I targeted it at beginners who had no prior knowledge of APIs. For this post though, I’m not going to talk about the workshop, but more about what happens next. Several people have asked me about a more advanced workshop and what they could study in their own time next. I don’t have a quick or easy answer to this as I feel there is lots more to learn and it really depends on your context. However, I’m going to try and discuss some areas and ideas.
Where are we starting from?
Before I get started, I’d like to clarify where my workshop left people and what this post is assuming you’re already familiar with.
- What APIs are.
- What do APIs look like and how they work.
- Why they are useful to understand for testing.
- What API documentation looks like.
- What paths and query strings are.
- How methods work.
- What status codes mean.
- The concepts of authentication and authorisation.
- How to create requests with authentication using Basic Auth.
- The concept of resources and IDs.
- What headers are.
- What request and response bodies are.
- An introduction to JSON & XML data formats and data types.
- Using Postman’s basic features.
- Understanding how Postman collections could be used.
- Awareness of how basic automation can be created using Postman’s runner.
If you feel lacking in these areas, I would still spend more time on understanding the basics of these before starting on anything more advanced.
So with that, I’ll go over some areas that we could explore for a more advanced workshop or for you to explore in your own time.
Try testing other, more complex APIs
On my workshop, we learned to interact with a simple API that myself and my friend Lee Goodman built together. This API was intentionally designed in a way that allowed attendees to learn in stages, introducing different concepts at each stage. However, this is not how a API will look in reality, you won’t have nice ways to learn it in stages, they will immediately throw everything at you. You typically won’t be able to interact with most APIs without authentication (which will be more complex and won’t be the same for every API) and they will provide varying levels of quality of documentation.
One of the aspects of APIs that I didn’t cover in my workshop is that they are an abstract representation of the resources, objects, functions and capabilities of an application or system. In plain English this means when you learn to use an application via an API, the picture you build up in your head based on the API’s structure and responses is based on a translation of the system underneath the API. Just as with translating from Japanese to English, there are concepts that are not easy or even possible to express in an API. People also make mistakes in translation or have many different ways of creating the translation. It’s useful to get some experience of this by using more APIs, you may start to notice these differences and get a feel for what works well, what doesn’t and perhaps get some sense of the compromises made. You will also see where some of the language that even I use is not consistent across all APIs.
You can try out various public APIs for free, one example being twitter’s API. You can find documentation for lots of public APIs you can try here:
There is also simpler and neater API to to play with produced by Swagger here:
Learn about more complex forms of authentication
In the workshop I only covered the basics of how to authenticate your requests in Postman with one of simplest forms of authentication called Basic Auth. There are many many more types and technologies for authentication that you could learn about, some of them are very complicated and deserve an entire workshop in themselves! I don’t feel it’s necessary to understand them all because you probably won’t come across many of them. But it could be useful to understand the more popular (and secure) kinds of authentication such as OAuth 1.0 and OAuth 2.0.
Learn about different kinds of headers
I briefly talked about headers in the workshop, mainly in reference to the “Authorization” header (which is the one Postman creates for you when you add authentication) and the “Content-type” header which we used in the workshop to tell the server whether we were sending JSON or XML with our POST and PUT requests (again automatically generated by Postman). There are a couple of more headers that you can experiment with when sending requests such as the “Accept” header which can tell the server to return responses in a different format, in the same was the Content-type header. This means you can do weird stuff like send JSON data but demand the response is in XML. I have accidentally killed servers in the past with typos in my headers too! You can read more about different kinds of HTTP headers here.
Learn to use the more advanced features of Postman
Postman has a lot of neat features which can be used to augment your testing in different ways. Learning to use collections can allow you to create documentation of an API you’re exploring which you can share with other testers and developers (particularly useful when a new person starts on a project, you can give them a collection to get them up and running much faster). If you’re finding yourself repeating the same requests a lot, especially to create data, using Postman’s collection runner can allow you to create automated scripts of requests that quickly generate test data for you.
You can further extend the capabilities of your collections to automation checks which can be rapidly run and tell you if the API you are testing is ready for deeper exploratory testing or if there is something significantly wrong. You can do this by using Postman’s test scripts function. While these tests are written in Javascript, it’s possible to write these scripts with little knowledge of Javascript using the example snippets. However, it may be helpful to learn a little bit about Javascript to get the most out of these scripts. You can learn Javascript via sites like this one, which is a free 30 day coding challenge.
Combined with Postman’s pre-request script functionality, you can then create more complex collections using functions such as loops and branching. In addition to these, you can also learn about environments and variables, which let you parameterise data that needs to change every time you run a request. The most common examples of using environments is where you have multiple test environments with different domain names (e.g. wwww.live.test.com & www.stage.test.com) but you don’t want to keep re-writing the requests.
These features can allow you to chain requests together, so rather than manually copying an ID from one request to use in another, Postman can run the two requests together for you. This is a good blog post explaining how to do this.
Try integrating your Postman collections as part of a CI pipeline
One of the most popular topics in software development currently is DevOps and the related topics of CI (Continuous Integration) and CD (Continuous Delivery). Typically a team that works to these methodologies has a ‘deployment pipeline’ where they build their code and run unit and integration tests. If you work with a team like this, you can setup their deployment pipeline to run your Postman collections for you. This means that your collections that help you create test data or check the environment is ready for deeper exploratory testing can be run for you every time you create a new build of the codebase. The tool that allows you to do this is called Newman. Newman simply allows Postman collections to be run on command line, however this means any CI build tool can run it too such as Jenkins, Bamboo, TeamCity or GoCD. Here is a blog about how to do that with Jenkins.
Have a look at any existing APIs you may work with already
This seems obvious, but ask around about any APIs you might already work with, there may be systems you didn’t realise existed that you could have a look at. Or there could be third party integrations that your team use within your application. There may be some existing documentation or even monitoring, it can be especially interesting to have a look at any monitoring you already have. Tools such as AppDynamics gather a lot of data such as API requests, their speed and responses. This can give you an insight into how people use those APIs and what problems may already be occurring.
Try out other tools
Postman isn’t the only way to interact with APIs, it’s a great tool and I especially like using it to teach with because it’s popular and has a nicer interface which isn’t as cluttered as some others. Your team may use a totally different tool or you may need to use another tool in future and they all have different strengths and weaknesses. So it may be useful to learn some other tools such as:
Another popular tool for interacting with APIs is Jmeter, however I highlight this separately as it’s actually a load testing tool. It can be used in a similar fashion to the Postman collections but is designed for running many, many concurrent requests and designing performance test runs. However, I have seen it successfully used as an automated functional checking tool and can be integrated into build pipelines too.
Write automated checks in a scripting or programming language
There will come a point where creating automated checks using Postman becomes very complicated or unwieldy. This is a point where it's typically easier to write it in a scripting or programming language. Why? Postman (and also Jmeter) are GUI-based and so they enforce a particular pattern and design to your tests in order to work. Sometimes what you are trying to do doesn’t neatly fit their structure or sometimes you want to integrate with more systems or perform functions they don’t provide.
Which programming language? It pretty much doesn’t matter, almost every popular programming language comes with libraries for making HTTP requests and frameworks for running tests. What you decide to learn should be guided by:
- Your level of experience with programming.
- Who is going to write and maintain these checks.
- What languages people use around your workplace.
- What languages are used for the application you are testing.
If you are working with a Java application and your team is happy to share the work and help out, it may make the most sense to write your automated checks in Java using frameworks like Junit and libraries like RestAssured. This means they can be easily incorporated into the rest of the integration tests your team already has and removes the need to find more tools to run it in your pipeline.
However, you may not be closely working with a team like this or have developers to support you. You may have decided that it will be best for you to maintain the tests. In this case it’s more important to choose a language you are comfortable learning. In this scenario I personally like teaching people about Python and the requests library because Python can be easier for newbies to learn programming. However, there are lots of other languages such as Ruby, Javascript, C# and more and none of them are bad to learn. They all have the capability to create these checks and much of what you learn will be transferrable to other languages.
Learn how to work with mocks
Sometimes you may need to test an application that relies upon a third party API. Maybe it’s a website that hasn’t got a back-end finished yet. Perhaps you are working with lots of other development teams and some of the work has been finished early. In these situations it’s useful to be able to create pretend versions of these APIs so you can test as if they were there. These are referred to as ‘mocks’. You can have a play with websites such as this one to create a fake API that responds exactly the same as an application you want to fake. You can then point an application you are testing at it to begin testing against your contract.
Learn about contract testing
Speaking of contracts and mocks, there are now tools that let you create these mocks in a more reliable and automated way. Tools such as Pact allow teams to run automated checks against each other’s services without having to understand how to run the services. While not strictly about APIs, learning about how you could automatedly check an API you provide for another team or vice versa can be a lot more useful than creating massive Postman collections or custom mocks that fall out of date as teams update the behaviour of their services.This video gives a helpful explanation in a conversational style about Pact and contract testing (thanks Conny!):
No comments:
Post a Comment