Skip to main content

Top 5 traps of test automation

This article by me was originally published on TheServerSide.

There’s a common phrase in testing: if you do something more than once – automate it. Software testing, where we routinely perform similar actions, is a perfect base for automation. In modern software development, with the use of microservices and continuous deployment approach, we want to deliver features fast and often. Therefore, test automation becomes even more important, yet still facing some common problems. Based on my experience, here is my list of top 5 mistakes that teams make in acceptance test automation.

Stability 

False Fails? Always-red plans? We all know that. Stability of automated tests is one of the most obvious issues, yet most difficult to obtain. Even big players like LinkedIn or Spotify admit to have struggled with it. While designing your automation, you should put extra attention to stability, as it is the most frequent cause of failure and test inefficiency. Writing tests is just a beginning, thus you should always plan some time in sprint for maintenance and revisions.

UI perspective 

Majority of modern applications are web-based, therefore the most preferable way for functional testing is from the UI perspective. Despite their usefulness, browser tests also have some substantial problems like slow execution time or stability randomness. Since we’re in the world of microservices, it’s worth to consider dividing tests into layers – testing your application features directly through webservices integration (or backend in general) and limiting UI tests to a minimal suite of smoke tests would be more efficient.

Overmocking 

Because of many dependencies over the systems, mocking services become a popular pattern and are also often forced over test environment limitations. However, you should pay great attention to the volume of your mocked checks – mocks can miss the truth or be outdated, so your development would be held on false assumptions. There’s a saying: “Don’t mock what you don’t own”, which means you can stub only these pieces of architecture that you’re implementing. This is a proper approach when you test integration with the external system, but what if you want to assume stable dependencies and test only your implementation? Then yes, you mock everything except what-you-own. To sum up, the mocking and stubbing strategy can differ depending on test purposes.

Tight coupling with framework 

That’s a tricky one. Developers often tend to choose frameworks and tools based on the current trends. The same applies to test frameworks where we have at least a few great frameworks to use just for a test runner, not to mention the REST client, build tool and so on. While choosing a technology stack, we should bear in mind the necessity to stay as much independent from tools as we can – it’s the test scenario that is the most important, not the frameworks.

Keep it simple 

Depending on your team structure, acceptance tests are implemented by developers or testers. Usually the developers are better in writing code, while the testers have a more functional approach (it’s not a rule, though). Automated test are not a product itself, but rather a tool, therefore I would put functionality over complexity. Your test codebase is nearly as big as the tested system? Try to categorize tests according to their domain or type. Adding new tests requires a time-consuming code structure analysis? Sometimes a more verbose (but more readable) code is better for your tests than complex structures.

Summary 

The worst-case scenario that can happen to your automated tests is abandoning them due to relatively simple, yet common issues. Time saved by automating simple test cases can be used for executing more complex scenarios manually, and that leads to better software quality and higher employee motivation in general. If you have some other interesting experiences with test automation, let me know!

Popular posts from this blog

Test Automation: Good, Bad and Ugly

The modern approach to software quality and software development life cycle requires that business guys, developers and testers understand that the long manual test phase, although often still necessary, must be reduced to a minimum and replaced by test automation. Working in continuous delivery and continuous integration environment requires us to create automated tests that run on demand, checking our application integration and it’s core functionality correctness. However, there are still many problems with designing and writing automated tests, resulting in their costly maintenance or abandonment in favour of a return to manual processes.
In this article I will focus on describing common good practices of test automation. This post is more than an overview than complete reference guide. Broader aspects, such as the Page Object pattern or Page Factory will be described in detail in a separate article on this blog. Although most practices apply for every types of automated tests, thi…

REST-Assured framework overview

In modern software development, REST services becomes most popular choice for implementing distributed and scalable web application. They are light and easy to maintain, which results in faster and more effective implementation and system integration.
I recommend you also my other posts about REST-Assured and building microservice’s test automation frameworks: REST-Assured: going deeperBuilding microservices testing framework
With the increase popularity of RESTful services, there is a need for fast and lightweight tool for REST webservices testing automation. One of the most popular choice is Rest-Assured framework from Jayway. It introduces simplicity of testing web services from dynamic languages like groovy or ruby to java. In this post we will get our hands dirty and write automatic test in Rest-Assured framework.
In order to create complete implementation of automated tests in Rest-Assured framework, we need to write our code against some example API. We’ll use standalone Wiremock m…

REST-Assured: going deeper

In my previous post I described the basic REST-Assured usage – the lightweight framework for testing RESTful services. Despite the fact that described range of functionalities would be enough in most cases,REST-Assured has a lot more to offer. In this post I would like to bring some more advanced examples of the framework usage.



I recommend you also my other posts about REST-Assured and building microservice’s test automation frameworks:

REST-Assured – framework overviewBuilding microservices testing framework
Object Mapping Sending request’s body as string is easy and straightforward, but it can be inconvenient in the case of more complex operations on request / response properties. Proven solution for this is a good-known serialization of request/response body to objects. REST-Assured supports object mapping to (and from) JSON and XML. For JSON you need either to have Jackson or Gson in your classpath and for XML you need JAXB. Here is an example of request object serialization using J…