Skip to main content

Top 5 traps of test automation

This article by me was originally published on TheServerSide.

There’s a common phrase in testing: if you do something more than once – automate it. Software testing, where we routinely perform similar actions, is a perfect base for automation. In modern software development, with the use of microservices and continuous deployment approach, we want to deliver features fast and often. Therefore, test automation becomes even more important, yet still facing some common problems. Based on my experience, here is my list of top 5 mistakes that teams make in acceptance test automation.

Stability 

False Fails? Always-red plans? We all know that. Stability of automated tests is one of the most obvious issues, yet most difficult to obtain. Even big players like LinkedIn or Spotify admit to have struggled with it. While designing your automation, you should put extra attention to stability, as it is the most frequent cause of failure and test inefficiency. Writing tests is just a beginning, thus you should always plan some time in sprint for maintenance and revisions.

UI perspective 

Majority of modern applications are web-based, therefore the most preferable way for functional testing is from the UI perspective. Despite their usefulness, browser tests also have some substantial problems like slow execution time or stability randomness. Since we’re in the world of microservices, it’s worth to consider dividing tests into layers – testing your application features directly through webservices integration (or backend in general) and limiting UI tests to a minimal suite of smoke tests would be more efficient.

Overmocking 

Because of many dependencies over the systems, mocking services become a popular pattern and are also often forced over test environment limitations. However, you should pay great attention to the volume of your mocked checks – mocks can miss the truth or be outdated, so your development would be held on false assumptions. There’s a saying: “Don’t mock what you don’t own”, which means you can stub only these pieces of architecture that you’re implementing. This is a proper approach when you test integration with the external system, but what if you want to assume stable dependencies and test only your implementation? Then yes, you mock everything except what-you-own. To sum up, the mocking and stubbing strategy can differ depending on test purposes.

Tight coupling with framework 

That’s a tricky one. Developers often tend to choose frameworks and tools based on the current trends. The same applies to test frameworks where we have at least a few great frameworks to use just for a test runner, not to mention the REST client, build tool and so on. While choosing a technology stack, we should bear in mind the necessity to stay as much independent from tools as we can – it’s the test scenario that is the most important, not the frameworks.

Keep it simple 

Depending on your team structure, acceptance tests are implemented by developers or testers. Usually the developers are better in writing code, while the testers have a more functional approach (it’s not a rule, though). Automated test are not a product itself, but rather a tool, therefore I would put functionality over complexity. Your test codebase is nearly as big as the tested system? Try to categorize tests according to their domain or type. Adding new tests requires a time-consuming code structure analysis? Sometimes a more verbose (but more readable) code is better for your tests than complex structures.

Summary 

The worst-case scenario that can happen to your automated tests is abandoning them due to relatively simple, yet common issues. Time saved by automating simple test cases can be used for executing more complex scenarios manually, and that leads to better software quality and higher employee motivation in general. If you have some other interesting experiences with test automation, let me know!

Popular posts from this blog

Notes after TestingCup 2018

On May 28-29th I attended TestingCup conference in Łódź. Having quite unique perspective: this was my second year in row as a Speaker at this conference I want to share some thoughts on the event. Dust has settled, lets go! 



Championship Originally TestingCup is a software testing championship. Wait, what? Yes, the formula is unique: teams and individuals from all around Poland are competing in finding the most bugs and defects in specially prepared application - Mr. Buggy. I don’t have specific data, but since this year’s conference was all english I guess competitors were not only from Poland. As a spectator, I must say that the whole competition looked very professional. There were team shirts and names, podium and trophies (gold cup and cash). 
Some cons? Testing championship is held at the first day of the conference. So this is already a conference, but if you’re not taking part in the championship… there’s not much to do, since all the talks are in the next day. Organizers are aw…

REST-Assured framework overview

In modern software development, REST services becomes most popular choice for implementing distributed and scalable web application. They are light and easy to maintain, which results in faster and more effective implementation and system integration.
I recommend you also my other posts about REST-Assured and building microservice’s test automation frameworks: REST-Assured: going deeperBuilding microservices testing framework
With the increase popularity of RESTful services, there is a need for fast and lightweight tool for REST webservices testing automation. One of the most popular choice is Rest-Assured framework from Jayway. It introduces simplicity of testing web services from dynamic languages like groovy or ruby to java. In this post we will get our hands dirty and write automatic test in Rest-Assured framework.
In order to create complete implementation of automated tests in Rest-Assured framework, we need to write our code against some example API. We’ll use standalone Wiremock m…

Building microservices testing framework

RESTful services and microservice architecture in general are big trends in the industry right now. I’ve noticed it also from this blog’s traffic analytics, where topics related with REST testing get the highest interest. Therefore I’ve decided to create some kind of example framework for RESTful APIs functional testing. Project is based on REST-Assured as a services client, Spock as a testing framework, Groovy and Gradle. You can clone full repository from my github. Tests are run against Wiremock API described in this post. Please consider this project as a kind of bootstrap, since it’s an example, not full-blown test framework. Ok, so as Linus said – talk is cheap, show me the code!
Dependencies Usually, first thing for me after importing new project is opening build.gradle (or pom.xml, in case of maven). In our example, the most important dependencies are REST-Assured and Spock. We’ve also Jackson, for serializing objects, Logback for our logs and snakeyaml for reading .yaml files,…