Skip to main content

Test Automation: Good, Bad and Ugly

The modern approach to software quality and software development life cycle requires that business guys, developers and testers understand that the long manual test phase, although often still necessary, must be reduced to a minimum and replaced by test automation. Working in continuous delivery and continuous integration environment requires us to create automated tests that run on demand, checking our application integration and it’s core functionality correctness. However, there are still many problems with designing and writing automated tests, resulting in their costly maintenance or abandonment in favour of a return to manual processes.

In this article I will focus on describing common good practices of test automation. This post is more than an overview than complete reference guide. Broader aspects, such as the Page Object pattern or Page Factory will be described in detail in a separate article on this blog. Although most practices apply for every types of automated tests, this article refers to independent functional and acceptance testing projects. Example used in article can be cloned from my github.

Story telling 

Tests are the best project documentation. They are always ‘alive’ and up to date with all the changes in the project, because otherwise usually we will not be able to perform application build. We should also pay attention to their readability – test classes should include a ’story telling’, readable code, similar to the common, human language. Below you can see an example of the logically correct test, which readability, however, leaves much to be desired:

This test definitely needs to be refactored. Proper test should have all the logic implementation hidden behind helper methods, and the test class should contain only the test case essence to illustrate the flow. Example below (test only, without helper methods):

Given When Then 

Test code fragmentation into Given When Then sections is one of the most common practices in writing of tests. The Given block should contain initial conditions, which are not a part of test scenario, but are required to perform it. In this section you can implement, for example, database connection inicialization, or login in to online store. When block should contain test case implementation. In our case, it may be an attempt to make a transaction on the database or to buy an item at the online shop. Then block should contain result from When block. It would include assertion for database transaction denied, or charging credit card from our online shopping. Let’s add Given When Then block to the test from the previous example:

Naming Convention 

One of the best ways to increase readability of your tests is to adopt a naming convention defining what should be the positive test result. This allows you to find out test case only by the method name. As you can see in our example with Facebook logging in test, test method name shouldNotLoginWithWrongPassword() immediately tells us about the test case context and expected result. Of course naming convention with should, isn’t the only one possible. There are many approaches, and you can also create your own convention. However, it is important to use descriptive names. The same applies to the test class names – the class name should correspond, for example, to the name of application functionality under test. 

Before and After 

Many tests contains repeated steps. This can be either opening the browser, downloading authorization token or just initialization of the objects. Most modern testing frameworks have features that allows to mark methods to be perform before / after each test or before / after each test class. In case of one of the most popular test framework in java world – junit, these are methods annotated with @Before /@After (the method is executed before / after each @Test method) and @BeforeClass / @AfterClass (the method is executed before / after each test class). Another important thing is to make test cases independent of each other and to “clean up” after each tests. Test results should be identical if you fire up a single test and if you run all the tests in the project. For example, in case of Webdriver, initializing web driver object in @BeforeClass method would be bad practice, since each test should be operated in independent environment, and therefore it should be rather initialized in @Before method, and “killed” in @After method (new browser instance is opened at the beginning of each test and closed when test is done). Here is the example of using junit annotations in our test:

Externalize Constants 

The most common argument against automated functional tests is their high cost of maintenance. Test implementation is mostly based on the shared API or the structure of html website and while their changing, a single test require constant updating to a new contract. Good practice that allows to reduce maintenance cost is externalizing as many constants as much as is required. A very good example are webdriver tests, where html selectors are one of the most frequently-changing elements of the page. Here is an example of externalizing html selectors of our facebook login test:


Proper and effective test automation can be a difficult task. However, there is a group of techniques and practices, which noticeably reduces the maintenance costs of tests. Well implemented automated functional test plan is a great improvement in the software development lifecycle and should be taken into account in the development process of every team.

Popular posts from this blog

REST-Assured framework overview

In modern software development, REST services becomes most popular choice for implementing distributed and scalable web application. They are light and easy to maintain, which results in faster and more effective implementation and system integration.
I recommend you also my other posts about REST-Assured and building microservice’s test automation frameworks: REST-Assured: going deeperBuilding microservices testing framework
With the increase popularity of RESTful services, there is a need for fast and lightweight tool for REST webservices testing automation. One of the most popular choice is Rest-Assured framework from Jayway. It introduces simplicity of testing web services from dynamic languages like groovy or ruby to java. In this post we will get our hands dirty and write automatic test in Rest-Assured framework.
In order to create complete implementation of automated tests in Rest-Assured framework, we need to write our code against some example API. We’ll use standalone Wiremock m…

Testing Asynchronous APIs: Awaitility tutorial

Despite the growing popularity of test automation, most of it is still likely to be done on the frontend side of application. While GUI is a single layer that puts all the pieces together, focusing your automation efforts on the backend side requires dealing with distributed calls, concurrency, handling their diversity and integration.
Backend test automation is especially popular in the microservices architecture, with testing REST API’s. I’ve noticed that dealing with asynchronous events is particularly considered as challenging. In this article I want to cover basic usage of Awaitility – simple java library for testing asynchronous events. All the code examples are written in groovy and our REST client is Rest-Assured.
Synchronous vs Asynchronous In simple words, synchronous communication is when the API calls are dependent and their order matters, while asynchronous communication is when the API calls are independent. Quoting Apigee definition:
Synchronous  If an API call is synchrono…

Notes after TestingCup 2018

On May 28-29th I attended TestingCup conference in Łódź. Having quite unique perspective: this was my second year in row as a Speaker at this conference I want to share some thoughts on the event. Dust has settled, lets go! 

Championship Originally TestingCup is a software testing championship. Wait, what? Yes, the formula is unique: teams and individuals from all around Poland are competing in finding the most bugs and defects in specially prepared application - Mr. Buggy. I don’t have specific data, but since this year’s conference was all english I guess competitors were not only from Poland. As a spectator, I must say that the whole competition looked very professional. There were team shirts and names, podium and trophies (gold cup and cash). 
Some cons? Testing championship is held at the first day of the conference. So this is already a conference, but if you’re not taking part in the championship… there’s not much to do, since all the talks are in the next day. Organizers are aw…