Skip to main content

Test Automation: Good, Bad and Ugly

The modern approach to software quality and software development life cycle requires that business guys, developers and testers understand that the long manual test phase, although often still necessary, must be reduced to a minimum and replaced by test automation. Working in continuous delivery and continuous integration environment requires us to create automated tests that run on demand, checking our application integration and it’s core functionality correctness. However, there are still many problems with designing and writing automated tests, resulting in their costly maintenance or abandonment in favour of a return to manual processes.

In this article I will focus on describing common good practices of test automation. This post is more than an overview than complete reference guide. Broader aspects, such as the Page Object pattern or Page Factory will be described in detail in a separate article on this blog. Although most practices apply for every types of automated tests, this article refers to independent functional and acceptance testing projects. Example used in article can be cloned from my github.

Story telling 

Tests are the best project documentation. They are always ‘alive’ and up to date with all the changes in the project, because otherwise usually we will not be able to perform application build. We should also pay attention to their readability – test classes should include a ’story telling’, readable code, similar to the common, human language. Below you can see an example of the logically correct test, which readability, however, leaves much to be desired:


This test definitely needs to be refactored. Proper test should have all the logic implementation hidden behind helper methods, and the test class should contain only the test case essence to illustrate the flow. Example below (test only, without helper methods):


Given When Then 

Test code fragmentation into Given When Then sections is one of the most common practices in writing of tests. The Given block should contain initial conditions, which are not a part of test scenario, but are required to perform it. In this section you can implement, for example, database connection inicialization, or login in to online store. When block should contain test case implementation. In our case, it may be an attempt to make a transaction on the database or to buy an item at the online shop. Then block should contain result from When block. It would include assertion for database transaction denied, or charging credit card from our online shopping. Let’s add Given When Then block to the test from the previous example:


Naming Convention 

One of the best ways to increase readability of your tests is to adopt a naming convention defining what should be the positive test result. This allows you to find out test case only by the method name. As you can see in our example with Facebook logging in test, test method name shouldNotLoginWithWrongPassword() immediately tells us about the test case context and expected result. Of course naming convention with should, isn’t the only one possible. There are many approaches, and you can also create your own convention. However, it is important to use descriptive names. The same applies to the test class names – the class name should correspond, for example, to the name of application functionality under test. 

Before and After 

Many tests contains repeated steps. This can be either opening the browser, downloading authorization token or just initialization of the objects. Most modern testing frameworks have features that allows to mark methods to be perform before / after each test or before / after each test class. In case of one of the most popular test framework in java world – junit, these are methods annotated with @Before /@After (the method is executed before / after each @Test method) and @BeforeClass / @AfterClass (the method is executed before / after each test class). Another important thing is to make test cases independent of each other and to “clean up” after each tests. Test results should be identical if you fire up a single test and if you run all the tests in the project. For example, in case of Webdriver, initializing web driver object in @BeforeClass method would be bad practice, since each test should be operated in independent environment, and therefore it should be rather initialized in @Before method, and “killed” in @After method (new browser instance is opened at the beginning of each test and closed when test is done). Here is the example of using junit annotations in our test:


Externalize Constants 

The most common argument against automated functional tests is their high cost of maintenance. Test implementation is mostly based on the shared API or the structure of html website and while their changing, a single test require constant updating to a new contract. Good practice that allows to reduce maintenance cost is externalizing as many constants as much as is required. A very good example are webdriver tests, where html selectors are one of the most frequently-changing elements of the page. Here is an example of externalizing html selectors of our facebook login test:


Summary 

Proper and effective test automation can be a difficult task. However, there is a group of techniques and practices, which noticeably reduces the maintenance costs of tests. Well implemented automated functional test plan is a great improvement in the software development lifecycle and should be taken into account in the development process of every team.

Popular posts from this blog

REST-Assured framework overview

In modern software development, REST services becomes most popular choice for implementing distributed and scalable web application. They are light and easy to maintain, which results in faster and more effective implementation and system integration.
I recommend you also my other posts about REST-Assured and building microservice’s test automation frameworks: REST-Assured: going deeperBuilding microservices testing framework
With the increase popularity of RESTful services, there is a need for fast and lightweight tool for REST webservices testing automation. One of the most popular choice is Rest-Assured framework from Jayway. It introduces simplicity of testing web services from dynamic languages like groovy or ruby to java. In this post we will get our hands dirty and write automatic test in Rest-Assured framework.
In order to create complete implementation of automated tests in Rest-Assured framework, we need to write our code against some example API. We’ll use standalone Wiremock m…

REST-Assured: going deeper

In my previous post I described the basic REST-Assured usage – the lightweight framework for testing RESTful services. Despite the fact that described range of functionalities would be enough in most cases,REST-Assured has a lot more to offer. In this post I would like to bring some more advanced examples of the framework usage.



I recommend you also my other posts about REST-Assured and building microservice’s test automation frameworks:

REST-Assured – framework overviewBuilding microservices testing framework
Object Mapping Sending request’s body as string is easy and straightforward, but it can be inconvenient in the case of more complex operations on request / response properties. Proven solution for this is a good-known serialization of request/response body to objects. REST-Assured supports object mapping to (and from) JSON and XML. For JSON you need either to have Jackson or Gson in your classpath and for XML you need JAXB. Here is an example of request object serialization using J…