Skip to main content

Test Automation: Good, Bad and Ugly

The modern approach to software quality and software development life cycle requires that business guys, developers and testers understand that the long manual test phase, although often still necessary, must be reduced to a minimum and replaced by test automation. Working in continuous delivery and continuous integration environment requires us to create automated tests that run on demand, checking our application integration and it’s core functionality correctness. However, there are still many problems with designing and writing automated tests, resulting in their costly maintenance or abandonment in favour of a return to manual processes.

In this article I will focus on describing common good practices of test automation. This post is more than an overview than complete reference guide. Broader aspects, such as the Page Object pattern or Page Factory will be described in detail in a separate article on this blog. Although most practices apply for every types of automated tests, this article refers to independent functional and acceptance testing projects. Example used in article can be cloned from my github.

Story telling 

Tests are the best project documentation. They are always ‘alive’ and up to date with all the changes in the project, because otherwise usually we will not be able to perform application build. We should also pay attention to their readability – test classes should include a ’story telling’, readable code, similar to the common, human language. Below you can see an example of the logically correct test, which readability, however, leaves much to be desired:


This test definitely needs to be refactored. Proper test should have all the logic implementation hidden behind helper methods, and the test class should contain only the test case essence to illustrate the flow. Example below (test only, without helper methods):


Given When Then 

Test code fragmentation into Given When Then sections is one of the most common practices in writing of tests. The Given block should contain initial conditions, which are not a part of test scenario, but are required to perform it. In this section you can implement, for example, database connection inicialization, or login in to online store. When block should contain test case implementation. In our case, it may be an attempt to make a transaction on the database or to buy an item at the online shop. Then block should contain result from When block. It would include assertion for database transaction denied, or charging credit card from our online shopping. Let’s add Given When Then block to the test from the previous example:


Naming Convention 

One of the best ways to increase readability of your tests is to adopt a naming convention defining what should be the positive test result. This allows you to find out test case only by the method name. As you can see in our example with Facebook logging in test, test method name shouldNotLoginWithWrongPassword() immediately tells us about the test case context and expected result. Of course naming convention with should, isn’t the only one possible. There are many approaches, and you can also create your own convention. However, it is important to use descriptive names. The same applies to the test class names – the class name should correspond, for example, to the name of application functionality under test. 

Before and After 

Many tests contains repeated steps. This can be either opening the browser, downloading authorization token or just initialization of the objects. Most modern testing frameworks have features that allows to mark methods to be perform before / after each test or before / after each test class. In case of one of the most popular test framework in java world – junit, these are methods annotated with @Before /@After (the method is executed before / after each @Test method) and @BeforeClass / @AfterClass (the method is executed before / after each test class). Another important thing is to make test cases independent of each other and to “clean up” after each tests. Test results should be identical if you fire up a single test and if you run all the tests in the project. For example, in case of Webdriver, initializing web driver object in @BeforeClass method would be bad practice, since each test should be operated in independent environment, and therefore it should be rather initialized in @Before method, and “killed” in @After method (new browser instance is opened at the beginning of each test and closed when test is done). Here is the example of using junit annotations in our test:


Externalize Constants 

The most common argument against automated functional tests is their high cost of maintenance. Test implementation is mostly based on the shared API or the structure of html website and while their changing, a single test require constant updating to a new contract. Good practice that allows to reduce maintenance cost is externalizing as many constants as much as is required. A very good example are webdriver tests, where html selectors are one of the most frequently-changing elements of the page. Here is an example of externalizing html selectors of our facebook login test:


Summary 

Proper and effective test automation can be a difficult task. However, there is a group of techniques and practices, which noticeably reduces the maintenance costs of tests. Well implemented automated functional test plan is a great improvement in the software development lifecycle and should be taken into account in the development process of every team.

Popular posts from this blog

Notes after TestingCup 2018

On May 28-29th I attended TestingCup conference in Łódź. Having quite unique perspective: this was my second year in row as a Speaker at this conference I want to share some thoughts on the event. Dust has settled, lets go! 



Championship Originally TestingCup is a software testing championship. Wait, what? Yes, the formula is unique: teams and individuals from all around Poland are competing in finding the most bugs and defects in specially prepared application - Mr. Buggy. I don’t have specific data, but since this year’s conference was all english I guess competitors were not only from Poland. As a spectator, I must say that the whole competition looked very professional. There were team shirts and names, podium and trophies (gold cup and cash). 
Some cons? Testing championship is held at the first day of the conference. So this is already a conference, but if you’re not taking part in the championship… there’s not much to do, since all the talks are in the next day. Organizers are aw…

REST-Assured framework overview

In modern software development, REST services becomes most popular choice for implementing distributed and scalable web application. They are light and easy to maintain, which results in faster and more effective implementation and system integration.
I recommend you also my other posts about REST-Assured and building microservice’s test automation frameworks: REST-Assured: going deeperBuilding microservices testing framework
With the increase popularity of RESTful services, there is a need for fast and lightweight tool for REST webservices testing automation. One of the most popular choice is Rest-Assured framework from Jayway. It introduces simplicity of testing web services from dynamic languages like groovy or ruby to java. In this post we will get our hands dirty and write automatic test in Rest-Assured framework.
In order to create complete implementation of automated tests in Rest-Assured framework, we need to write our code against some example API. We’ll use standalone Wiremock m…

Building microservices testing framework

RESTful services and microservice architecture in general are big trends in the industry right now. I’ve noticed it also from this blog’s traffic analytics, where topics related with REST testing get the highest interest. Therefore I’ve decided to create some kind of example framework for RESTful APIs functional testing. Project is based on REST-Assured as a services client, Spock as a testing framework, Groovy and Gradle. You can clone full repository from my github. Tests are run against Wiremock API described in this post. Please consider this project as a kind of bootstrap, since it’s an example, not full-blown test framework. Ok, so as Linus said – talk is cheap, show me the code!
Dependencies Usually, first thing for me after importing new project is opening build.gradle (or pom.xml, in case of maven). In our example, the most important dependencies are REST-Assured and Spock. We’ve also Jackson, for serializing objects, Logback for our logs and snakeyaml for reading .yaml files,…