The Pros and Cons of Test Automation

Sep 9, 2019

hire a test automation team qulixQA

The expectations from software testing are rising, while the time for it is getting compressed. Despite all the advantages of manual testing, it may become a bottleneck for the SDLC: the process of software development requires timely evaluation of risks before the new software versions are released. 

Of course, most of the tests can be carried out manually, especially when the project scale is smaller, but is it effective? As far as regression tests are concerned: the number of efforts is growing with every version of the app. The team of testers is receiving more and more tasks sprint by sprint. With this type of tests (and with a number of other ones) test automation can cope with better effectiveness. 

In theory, it looks simple: in order to solve the testing issues, all you need it to invest in automating QA. However, test automation is not a cure-all. 

A realistic idea of its pros and cons helps to ensure a balanced approach to the QA services and provides the testing team with the opportunity to use a creative approach in manual testing. It helps to check risk-prone components more thoroughly and reveal major defects at the initial stages. By combining manual and automated testing a QA company can build a smooth testing process, but it’s important to build it the right way.

Here are the TOP-3 situations, when automation is the best option:

  • Continuous Integration and Continuous Delivery approach (CI/CD);
  • projects, which need to adhere to specific certification standards and/or internal standards (ex., software for banking and finance);
  • testing of non-GUI apps (drivers, libraries, and etc.)

According to this, here is a list of tests, which match the specifics of Test Automation:

tests suitable for test automation qulixQA

Best Practices of Test Automation

When studying the subject in-depth, it becomes clear that automated test significantly reduce testing team efforts and time, as well as increase the percent of test coverage in case of scripted and regression testing. The number of tests can be decreased, and they can be used more frequently. At the same time automation is not created to replace manual testing. Even in case, it was, it is hardly possible.

According to the “Lessons Learned in Software Testing” book by C. Kener, J. Bach and B. Pettichord, “automating without good test design may result in a lot of activity, but little value”. This way, to implement automation, first it’s necessary to create a test case manually, identify the prerequisites, the test data and to clearly write the test case objectives. It may be surprising that many drawbacks are identified before the execution, as early as at the test automation development stage. It will save the team a lot of time later.

Automation should be treated like software development. It also needs code reviews, a predefined framework to follow and the maintenance. To a certain extent, it is useful, while most software development practices can be used for test automation as well.

As far as the test automation ROI (return on investment) is concerned, we need to say that it’s calculation is complicated and doesn’t always provide reliable results. 

The rule of thumb here is the following: the ROI is higher for long-term projects with a clear percent of test coverage. Is that true for your project? If you are not sure, contact our experienced test automation team for QA consulting

5 Ideas to Help You Increase Test Automation ROI 

1) Use the help of experienced teams to determine the scope of automation before the project start:

  • main business features and how they can be reused in the future
  • use scenarios containing a lot of data
  • test case complexity
  • technical feasibility, and etc.

2) Select the framework and the automation tool. The perfect option matches the automation requirements and doesn’t increase training costs a lot. 

3) Check out if the test automation team follows these Scripting Standards:

  • Writing uniform scripts, comments and adhering to the code indentation rules;
  • Ensuring the error logging messages are understandable for the testing team due to the use of proper coding (standardizing);
  • Dealing with errors caused by unexpected system failure or unexpected app behavior in the right way. 

4) Monitor metrics. Test automation metrics implies a lot more than purely comparing the ROI of manual QA with the ROI of test automation. It makes sense monitoring the percent automatable, the percent of automated test coverage and automation process in the first place.

5) Reveal and keep track of regressions carefully. Quite often it happens that regressions are found after the automation. Such neglection takes place while previously regressions could be detected and fixed very quickly without reporting them. Such scenarios are possible at projects with no rigid test documentation requirements. However, by tracking the real amount of regressions the team will have reliable data to calculate test automation ROI. 

The Bottom Line

Test automation is recommended for the following test types:

  • tests of repeatedly used functionality, which increases the risk of app failure
  • tests running for several app versions 
  • tests that are difficult to perform manually due to a high risk of a human error or a lot of effort needed for their performance. 

Here are tests, which are not likely to be effective, when automated:

  • ad-hoc tests;
  • newly created test cases (without running them manually at least once);
  • test cases with changing requirements. 

The most important tip for test automation: tests shouldn’t necessarily be automated just because they are automatable. 

At QulixQA we deliver results based on the concept of lean production. We are not going to perform test automation if it doesn’t seem effective for your project. Before starting any project, we perform careful planning and a feasibility study to ensure that automation is beneficial to your project in the long run. 

In case you are searching for professional QA solutions tailored to your area of business, contact us.