Although Continuous Delivery and Continuous Integration are there for a while, Continuous Testing may sound unfamiliar. The concept appeared about 5 years ago, but it’s not widely used. In this article, we are going to answer the question: why it’s not that easy to transform the QA world.
What’s the reason for all that fuss about Continuous Testing?
You definitely noticed that testing, previously ignored until the final stage of the SDLC, is now revolving around every project stage and is acknowledged as important as software development. The thing is that throughout the history of development, testers were always pressed for time and resources when compared to developers. QA procedures used to be accepted as supporting services, and the responsibility for the final product was on the shoulders on the dev team.
Finally, such a scenario showed its weak spots, so the situation changed to a great extent. Today releases take place a lot more often, and the requirements are a lot firmer in comparison to what the industry observed, several years ago. New app versions are released in no time, so teams picked up a brand-new approach in coding and testing, making a traditional Waterfall model a legacy of the past. The software development industry started following the Agile Manifesto and hiring DevOps.
What happened to software testing next? The time came for the rise of test automation. When well-planned, test automation helps to speed up the delivery, significantly reduce costs and protect from human errors. However, it’s not a cure-all: there are lots of projects and lots of companies, which require manual testing. Let’s face it: about 70% of all testing procedures are carried out manually.
It’s High Time for Continuous Testing
At this point Continuous Testing (CT) appears to join Continuous Delivery which helps to reach two objectives at once: provides fast delivery matching with Agile principles and supports test automation. Let’s look at the concept in-depth.
The goal of Continuous Testing is to reveal defects as soon as possible using small independent test units, which have nothing to do with other tests and run every time there are changes and adjustments within the system. This way CT provides timely and informative feedback on the current system state and provides data on how the changes impact the system. As a follow-up, it enables developers to obtain go or no-go results required for effective coding.
The three main factors affecting the results of CT are people, processes and technologies.
- People. Developers acquire a necessary amount of testing capabilities without the need to get the nitty-gritty. The idea of Continuous Testing blurs the borders with the development and testing so that the dev and agile teams treat testing as a part of their routine.
- Processes. In CT processes can be defined as ultimate automation.
- Technologies. Intuitive, open-source and top-notch technologies augment automation and provide developers with almost unlimited opportunities.
These three factors may impact both in a positive or in a negative way. Let’s consider the negative aspect in detail.
Most people avoid wearing many hats, however, Continuous Testing is exactly about becoming a one-man-army. For managers, the task of motivating employees and applying CT philosophy may appear complicated. It takes a lot of effort to hire and/or retrain the staff in order to get tech-specialist, who are capable of both coding and testing. And that’s exactly what the concept implies.
Another complicated aspect is the movement towards test automation. Test automation isn’t always the optimal solution because of high expenses and limitations for its use. This way some companies decide in favor of manual testing, as it turns out to be safer. This point of view is explained in the study “Impediments for software test automation”, in which the researchers provided a list of human-related obstacles to the use of test automation.
To paraphrase the main idea of the research: teams pay attention to cases, for which test automation is not the best option. Such an approach can decrease the urge to spend resources on automation. As a follow-up it results in less time spent, fewer testers performing the work and lower quality automation. It shows how low-quality automation decreases the perceived value and closes the loop.
This way Continuous Testing is about highly motivated and very talented people, so the main objective is to ensure continuous training and improving the skills of the team. It is necessary to establish the corporate processes the right way to get all of the benefits the CT trend offers.
Properly-tuned CT processes are backed up with targeted automated tests to reveal new deficiencies after the introduction of the new lines of code. The adherents of Continuous Testing principles aren’t scared of the “fail fast” approach. There is no need to worry about minor bugs, as fail-free testing isn’t a realistic outcome anyway.
However, more significant defects can put the app at risk, so the processes need to be adjusted to test small code portions with the help of automated tests. At first sight, it looks logical, but later it turns out that automation has limited potential and can’t be applied to certain types of tests or projects. There are only two types of tests, which can be automated with a notable extent of reliability: functional testing and performance testing. Why? They are ideal candidates for implementing the CT approach as they meet the following requirements:
- provide immediate detection of bugs, which are hard to find out manually;
- ensure the maximum test coverage within a given time frame allocated for testing;
- can be run at any moment.
As far as security test automation is concerned, Continuous Testing still has a long way to go. Manual security testing is preferred more because it relieves concerns auditable and verifiable trails. There are regulation standards in each industry, so no weak points should appear, making it very likely to be breached. The same approach applies to UI/UX testing, exploratory testing and in some cases to the API testing as well (which is surprising, as API test automation pays off in terms of time and budget).
It’s almost not possible for developers to test their work without dev-friendly tools. Luckily, it’s not a problem at all, because there are lots of tools meeting high standards: API, GUI testing tools as well as tools for security test automation. However, the cost is the limiting factor for implementing them. The efforts spent on picking, setting up and configuring the tools and frameworks are huge.
Are there any ways to attain CT goals without using expensive test automation tools? Unfortunately, no. A lot of companies, who adopted the CT approach are still using old-fashioned tools from the times of Waterfall. Surely, it’s difficult to keep up the pace of Agile this way. These tools don’t provide the required support for test automation.
The Bottom Line
Continuous testing is the turning point in the testing industry, so no wonder it is associated with certain challenges. Nevertheless, we believe in a full-scale transformation of the area. Continuous Testing looks promising in terms of speed, quality, and resource optimization. It will definitely find the acknowledgment by the global software development community in the future - just like it happened with DevOps and CI/CD.
Contact us to request a quote or to learn more about Test Automation, tools and frameworks.