When you shift testing left, among many other benefits, you can achieve continuous testing, executing
automated checks as part of the software delivery pipeline to obtain immediate feedback on the business risks
associated with a software release candidate. Continuous testing also involves early and frequent testing,
proper code management, and adjusting the various quality-related tasks over time to maximize the efficiency
of the whole process.
Some of the key benefits of continuous testing include:
- Lower the cost of development
- Reduce waste
- Reduce risk upon release
- Increase system reliability
- Release to production faster
- Compete more fiercely in the
Many teams today are trying to build or refine their continuous testing machine. We believe that in order to
improve the results of software production, it’s necessary to consider three fundamental pillars that are
closely linked together: processes, tools and people. Teams can improve by bettering their processes,
but the tools and the team members must also adapt in order for the new and improved processes to stick.
For example, within Agile, teams may find themselves in frameworks like Scrum that create a process,
for which they will find themselves using various types of tools for everything from communication to
task management and there will typically be great focus on the motivation and commitment of the team.
All aspects of Agile approaches are well designed to be adaptable to change, which is its main focus. The
idea is to assume that what the customer needs is not fixed nor established in a contract. Thus, it is
essential to work on constant adaptations in a way that does not cause the project costs to skyrocket or
What is needed to adapt to change and yet maintain a high level of quality?
For any team, the most typical problem that arises when introducing changes in a system is fear: fear of
breaking something that was already working before and not realizing it until after it has reached the
customer's hands. When a user is given a new version of the application, there is nothing worse than finding
that what always used to work no longer does.
To address this problem, teams must be able to detect any errors of any kind as soon as possible.
When speaking of errors, we are referring to any kind of error or bug: a miscalculation, performance
problems, security vulnerability, aspects of usability, maintainability problems (e.g., code quality factors)
or anything that may inconvenience a user. And what we mean by “as soon as possible” is as soon after when
the error was inserted into the system as possible.
Thus, important practices like continuous testing, which goes hand in hand with continuous integration
(CI), emerge and gain importance for Agile teams.
Basically, continuous integration proposes building and testing the solution frequently, building a
potentially customer-shippable product with a frequency that depends on each team, varying from one new
version after each commit to one every day or week. Continuous integration tools allow for defining a
sequence of tasks to be performed automatically, among which typically include:
- Updating code to the latest version
- Building the solution
- Automated checks
- Code quality checks
- Security checks
And the list can go on with hundreds of integrations and tests that one may wish to add.
There are dozens of tools to support these automated systems, some as old as Ant, Maven and MSbuild,
configuration management tools as Chef, Puppet or Ansible, to popular CI tools such as Jenkins,
CruiseControl, Bamboo and TeamCity. And if that were not enough, there are also cloud solutions like Amazon
Pipeline Code, TravisCI, CircleCI, Codeship and Microsoft’s Visual Studio Team Services (to name a few).
So, all one has to do to have continuous testing and continuous integration is simply implement one of
these CI tools and then quality will be baked into everything. Right?
The problem is that many teams fall into this very fallacy and overlook many preconditions for continuous
testing and CI that must be met beforehand, resulting in failure. Some of the preconditions include the
proper handling of code version management, incident management, data and environment management, automated
checks at different testing levels (unit, build tests, API, UI), performance tests, internal code quality and
Fortunately, this guide will help you navigate through several of the hoops your
test team has to jump through on your path to continuous testing (the highest level of testing maturity
according to our maturity model) all the while igniting the shift from traditional quality assurance to