Agile Testing
Optimization.

Stop wasting time building unnecessary tests and accelerate your sprints.

Agile Testing
Optimization.

Stop wasting time building unnecessary tests and accelerate your sprints.

What is Agile Testing?

Agile teams are required to release new features to the market in a matter of weeks. Quality is essential to the agile method because its main focus is on satisfying customer requirements. The challenge is how to test features rapidly and comprehensively as development velocity increases. In agile methodology, testing begins at the onset of the project, before development has started.


In agile testing, testers are part of the agile development team. Thus, there is no separate organizational unit for testers and no QA department. Many agile organizations don’t have dedicated testers or QA engineers, as everyone on the team is responsible for quality. Some agile teams do however use test specialists that work closely with developers throughout the software development cycle.

What is Agile Testing?

Agile teams are required to release new features to the market in a matter of weeks. Quality is essential to the agile method because its main focus is on satisfying customer requirements. The challenge is how to test features rapidly and comprehensively as development velocity increases. In agile methodology, testing begins at the onset of the project, before development has started.


In agile testing, testers are part of the agile development team. Thus, there is no separate organizational unit for testers and no QA department. Many agile organizations don’t have dedicated testers or QA engineers, as everyone on the team is responsible for quality. Some agile teams do however use test specialists that work closely with developers throughout the software development cycle.

Agile Testing Challenges in
Development Organizations.
.

Not Enough
Information

Modern development teams don’t have the data they need to answer questions such as: How successful were our bug fixes or tests? What is the quality of our tests? And, have these tests improved, if so by how much? Because teams do not have this level of visibility, there is usually no way to plan for maintenance in future sprints.

Frequent Regression
Cycles

Agile teams move fast, and regression tests can become a burden. In large organizations, prioritizing regression tests is a must. Development teams are forced to prioritize their regression tests based on “tribal knowledge” error-prone product areas, anecdotal evidence from production faults, and imperfect metrics like unit test code coverage and defect density.

No Quality
Measurement

Teams do not discuss code quality activities in sprint planning. This makes it difficult for software teams to identify test gaps and to define which new tests they need to develop. Quality issues manifest themselves in production issues, architectural problems, and poor legacy code, and become a major burden on developers.

Agile Testing Challenges in
Development Organizations.
.

Not Enough
Information

Modern development teams don’t have the data they need to answer questions such as: How successful were our bug fixes or tests? What is the quality of our tests? And, have these tests improved, if so by how much? Because teams do not have this level of visibility, there is usually no way to plan for maintenance in future sprints.

Frequent Regression
Cycles

Agile teams move fast, and regression tests can become a burden. In large organizations, prioritizing regression tests is a must. Development teams are forced to prioritize their regression tests based on “tribal knowledge” error-prone product areas, anecdotal evidence from production faults, and imperfect metrics like unit test code coverage and defect density.

No Quality
Measurement

Teams do not discuss code quality activities in sprint planning. This makes it difficult for software teams to identify test gaps and to define which new tests they need to develop. Quality issues manifest themselves in production issues, architectural problems, and poor legacy code, and become a major burden on developers.

The Sealights Platform

Optimizing Testing in an Agile Environment

When practicing agile testing, prioritize tasks and ensure you remain oriented towards your user. You can’t test everything—so pay attention to features that your users value. Most teams don’t have the data they need to determine which features or areas of their product can have an impact on their customers.
Teams can use a new category of tools, called Quality Intelligence Platforms, to access this data and to focus their work on customer-facing quality issues.

SeaLights is a platform that collects data about test execution at all testing levels—unit testing, integration, and acceptance testing. It also tracks code changes and production usage of the product. It analyzes the data and identifies “testing gaps”: areas of the product which have recently changed, are used in production, but are not adequately tested. Agile teams should focus their testing efforts on test gaps. Instead of over-testing, or reacting to previous production faults, teams can target areas of the product which are at high risk of quality issues. These are the areas that customers care about the most.

The Sealights Platform

Optimizing Testing in an Agile Environment

When practicing agile testing, prioritize tasks and ensure you remain oriented towards your user. You can’t test everything—so pay attention to features that your users value. Most teams don’t have the data they need to determine which features or areas of their product can have an impact on their customers.
Teams can use a new category of tools, called Quality Intelligence Platforms, to access this data and to focus their work on customer-facing quality issues.

SeaLights is a platform that collects data about test execution at all testing levels—unit testing, integration, and acceptance testing. It also tracks code changes and production usage of the product. It analyzes the data and identifies “testing gaps”: areas of the product which have recently changed, are used in production, but are not adequately tested. Agile teams should focus their testing efforts on test gaps. Instead of over-testing, or reacting to previous production faults, teams can target areas of the product which are at high risk of quality issues. These are the areas that customers care about the most.

How Software Quality Intelligence
Can Help Optimize Agile Testing

Ideally, developers and their managers want to allocate development resources towards new functionality that benefits users and enhances revenue. But in reality, developers spend a lot of time on maintenance and bug fixing. Optimizing this activity is key to making development organizations more effective.

Test Impact Analysis.

A technique that helps determine which subset of tests is needed for a given set of changes. With this approach, not all tests have to be executed in every production source file.

Sealights identifies which tests should be executed when a specific test file or method is changed. This unique feature enables to run only those tests that are required to test the recent changes. Organizations can save a lot of time by running tests selectively during each release cycle.

Quality Risks Analysis.

Code that developers have recently modified is much riskier than code that has been in your system for years and is frequently executed by users.

SeaLights performs real-time analytics on hundreds of thousands of test executions, code changes, builds and production events to assess the readiness of a release, helping you decide which build provides the highest quality for your users.

Test Gap Analysis.

Identifies areas where the code was recently changed or executed in production but has not been tested—this represents a “test gap”. Test gaps are the best place to invest R&D and QA resources to improve quality.

How Software Quality Intelligence
Can Help Optimize Agile Testing

Ideally, developers and their managers want to allocate development resources towards new functionality that benefits users and enhances revenue. But in reality, developers spend a lot of time on maintenance and bug fixing. Optimizing this activity is key to making development organizations more effective.

Test Impact Analysis.

A technique that helps determine which subset of tests is needed for a given set of changes. With this approach, not all tests have to be executed in every production source file.

Sealights identifies which tests should be executed when a specific test file or method is changed. This unique feature enables to run only those tests that are required to test the recent changes. Organizations can save a lot of time by running tests selectively during each release cycle.

Quality Risks Analysis.

Code that developers have recently modified is much riskier than code that has been in your system for years and is frequently executed by users.

SeaLights performs real-time analytics on hundreds of thousands of test executions, code changes, builds and production events to assess the readiness of a release, helping you decide which build provides the highest quality for your users.

Test Gap Analysis.

Identifies areas where the code was recently changed or executed in production but has not been tested—this represents a “test gap”. Test gaps are the best place to invest R&D and QA resources to improve quality.

Leveraging Quality Intelligence
to Optimize Agile Testing

Monitoring tests and test frameworks at all levels—unit, functional, integration,
end-to-end, including manual tests—to understand what code is actually executed as part of the test.

Collecting data about which tests exist, which tests are run as part of the build and their results, to understand test duplication and overlap.

Tracking code changes, to identify which major changes in the current release have been sufficiently tested—and thus don’t represent a quality risk.

Tracking code usage in production, to identify which features are used by end users and what is essentially “dead code”, namely code that does not need to be tested.

Correlating tests with code changes and production usage, to identify which high priority features the team is testing and to what extent they are testing them.

Visualizing the data to provide visibility into which tests exist, which do not, which parts of the software are at risk of quality issues, and which tests are needed to improve software quality.

Leveraging Quality Intelligence
to Optimize Agile Testing

Monitoring tests and test frameworks at all levels—unit, functional, integration,
end-to-end, including manual tests—to understand what code is actually executed as part of the test.

Collecting data about which tests exist, which tests are run as part of the build and their results, to understand test duplication and overlap.

Tracking code changes, to identify which major changes in the current release have been sufficiently tested—and thus don’t represent a quality risk.

Tracking code usage in production, to identify which features are used by end users and what is essentially “dead code”, namely code that does not need to be tested.

Correlating tests with code changes and production usage, to identify which high priority features the team is testing and to what extent they are testing them.

Visualizing the data to provide visibility into which tests exist, which do not, which parts of the software are at risk of quality issues, and which tests are needed to improve software quality.

Learn More About Agile Metrics, Read our White Paper

Regression Testing in the Microservices Age

Most applications based on a microservices architecture have between dozens to hundreds of services. This means that organizations moving from a monolithic to a microservices infrastructure go from one build and one test pipeline (however large) to hundreds of builds and multiple test pipelines.

In this white paper, we explain how testing in a microservices environment is dramatically different than in a monolithic application.

These differences create five major challenges for teams maintaining high-velocity microservices apps:



  • High complexity with low visibility



  • Complex and unpredictable rollback of changes



  • Limited visibility across the organization for changes made to individual components



  • Very long run time for full regression across all microservices



  • Decreasing marginal benefit of new tests

We will show how Quality Intelligence Platforms can mitigate these challenges, create visibility and avoid slowing down development with unplanned, ineffective, or slow running tests.

By submitting this form I agree that SeaLights may process my data in the manner described in SeaLights Privacy Policy.

Learn More About Agile Metrics, Read our White Paper

Regression Testing in the Microservices Age

Most applications based on a microservices architecture have between dozens to hundreds of services. This means that organizations moving from a monolithic to a microservices infrastructure go from one build and one test pipeline (however large) to hundreds of builds and multiple test pipelines.

In this white paper, we explain how testing in a microservices environment is dramatically different than in a monolithic application.

These differences create five major challenges for teams maintaining high-velocity microservices apps:



  • High complexity with low visibility



  • Complex and unpredictable rollback of changes



  • Limited visibility across the organization for changes made to individual components



  • Very long run time for full regression across all microservices



  • Decreasing marginal benefit of new tests

We will show how Quality Intelligence Platforms can mitigate these challenges, create visibility and avoid slowing down development with unplanned, ineffective, or slow running tests.

By submitting this form I agree that SeaLights may process my data in the manner described in SeaLights Privacy Policy.