Test Gap
Analysis.

Stop wasting time building unnecessary tests and accelerate your sprints.

Building tests is necessary.
Building the wrong tests is not
.

Today’s agile methodologies were designed to deliver more business value faster. However, rapid changes in the codebase mean your teams are writing more and more tests to ensure the quality of the application. But how do you know if you’re building tests you actually need—or wasting time building those you don’t? You need visibility into high-risk areas, exposing untested code, poor design and coding practices, and even unused legacy code. Such insights can enable your agile teams to clean up the code and data and ensure the codebase quality remains at optimal levels.

Test Gap
Analysis.

Stop wasting time building unnecessary tests and accelerate your sprints.

Building tests is necessary.
Building the wrong tests is not.
.

If there’s one thing the market demands of both software development and QA managers, it’s velocity. Today’s agile methodologies were designed to deliver value faster with incremental features via shorter sprints. However, this often comes with rising technical debt—that trade-off between developing rapidly and developing quality, maintainable code. Rapid changes in the code base mean you have to develop more and more tests, and the more tests you build, the longer it takes to complete each sprint.

Analyzes multiple data sources.

SeaLights does more than scan a codebase to see what tests are needed for the current build. It analyzes data from multiple sources to identify where your resources should focus their test development:



  • Builds – TGA analyzes what code changed from the previous build, when, by whom, and whether it was previously tested.



  • Test Coverage – TGA examines your existing tests to determine what type/stage of coverage applies to the application under test.



  • Production – TGA examines how users actually interact with the application in production, determining if tests on certain features/areas of code are even needed.

 

The result? You get an accurate picture of what’s changed and what’s already covered by tests—as well as whether some tests are still needed based on actual usage.

Identifies gaps for any type of test.

While most software teams rely on static code analysis to detect defects, SeaLights provides software engineering and QA managers visibility into their test gaps during the test phase. Going beyond the unit test level, SeaLights runs dynamic analyses across regression tests, functional tests, API tests, integration tests, and exploratory and automated tests—including manual tests.

That means your test developers don’t waste precious sprint time examining different types of tests with different tools—or manually—to see whether any of them cover the changes in the current build. If the existing tests don’t cover the changed code, SeaLights shows which modules require new tests and even allows them to create manual tests right in the dashboard.

Classifies test gaps by risk level.

Not all test gaps represent the same level of risk and therefore don’t need to have the same level of test coverage. SeaLights divides gaps into three groups, each group representing a different level of risk.



  • High Risk – An untested method that was recently modified and used in production.



  • Medium Risk – An untested method that was recently modified and not used in production.



  • Low Risk – An untested method that wasn’t modified recently and used in production.

 

It also identifies “no risk” gaps—code that hasn’t changed and is not used in production—which may indicate dead code that could be cleansed when opportunity allows.

This classification allows managers and team leaders to focus each sprint’s test-building activities on those which will eliminate the defects that might otherwise affect the most users in production actions—and not waste time building unnecessary tests.

Configurable range for previous sprints.

SeaLights TGA allows you see the end state of testing for the previous sprint(s), including all changes, all executions, and all testing activities associated with the previous sprint. But TGA also allows you to configure the range of time or number of builds to be considered in its test gap analysis.

You can specify the analysis will consider all changes and activities:



  • Since the last build



  • During a specific date range



  • During a repeatable, rolling time frame (e.g., the last two weeks)

That means the TGA report will contain exactly the amount of information and granularity you choose.

Drills down on each component at risk.

The three-level classification of test gaps shows test developers exactly which changed code has the highest risk of incurring technical debt. To see more information on why the code has been flagged for insufficient or lack of test coverage, SeaLights lets them drill down by simply clicking on the component right in the SeaLights dashboard. SeaLights shows them a list of the exact files TGA identified as containing test gaps, who committed the new code, and when.

This means no more hunting or guessing as to which modules are at risk, or even which types of tests are missing or are in need of attention. Test engineers know which module they need to write a test for and why developers know exactly which files might need their attention.

Points out the source and developer associated with each gap.

SeaLights TGA lets test engineers drill down on each identified test gap, so they are instantly aware of the types of tests that do (and do not) need to be built. But for software developers, it goes even further. Because TGA identifies test gaps down to the method and line number, the SeaLights plugin allows your developers to view the individual methods and lines of code that need attention, including who changed it and when, right in the source code.

Users simply click the Github icon, provide the appropriate login credentials (SeaLights has no direct access to your source code without your login), and view the source code where TGA identified test gaps. The source code is overlaid with TGA’s “covered/uncovered” identifiers, color-coded to indicate the type and severity of each risk. That means test developers can pinpoint the exact lines of code at risk, all flagged with their corresponding risk level.

Configurable gap analyses, filters, and advanced settings.

With different sprint and build intervals, levels of code changes, and testing intervals, no two development shops are the same. That’s why SeaLights allows development and QA managers the granularity to control the frequency and behavior of their test gap analyses. Similarly, not everyone needs to see the same information to get their job done quickly. Developers and test engineers each have their own areas of responsibility and their individual preferences for notifications and more.

With the settings feature users can configure TGA’s activities and set up filters and preferences, so they can see exactly what they need to see and when. For example:



  • Define Test Gaps Report – Set the frequency of test gap analysis and reporting by individual applications.



  • Set Notification Rules – Define your preferred type of report and results notifications, as well as the method of delivery (such as Slack or email).



  • Flag Ignored Code – Filter out the ”noise” by flagging legacy code or dead code, so it is ignored in future gap analyses.

Analyzes multiple data sources.

SeaLights does more than scan a codebase to see what tests are needed for the current build. It analyzes data from multiple sources to identify where your resources should focus their test development:



  • Builds – TGA analyzes what code changed from the previous build, when, by whom, and whether it was previously tested.



  • Test Coverage – TGA examines your existing tests to determine what type/stage of coverage applies to the application under test.



  • Production – TGA examines how users actually interact with the application in production, determining if tests on certain features/areas of code are even needed.

 

The result? You get an accurate picture of what’s changed and what’s already covered by tests—as well as whether some tests are still needed based on actual usage.

Identifies gaps for any type of test.

While most software teams rely on static code analysis to detect defects, SeaLights provides software engineering and QA managers visibility into their test gaps during the test phase. Going beyond the unit test level, SeaLights runs dynamic analyses across regression tests, functional tests, API tests, integration tests, and exploratory and automated tests—including manual tests.

That means your test developers don’t waste precious sprint time examining different types of tests with different tools—or manually—to see whether any of them cover the changes in the current build. If the existing tests don’t cover the changed code, SeaLights shows which modules require new tests and even allows them to create manual tests right in the dashboard.

Classifies test gaps by risk level.

Not all test gaps represent the same level of risk and therefore don’t need to have the same level of test coverage. SeaLights divides gaps into three groups, each group representing a different level of risk.



  • High Risk – An untested method that was recently modified and used in production.



  • Medium Risk – An untested method that was recently modified and not used in production.



  • Low Risk – An untested method that wasn’t modified recently and used in production.

 

It also identifies “no risk” gaps—code that hasn’t changed and is not used in production—which may indicate dead code that could be cleansed when opportunity allows.

This classification allows managers and team leaders to focus each sprint’s test-building activities on those which will eliminate the defects that might otherwise affect the most users in production actions—and not waste time building unnecessary tests.

Configurable range for previous sprints.

SeaLights TGA allows you see the end state of testing for the previous sprint(s), including all changes, all executions, and all testing activities associated with the previous sprint. But TGA also allows you to configure the range of time or number of builds to be considered in its test gap analysis.

You can specify the analysis will consider all changes and activities:



  • Since the last build



  • During a specific date range



  • During a repeatable, rolling time frame (e.g., the last two weeks)

That means the TGA report will contain exactly the amount of information and granularity you choose.

Drills down on each component at risk.

The three-level classification of test gaps shows test developers exactly which changed code has the highest risk of incurring technical debt. To see more information on why the code has been flagged for insufficient or lack of test coverage, SeaLights lets them drill down by simply clicking on the component right in the SeaLights dashboard. SeaLights shows them a list of the exact files TGA identified as containing test gaps, who committed the new code, and when.

This means no more hunting or guessing as to which modules are at risk, or even which types of tests are missing or are in need of attention. Test engineers know which module they need to write a test for and why developers know exactly which files might need their attention.

Points out the source and developer associated with each gap.

SeaLights TGA lets test engineers drill down on each identified test gap, so they are instantly aware of the types of tests that do (and do not) need to be built. But for software developers, it goes even further. Because TGA identifies test gaps down to the method and line number, the SeaLights plugin allows your developers to view the individual methods and lines of code that need attention, including who changed it and when, right in the source code.

Users simply click the Github icon, provide the appropriate login credentials (SeaLights has no direct access to your source code without your login), and view the source code where TGA identified test gaps. The source code is overlaid with TGA’s “covered/uncovered” identifiers, color-coded to indicate the type and severity of each risk. That means test developers can pinpoint the exact lines of code at risk, all flagged with their corresponding risk level.

Configurable gap analyses, filters, and advanced settings.

With different sprint and build intervals, levels of code changes, and testing intervals, no two development shops are the same. That’s why SeaLights allows development and QA managers the granularity to control the frequency and behavior of their test gap analyses. Similarly, not everyone needs to see the same information to get their job done quickly. Developers and test engineers each have their own areas of responsibility and their individual preferences for notifications and more.

With the settings feature users can configure TGA’s activities and set up filters and preferences, so they can see exactly what they need to see and when. For example:



  • Define Test Gaps Report – Set the frequency of test gap analysis and reporting by individual applications.



  • Set Notification Rules – Define your preferred type of report and results notifications, as well as the method of delivery (such as Slack or email).



  • Flag Ignored Code – Filter out the ”noise” by flagging legacy code or dead code, so it is ignored in future gap analyses.

How SeaLights Test Gap Analysis for
Software Developers and QA Teams Works
.

Integrates seamlessly with your source code repository.

Compares current tests of all types to determine the level of overall test coverage—and coverage for the current build.

Scans your production application to determine what code actually executes (or what does not).

Provides comprehensive test gap report with recommendations for high, medium, and low risk tests.

Allows test engineers to drill down to determine exactly which test should (and should not) be worked on for the current sprint.

Shows affected methods to code developers, directly in the source control system.

Allows users to configure frequency and type of test gap analysis and individual views.

Increases velocity by reducing unnecessary test creation and maintenance.

How SeaLights Test Gap Analysis for
Software Developers and QA Teams Works
.

Integrates seamlessly with your source code repository.

Compares current tests of all types to determine the level of overall test coverage—and coverage for the current build.

Scans your production application to determine what code actually executes (or what does not).

Increases velocity by reducing unnecessary test creation and maintenance.

Allows test engineers to drill down to determine exactly which test should (and should not) be worked on for the current sprint.

Shows affected methods to code developers, directly in the source control system.

Provides comprehensive test gap report with recommendations for high, medium, and low risk tests.

Allows users to configure frequency and type of test gap analysis and individual views.

Increase velocity without sacrificing quality by
building only the tests you need.

SeaLights’s intelligent Test Gap Analysis lets you stop wasting
precious time building unnecessary tests. Period.

Still maintaining tests that, frankly, don’t deserve your time and attention? Truth is, there is plenty of code with relatively low risk of ever causing a problem in production. For example, code that has been running in production for six months, untested but unchanged, isn’t likely to suddenly break. Worse still, there’s code in production that is never executed. So why waste precious time building tests for it?

SeaLights TGA brings visibility to the process. It provides intelligent test development assistance, to reduce the time you spend writing and maintaining tests that will never benefit either the application developers or users, or increase the quality of the application.

TGA analyzes which code has been modified and which is actually used in production, then compares the current sprint with the testing results for the previous sprints . By knowing the technical debt at the end of the previous sprint, your team can plan which of those debts require adding or automating tests in the next sprint. That means you can focus your test-building activities right where they should be—giving your team the additional velocity it needs.

Increase velocity without sacrificing quality by
building only the tests you need.

SeaLights’s intelligent Test Gap Analysis lets you stop wasting
precious time building unnecessary tests. Period.

Still maintaining tests that, frankly, don’t deserve your time and attention? Truth is, there is plenty of code with relatively low risk of ever causing a problem in production. For example, code that has been running in production for six months, untested but unchanged, isn’t likely to suddenly break. Worse still, there’s code in production that is never executed. So why waste precious time building tests for it?

SeaLights TGA brings visibility to the process. It provides intelligent test development assistance, to reduce the time you spend writing and maintaining tests that will never benefit either the application developers or users, or increase the quality of the application.

TGA analyzes which code has been modified and which is actually used in production, then compares the current sprint with the testing results for the previous sprints . By knowing the technical debt at the end of the previous sprint, your team can plan which of those debts require adding or automating tests in the next sprint. That means you can focus your test-building activities right where they should be—giving your team the additional velocity it needs.

What if you could determine exactly which tests you need to build or update,
based on the content of the current sprint?

SeaLights Test Gap Analytics (TGA) analyzes application code and test executions across all test types—functional, regression, manual, automated—and all test stages —unit, integration, system, UAT, pre-prod. It then identifies and classifies gaps in existing tests as high, medium, and low risk, thus guiding your teams to build those automated tests that will have the greatest impact on quality, quickly and effectively. SeaLights TGA integrates directly with your existing repositories, tools, and processes and is quick to get up and running—just plug in TGA and start increasing your team’s velocity, without sacrificing quality.

What if you could determine exactly which tests you need to build or update, based on the content of the current sprint?

SeaLights Test Gap Analytics (TGA) analyzes application code and test executions across all test types—functional, regression, manual, automated—and all test stages —unit, integration, system, UAT, pre-prod. It then identifies and classifies gaps in existing tests as high, medium, and low risk, thus guiding your teams to build those automated tests that will have the greatest impact on quality, quickly and effectively. SeaLights TGA integrates directly with your existing repositories, tools, and processes and is quick to get up and running—just plug in TGA and start increasing your team’s velocity, without sacrificing quality.

We support everything.

 

BitBucket

 

Circle CI

 

GitHub

 

GitLab

 

Jenkins

 

Mocha

 

Jest

 

JUnit

 

Angular

 

Apache Ant

 

Apache

 

AVA

 

Babel

 

Blaze Meter

 

C#

 

Cucumber

 

Gradle

 

Jasmine

 

Java

 

Webpack

 

Load Runner

 

NUnit

 

Protractor

 

Python

 

React

 

Sahi

 

Soasta

 

Team City

 

TFS

 

TestNG

Seamless integration with your CI.

 

BitBucket

 

Circle CI

 

GitHub

 

GitLab

 

Jenkins

 

Mocha

 

Jest

 

JUnit

 

Angular

 

Apache Ant

 

Apache

 

AVA

 

Babel

 

Blaze Meter

 

C#

 

Cucumber

 

Gradle

 

Jasmine

 

Java

 

Webpack

 

Load Runner

 

NUnit

 

Protractor

 

Python

 

React

 

Sahi

 

Soasta

 

Team City

 

TFS

 

TestNG

Trusted and loved by.

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Trusted and loved by.