Quality Infrastructure Built For Speed
Velocity is a top priority for the cyber security company. To enable rapid, incremental releases they adopted CI/CD, test automation, and a quality culture. Quality is paramount as their product aims to be the best in the breed of security solutions. Their Director of Engineering was tasked with building a team focused on infrastructure to support hundreds of software developers and test automation engineers. Both engineering and QA were tapped for the team, and are working to provide all of their 150+ microservices automatically get quality infrastructure throughout their Continuous Delivery pipeline.
They Realized They Were Wasting Time On Repetitive Tests Because They Couldn’t Differentiate Between Tested And Untested Code Across Their Test Suites
They couldn’t develop and test everything on each build and still ship at the rate that they needed to. Because they couldn’t see the test suite overlay between unit, integration or end-to-end suites they were developing too many repetitive and unnecessary tests.
To develop and test in parallel they needed to take back visibility and control. Then they would be able to identify where to write tests, and how to achieve the best results with the least amount of tests.
They Realized They Needed Quality To Make Sure They Developed Only The Right Tests
The SeaLights Quality Intelligence Platform gave them cross-stack Test Quality Analytics in real time. With instantaneous visibility their teams were able to adopt a “Less Tests Approach” (Achieve higher quality with just the right amount of tests – Learn more about “Less Test Approach”) to identify exactly where they needed to augment tests so we can write just the right amount of tests to increase our test suite effectiveness. SeaLights provided guidance to develop a pragmatic set of tests with the least potential production risks.
They Doubled Velocity And Increased Team Efficiency By 60%
Before SeaLights they needed at least one dedicated test engineer plus developers to develop tests for a microservice. Now that they can identify exactly where they need tests they achieve better results with less test development. In the last sprint alone they increased integration test effectiveness by over 20%. With SeaLights Test Quality Analytics they get real-time insights into their builds that correlate tests with changing code areas and tells them exactly where to prioritize test development
Adopting “Less Tests Approach” In Sprints
SeaLights is integrated into their quality infrastructure and all of their microservice teams receive quality analytics out of the box. Integrating SeaLights was just a matter of updating their microservices template to add it to all of their services. Whenever tests ran the test metadata is immediately sent to SeaLights providing the teams with real-time feedback on their tests’ effectiveness. Providing them with test metadata to become more data-informed and efficient.
- Sprint planning: SeaLights analyzed their test gaps in each microservice, enabling them to plan and prioritize testing activities.
- Sprint execution: developers and managers got real-time feedback that ensured that all the important code changes in the sprint were tested, and the regression suite was up to date.
- Sprint retrospective: the team got feedback from SeaLights that all testing gaps have been closed, what the release readiness status is and what new gaps need to be added to the next sprint plan.
Next Steps
SeaLights Test Quality Analytics is already integrated into the company’s Quality Infrastructure and all their scrum teams are starting to use its data-informed insights to increase their test effectiveness. They are currently onboarding SeaLights Production Insights which will enable them to prioritize testing based on end-user code execution and eliminate dead code from their microservices.
About SeaLights
SeaLights is the #1 Quality Intelligence Platform for Software Development. It helps you increase the velocity of your engineering teams by using cross-stack quality analytics throughout your code activity, tests, and releases. SeaLights discovers Test Quality Analytics to help you get more from tests with a “less tests approach”, utilizes Test Impact Analytics to test smarter, and applies release Quality Analytics to determine your release readiness.