SeaLights Case Study

How a Leading Telecommunications Company Increased their Velocity by Over 49% with SeaLights Quality Intelligence

Overview

A known telecommunications company adopted CI/CD in order to increase its development velocity and as a result, it restructured its organization to enable rapid, incremental releases and fast iterations. Part of the reorganization was the transition from centralized QA teams to distributed QA teams to increase collaboration between developers and testers. 

As a consequence of distributing the QA teams, the development teams were now responsible for the product’s quality. They began spending too much time developing too many tests without knowing if the tests were effective. The teams needed to find a way to stop unnecessary test development before it would begin to impair their velocity. Once the company began using SeaLights, its data-driven insights guided the development teams to focus their test development and ensure that only the minimum set of necessary tests were developed, thereby increasing their velocity. The company’s main achievement with SeaLights was their ability to cut their CI time in half; they were able to achieve in 6 sprints what previously took them 12 sprints.

Cut CI Time
in Half

Doubled Test Effectiveness

The Problem

In order to support its new CI/CD processes, the company’s development department experienced a reorganization.

The reorganization included:

  1. Moving from centralized QA teams to distributed QA teams: For example, the cloud team was reorganized into an autonomous team including an engineering manager, product owner, DevOps specialist, developers, and a testing specialist 
  2. Transitioning from manual testing to automated testing: The team had to take full ownership of developing automated API, integration, and Selenium tests 
  3. Developers contributing to all types of tests: Developers went from writing only unit tests to being involved in all types of test development

All of these new changes led to its development team focusing too much on test development and creating too many tests. The team did not know if the tests that they were developing were actually covering the code areas which were a priority. 

For every unnecessary test they developed, they had to maintain and execute it, lengthening their testing cycles. They had to find a way to stop unnecessary test development before they would lose control of their testing cycles and dramatically decrease their velocity.

Why SeaLights Release Quality Analytics?

The company’s development team lacked visibility into their testing and needed a solution that could help them quickly gain clarity so that it would not harm their velocity.

The SeaLights Quality Intelligence Platform provided them cross-stack Test Quality Analytics in real time so they could adopt an innovative approach and develop fewer tests while increasing their test suite effectiveness. Their engineering managers had data-driven analytics at their fingertips and could ensure that only the minimum set of necessary tests were developed, helping them achieve higher velocity. As a result, they were able to accomplish in 6 sprints what previously took them 12.

The company used SeaLights for:

  1. Sprint planning: SeaLights analyzed their test gaps in each stage, enabling them to plan and prioritize testing activities in those areas
  2. Sprint execution: Their developers and managers got real-time feedback that ensured that all the important code changes in the sprint were tested, and their regression suite was up to date
  3. Sprint retrospective: Their team, managers, product owner, and test specialist get feedback from SeaLights which of their testing gaps they closed and which ones they need to focus on for the future sprints

About The Telecommunications Company

The global company provides mobile telephone services, as well as fixed telephone services throughout the world.

About SeaLights

SeaLights is the #1 Quality Intelligence Platform for software development. The platform provides managers with actionable data analytics for both manual and automated tests so that they can focus their team’s development and testing cycles and make data-driven decisions about their releases, resulting in fewer incidents in production and a high-quality product. For more information, fill out the form to the right.below.

With SeaLights we know with confidence that we are covering 100% of the code that changes from build to build, and as a result, we have not had any untested code changes in the last 5 months with our legacy app of over 80,000 lines of code.

Ben Goldsmith, Software Engineering Manager, Pokémon