How The Pokémon Company International Reduced its Weekly Production Incidents to Less than One a Month with SeaLights Software Quality Intelligence Platform.

How The Pokémon Company International Reduced its Weekly Production Incidents to Less than One a Month with SeaLights Software Quality Intelligence Platform.


The Pokémon Company International’s software quality group is responsible for the company’s quality infrastructure. The group reported that even though their tests were passing there were still weekly incidents, indicating that risky untested code changes were reaching production.

The team was determined to eliminate this problem; they needed end-to-end visibility of their entire testing environment, with a clear connection between tests, tested functionality, and requirements. They needed 360-degree test coverage.

The company’s technology organization began using the SeaLights Software Quality Intelligence Platform in June 2018, providing the data-driven insights and visibility they were missing and reducing its incidents by 90%. With Software Quality Intelligence, The Pokémon Company International highlights the importance of quality in the organization and provides actionable insights to its employees on how to mitigate and control Quality Risks.

The Pokémon Company International’s Achievements with SeaLights Software Quality Intelligence:

1. Down From 2-3 Incidents a Week to Less than 1 Incident Per Month.

2. Manual Tests Cover 100% of All Code Changes.

3. No Missed Untested Code Changes in 5 Months.

The Problem.

The Pokémon Company International’s software quality team was handling 2-3 incidents a week. They concluded that this was a result of the team’s inability to correlate new code changes with existing legacy code dependencies, and where those coverage gaps were. They needed a tool that could measure 360-degree test coverage so the team could identify and prioritize risks while shoring up legacy code gaps, preventing bugs from reaching production.

The team also expressed that without any real-time risk analysis of code changes, they were wasting time guessing and developing tests that did not accurately cover all of the changes. They believed that if they had a tool to analyze the testing environment, they would have the necessary insights to remediate broken tests and add tests needed to mitigate risk.

Additionally, The Pokémon Company International focuses on conducting a lot of manual tests due to the nature of its products. It required a tool that could verify that both its manual and automated tests are actually covering risky code changes.

Why SeaLights Software Quality Intelligence Platform?

SeaLights Software Quality Intelligence is the only solution that can help QA testers automatically identify, manage, and control quality risks in fast-paced development pipelines.

The Pokémon Company International uses SeaLights to:

  • Identify and analyze quality risks: The team is able to track all code changes in every build and every release and verify that their regression tests are actually covering and testing these code changes, analyzing whether the existing tests are sufficient to mitigate the quality risk.
“What we really like about SeaLights Software Quality Intelligence is that we can quickly see our Quality Risks and easily understand what our tests are not covering as our code base changes over time.”
Software Quality Manager, The Pokémon Company International
  • Report and mitigate quality risks: With granular quality metrics, all teams are able to orient themselves to the organization’s quality goals. The SeaLights dashboard is easily implemented across the entire SDLC, allowing developers to provide direct feedback to testers so they know exactly which new tests they need to develop in order to quickly and accurately close gaps. Additionally, team leaders are able to make their own data-driven decisions about their releases and applications, based on the organization’s quality policies.
  • Control quality risks: The team sets up automatic quality gates, based on the organization’s quality policies, that block risky untested code changes before they are released. This data has become an integral part of the company’s SDLC because it automatically identifies untested code changes and enforces a go-no-go decision based on standardized quality metrics. The team and management have full visibility of their release quality and there are no surprises. The software quality manager added, “with SeaLights my team can work independently and make their own decisions about their releases.”
“With SeaLights Software Quality Intelligence we know with confidence that we are covering 100% of the code that changes from build to build, and as a result, we have not had any untested code changes in the last 5 months with our legacy app.”
Software Quality Manager, The Pokémon Company International

Next Steps.

In order to further increase velocity and quality, The Pokémon Company International wants its development team to start using SeaLights Pull Request Analytics as part of its Pull Request Process. These analytics prevent untested code from entering the codebase in the first place by providing those authorizing pull requests with smarter data around how well the code has been tested.

About The Pokémon Company International.

The Pokémon Company International manages the property outside of Asia and is responsible for brand management, licensing, marketing, the Pokémon Trading Card Game, the animated TV series, home