SeaLights Case Study

How SeaLights Enhanced Datafaction’s Test Driven Development (TDD)

Overview

Datafaction’s original accounting software was developed almost 35 years ago and was built on legacy text-based software. In 2010, City National Bank acquired Datafaction and its development department went through a significant transition in order to keep up with its new owner’s business goals. 

The most critical objective was that Datafaction had to modernize their legacy text-based software in a very short amount of time. To support this substantial undertaking, Datafaction’s software development team moved from waterfall to agile. 

With time-to-market being a priority, the team’s focus was on releasing a massive amount of code and therefore writing traditional tests cases to cover that code was cumbersome. The team needed to find a way to increase their quality without harming their velocity and initiated a different strategy with Test Driven Development (TDD). 

TDD was new to the development team and in order to ensure that they were maximizing its implementation, they used SeaLights. With SeaLights, Datafaction gained full visibility over the quality of its software, helping its research and development (R&D) team transition quickly and smoothly to their new process of TDD.

Reduced untested code changes by more than 70%

Challenge

Once Datafaction made the transition from waterfall to agile, they analyzed their delivery process and concluded that although the team’s velocity was increasing, the product’s quality was not keeping up. The team needed to make more adjustments to their development and testing processes to maintain high code quality. Datafaction’s software development manager, Ami Adler, outlined Datafaction’s main obstacles to a high-quality product:

  • Limited visibility of quality: The team wrote tests and ran them before a release, but they didn’t have a comprehensive view of how many tests existed, what they were covering, or if they contributed to the quality of the product.
  • Not enough test development: Datafaction’s team was rewriting their code and conducting manual tests just to keep up. The software was working but new defects were being introduced because a large amount of the system was untested. The team knew that they needed to move from manual to automated testing to fix this, but they didn’t want to waste time or resources by bringing in a new team to do it all at once. The team also didn’t have any data to prove which areas would be best to automate.
  • Overly focused on velocity: The development team was trained on how to do unit tests but they were constantly focused on filling the gaps from the old system, as well as adding new features so they could bring in new clients quickly. Because of this, the team was less concerned with testing and more concerned with development, creating the potential for quality issues.

Solution

Ami decided that in order to solve his team’s challenges, they needed to start practicing Test Driven Development (TDD). TDD incorporates testing into the development process, guaranteeing a focus on quality as well as quick feedback cycles which are key to agile development. Datafaction began using SeaLights to support this move to TDD. 

SeaLights optimizes Datafaction’s TDD by:

  • Validating TDD: SeaLights Quality Risk feature shows teams if they have untested code changes in order to prevent them from reaching production. Datafaction’s team uses this feature to measure if they are doing TDD properly; if there are no untested code changes (which there shouldn’t be because the tests are written first) then the team was successful with their TDD. 

“I’ve always wanted to build quality into our pipeline and SeaLights brought the tools, information, and metrics that I needed to be able to go down this path and see this vision come to fruition…it has been the tool that has enabled us to successfully implement TDD.”

Ami Adler, Software Development Manager, Datafaction

  • Identifying untested code changes across all test types: SeaLights Quality Risk shows untested code changes across all test stages and test types (not just unit tests). With SeaLights, both the engineering and quality assurance (QA) teams were able to see that their untested code changes decreased by over 70% as a result of TDD and validated that their test development was focused and effective. 
  • Identifying critical areas to add automated tests: By focusing on untested code changes, the Datafaction team can also begin developing their test automation in the areas that need it the most, saving them time and resources while increasing their quality. 

“We use SeaLights not just to identify untested code changes, but also to start thinking about where our automation should go.”

Ami Adler, Software Development Manager, Datafaction

  • Empowering the development team to focus on software quality: Since the development team started using the SeaLights dashboard along with TDD they feel more responsible for the quality of their code. The dashboard provides them with positive feedback; if they truly accomplish TDD then their code coverage increases and their untested code changes decrease.

Next Steps

Datafaction’s Agile team will use SeaLights to move their testing from manual to mostly automated. The team will also integrate SeaLights into its day-to-day-development process and use the Quality Risk feature to establish quality gates, ensuring higher-quality releases. 

About Datafaction

Datafaction services the needs of business managers, family offices and sport management firms that offer bill pay and accounting services. Founded in 1976, Datafaction has more than 40 years of industry leadership and expertise.

About City National Bank

With $53.6 billion in assets, City National Bank provides banking, investment, and trust services through 70 offices, including 19 full-service regional centers, in Southern California, the San Francisco Bay Area, Nevada, New York City, Nashville, Atlanta, Minneapolis and Washington, D.C. In addition, the company and its investment affiliates manage or administer $75.1 billion in client investment assets.

City National is a subsidiary of Royal Bank of Canada (RBC), one of North America’s leading diversified financial services companies. RBC serves more than 16 million personal, business, public sector and institutional clients through offices in Canada, the United States, and 34 other countries. 

*SeaLights is not affiliated with City National Bank

About SeaLights

SeaLights is the #1 Quality Intelligence Platform for software development. The platform provides managers with actionable data analytics for both manual and automated tests so that they can focus their team’s development and testing cycles and make data-driven decisions about their releases, resulting in fewer incidents in production. For more information, fill out the form to the right.below.

About Test Driven Development (TDD)

Test Driven Development (TDD) is a development practice intended to encourage discipline, structure, and quality code. It requires developers to write initially failing test cases based on requirements and then write code that will pass those tests. The cycle is as follows: add a test, run all tests and see if the new test fails, write the code, run tests, refactor code, and repeat. TDD forces development teams to focus on the quality of the software, as well as the velocity of their development.

With SeaLights we know with confidence that we are covering 100% of the code that changes from build to build, and as a result, we have not had any untested code changes in the last 5 months with our legacy app of over 80,000 lines of code.

Ben Goldsmith, Software Engineering Manager, Pokémon