Achieving high software quality is a goal of every development organization. However, in order to achieve high quality, you must be able to measure and understand the level of quality you have today. Software quality is complex and difficult to define, not to mention measure on an ongoing basis.
We’ll use the ISO/IEC 25010 standard model to define 8 dimensions of software quality, and suggest some readily-available metrics that can help you measure each one of them. We hope this will help you identify where you stand and start on a road to improving quality in your software product.
Why is Quality Important? 5 Negative Effects of Low Quality Releases
Software quality measurement helps you understand the level of quality of each release, even each build, produced by your development team. In many cases teams are not aware of the level of quality of a release or build, and do not have a clear indication how quality is progressing over time – is product’s quality getting better or worse?
There are several negative effects of low quality releases:
Reduced productivity—low quality releases are often characterized by poor code quality and a high level of complexity. Software complexity has a negative effect on developer productivity – any change to the software becomes much more difficult, bugs can be very difficult to isolate and fix, and as a result, technical debt can grow.
Critical production failures—beyond the obvious effect of production faults, it also affects developer productivity. Your best developers have to “stop everything” and attend to issues in production, which may be difficult to diagnose and fix.
Employee turnover—employees who work on high-quality code are usually more satisfied with their work. Working on low quality, complex code and hunting down bugs from the past is frustrating for developers, and can result in significant turnover rates.
The ISO/IEC 25010 standard provides a useful model of 8 software quality dimensions. Below we’ll explain each dimension and suggest how to measure it in your software project, letting you catch quality issues early and react to them.
How to Measure
How easily you can modify or adapt your software for use by other teams, new purposes or changing requirements. The more complex your code is, the harder it will be to maintain. High quality code is easier to work with and can be easily adapted to new requirements.
Lines of Code (LOC)
Static code analysis
Cyclomatic complexity (and other complexity measurements)
How easily you can move a software component to another environment. Robust software should be able to run on multiple operating systems and in different environments, for example on the cloud or on-premises, and dependencies on other software should be minimized and well defined.
Setting up realistic testing environments
Automating tests of installation/uninstallation
UI and regression tests to identify problems on specific platforms
How well software functions compared to user expectations. Functionality is the “what” – if the software does what the user needs it to do, it is functioning properly.
How well software performs under a workload. Performance includes response time and latency, scalability, and stability under maximum expected loads.
Page load times
How well software runs on different browsers, devices, or operating systems. Modern applications are expected to work seamlessly across a large matrix of devices and screen sizes.
Cross browser testing
How easily users can understand and use software functionality. UI/UX is a major focus in modern software development. Software needs to be prototyped and tested with usability in mind, and then continuously monitored to see if new usability issues come up.
User acceptance testing
Website engagement and conversion
Real user monitoring
How likely a software system is to fail or malfunction. The more reliable your software, the less likely users are to experience bugs in production.
How vulnerable software is to cyber attacks. The more secure your software, the better it protects users against breaches and service interruption as a result of malicious or accidental human actions.
Number of incidents
Number of breaches
Existing known vulnerabilities
Mean Time to Repair (MTTR)
Towards a Holistic Measurement of Software Quality
We presented several dimensions of software quality and readily-available metrics you can use to measure them. However, none of these metrics provides a holistic view of software quality. Development teams need actionable information about which parts of their products suffer from quality issues, and where to focus maintenance to have the biggest impact on software quality. A new category of tools called Software Quality Intelligence can help. For example, SeaLights is a quality intelligence platform that helps visualize test gaps – product features that are used in production, have undergone recent changes, and are not sufficiently tested across all relevant dimensions of software quality. SeaLights can report which areas of the product are at high risk of quality issues – helping you prioritize work in your sprints to quickly improve software quality. Request a live demo to see how SeaLights can help you visualize and improve software quality.