11 Scrum Metrics and Their Value to Scrum Teams
Why use scrum metrics, and which metrics should you use to meet your objectives and report to stakeholders? In this article you’ll learn:
- How scrum framework events can be used to glean important information about team progress
- 4 scrum metrics that can help measure delivery to customers
- 4 scrum metrics that can help measure team effectiveness in meeting business goals
- 3 scrum metrics that can be used to measure team and process health
- Which metrics to use to report to stakeholders
- The missing metric in scrum development projects: software quality
What is Scrum?
Scrum is an agile methodology for managing development projects. It uses the concept of time scoping to structure and estimate work. A scrum team operates in small increments called sprints, usually between 1-4 weeks. The scope of work is defined according to how much the team feels it can achieve within one sprint. In each sprint, the work delivered should be ready to be delivered to customers.
Scrum Events and Their Value to Measuring Progress
Scrum has several recurring events that help manage iterations and provide process control. Each of these can help provide meaningful metrics and data to the team and management:
- Sprint planning—A meeting at the start of a sprint where high-level story descriptions are broken down into detailed tasks. Provides a detailed estimate for the scope of work that can be produced during the sprint.
- Daily scrum—A daily stand up meeting in which team members share progress and problems they are facing. Provides reporting about hours remaining for sprint tasks, which helps generate the sprint burndown metric.
- Sprint retrospective—A summary meeting at the end of the sprint to share what went well, what went less well, and ideas for improvement. Provides qualitative information that can help assess the health of the scrum team and the process.
What are Scrum Metrics and KPIs?
Scrum metrics and KPIs are part of a broader family of agile KPIs. Agile metrics include lean metrics, which focus on the flow of value from an organization to its customers, and Kanban metrics, which focus on workflow and getting tasks done. While most agile metrics are applicable to scrum teams, scrum-specific metrics focus on predictable software delivery, making sure scrum teams deliver maximum value to customers with every iteration.
Scrum KPIs have three major goals:
- To measure deliverables of the scrum team and understand how much value is being delivered to customers.
- To measure effectiveness of the scrum team; its contribution to the business in terms of ROI, time to market, etc.
- To measure the scrum team itself in order to gauge its health and catch problems like team turnover, attrition and dissatisfied developers.
Scrum Metrics—Measuring Deliverables
The following metrics can help measure the work done by scrum teams and value delivered to customers:
1. Sprint Goal Success
A Sprint Goal is an optional part of the scrum framework. As defined by Scrum.org, it answers three questions: Why are we carrying out the sprint? How do we reach the sprint goal? What metric tells us the goal has been met? For example, a goal might be delivering a feature, addressing a risk, or testing an assumption.
Image Source: Roman Pichler
By defining sprint goals and then measuring how many sprints met the goal, you can get a qualitative assessment of a scrum team’s work. Not just how many story points are completed, but how frequently the objectives of the business are met.
2. Escaped Defects and Defect Density
Escaped defects is a crucial metric that shows how many bugs were experienced by users in production. Ideally, a scrum team should fully test stories and completely avoid escaped defects. In reality, this rarely happens, but the trend of escaped defects is a good signal of product quality.
Defect density is also worth watching—it measures number of defects per software size, for example per lines of code (LOC). While this metric can easily be skewed, it is valuable in fast-moving projects to check if growth in defects is “normal” given the growth of the underlying codebase.
3. Team Velocity
Velocity measures how many user stories were completed by the team, on average, in previous sprints. It assists in estimating how much work the team is able to accomplish in future sprints.
While velocity is a key metric to watch in any scrum project, experts warn against using it as a goal, or using velocity to compare teams to each other. Velocity is a subjective measure (based on each team’s definition of story points) that captures the team’s progress. Trying to artificially increase velocity can act to erode trust and reduce transparency between teams and management.
4. Sprint Burndown
The sprint burndown chart is the classic representation of progress within a sprint. It shows the number of hours remaining to complete the stories planned for the current sprint, for each day during the sprint. The sprint burndown shows, at a glance, whether the team is on schedule to complete the sprint scope or not.
Scrum Metrics—Measuring Effectiveness
The following metrics can help assess the effectiveness of scrum teams in meeting business goals:
1. Time to Market
Time to market is the time a project takes to start providing value to customers, or the time it takes to start generating revenue. The first can be calculated by taking the length of the number of sprints before a scrum team releases to production. The second could be longer, depending on the organization’s alpha and beta testing strategy.
2. ROI
Return on Investment (ROI) for a scrum project calculates the total revenue generated from a product vs. the cost of the sprints required to develop it. Scrum has the potential to generate ROI much faster than traditional development methods, because working software can be delivered to customers very early on. With each sprint, scrum teams create more features that can translate into growth in revenue.
3. Capital Redeployment
Capital Redeployment measures if it’s worthwhile to continue a scrum project, or if the economic value of the project now exceeds its costs. In this case the team should be redeployed to other, more profitable projects.
To determine Capital Redeployment, calculate the revenue value of the remaining items in the project backlog (V), the actual cost (AC) of the sprints needed to complete those items, and the opportunity cost (OC) of alternative product work the team could do. When V < AC + OC, the project should end and the team redeployed to other projects.
4. Customer Satisfaction
There are several well-known metrics used to measure customer satisfaction. One is the Net Promoter Score (NPS), which measures if users would recommend the software to others, do nothing, or recommend against it. Using a consistent customer satisfaction metric and measuring it for every release indicates whether the scrum team is meeting its end goal—to provide value to customers.
Scrum Metrics—Monitoring the Scrum Team
These metrics can help a scrum team monitor its activity and identify problems early on, before they impact development:
1. Daily Scrum and Sprint Retrospective
These two scrum events, if carried out regularly with well-documented conclusions, can provide an important qualitative measurement of team progress and process health.
2. Team Satisfaction
Surveying the scrum team periodically to see how satisfied they are with their work can provide warning signals about culture issues, team conflicts or process issues.
3. Team Member Turnover
Low turnover (replacement of team members) in a scrum team indicates a healthy environment, while high turnover could indicate the opposite. Also contrast this metric with overall company turnover, which can impact the scrum team.
Scrum Reporting—Which Metrics to Report to Stakeholders?
The most important thing stakeholders need to know about your scrum project is whether it is on track. The following metrics might help communicate this, and explain deviations from the expected project path:
- Sprint and release burndown—Gives stakeholders a view of your progress at a glance.
- Sprint velocity—A historic review of how much value you have been delivering.
- Scope change—The number of stories added to the project during the release, which is often a cause of delays (many agile tools can show this automatically).
- Team capacity—How many developers are on the team full time? Has work capacity been affected by vacations or sick leave? Developers pulled off to side projects?
- Escaped defects—provides a picture of how your software is faring in production.
The Missing Scrum Metric—Software Quality
There’s been one thing missing in all the scrum metrics we covered—software quality. The escaped defects metrics alone provides a view of quality, and it is an imperfect metric which identifies quality issues only after they are released to production.
Quality is critical to scrum projects. Stories completed do not provide value unless they are tested and working as the customer expects. Existing tooling only provides fragmented stats, such as unit test coverage and number of tests executed—it does not provide a good picture of the overall quality status.
To learn more about the missing metric in agile development, and how to gather the missing data to gain visibility of software quality, read our white paper, Quality: The Missing Metric for Development Managers .