Measures and The Learning Journey

At Lean Agile Intelligence, we believe that the Measures are a crucial part of the team's ability to ensure team members understand how to use metrics for improvement and decision making. We have organized the learning journey into four distinct stages - Developing, Emerging, Adapting, and Optimizing - and will cover each of these stages in-depth, providing you with practical advice and techniques to enhance your skills in this area.

 

Source - Actionable Agile

Metric Quick Tips

• As seen in the image above, metrics should not be targets but ways of driving rich conversations where we ask the right questions to help us improve.

• Always strive to achieve balance when tracking metrics. Focusing on only one area can be counterproductive to other areas.

• Utilize Lean Agile Intelligence's outcome-driven approach and map your own metrics to business outcomes.

Developing

Teams or an Organization “developing” an understanding of the value of Measures and adopting the foundational techniques should focus on the following improvements:

• The What: Metrics are tracked

  • The How: The first step in developing foundation techniques of metrics is simply tracking metrics. Two key points to focus on when starting to track:

    • Align on the metrics as a group and with your key stakeholders. Add your metrics to your charter after agreeing to the set of metrics to track and share your intended metrics with key stakeholders for feedback and alignment.

    • Have a centralized transparent “single source of truth” for your metrics. Make them easily accessible to anyone interested and include them in your norms.

• The What: Metrics are frequently reviewed

  • The How: It's important to ensure that the review of metrics is done in a collaborative and constructive manner, focusing on continuous improvement and team learning rather than blame or individual performance evaluation. Metrics should be used as a tool to facilitate discussions, identify areas for improvement, and drive team performance, rather than as a source of pressure or punishment. Having regular team meetings scheduled and following the agile practices of having Daily Stand Ups, Sprint Reviews, and Sprint Retrospectives will ensure that metrics are frequently reviewed. 

• The What: Metrics are actionable and drive improvements

  • The How: A common mistake teams make is tracking “vanity metrics” or metrics that someone ELSE wants them to track without being relevant to their context. In her post “Vanity Metrics vs Actionable Metrics: A Guide to Tracking the Right Stats” Laura Caveney outlines the key differences between the two styles of metrics. In summary, “vanity” metrics are easy to track but aren’t indicative of real performance and often are used to make the creator “look good.” In contrast, “actionable” metrics are tied to a specific outcome and can be used for conversations that lead to actionable improvement for that outcome. For a deeper dive, we recommend the book “Actionable Agile Metrics for Predictability” by Dan Vacanti.

 

 

Emerging

Teams or an Organization “emerging” beyond the foundational techniques of Measures and are embracing it as they become more proficient should focus on the following improvements:

• The What: Different metrics categories exist (i.e., quality, time to market, customer satisfaction, employee satisfaction) and are leveraged to drive improvements

  • The How: Teams often start with tracking delivery metrics, but quality metrics are arguably more important than tracking delivery metrics. Edwards Demming has a famous quote “Improve quality; you automatically improve productivity.” Start by tracking a few quality metrics that are lagging indicators like escaped defects, production issues, customer reports, etc, and measure leading indicators along the way such as the assessments in Lean Agile Intelligence focused on quality. Here is a blog post we wrote that goes into this topic In more detail.
    
    
  • The How: Tracking metrics is more of an Art than an exact science, the more you focus on one metric, it can have potential unintended consequences that you may not be aware of somewhere else. Part of achieving a balanced metric framework is considering the human experience. We recommend tracking employee satisfaction/happiness/engagement along with delivery, quality, and business metrics. Lean Agile Intelligence offers out-of-the-box practices you can utilize or even create your own survey/assessment from scratch. Another way to get started is to use an in-person or virtual version of the “Happiness Door.”

• The What: Flow Metrics (i.e., throughput, cycle time) are leveraged for work units (i.e., Epics, User Stories)

  • The How: There are four key flow metrics to focus on when tracking work items. These flow metrics apply to every level inside your organization, starting with agile teams. The metrics are:

    • Cycle Time: The amount of elapsed time between when a work item started and when a work item finished.

    • WIP: The number of work items started but not finished.

    • Throughput: The number of work items finished per unit of time. Note the measurement of throughput is the exact count of work items.

    • Work Item Age: The amount of elapsed time between when a work item started and the current time.

• The What: Metric trends are the focus, not absolute numbers

  • The How: While absolute numbers are easy to game and suspect to Goodhart’s Law, trends can be used to drive rich improvement conversations between teams. From a systems view, we can see trends across teams and identify impediments and areas of growth.

 

*   *   *   *   *   *

"Have a centralized transparent “single source of truth” for your metrics."

*   *   *   *   *   *

 

Adapting

Teams or an Organization that is “adapting” the Measures practice to extract the full benefit should focus on the following improvements:

• The What: Leading and lagging indicators exist for each metric category (i.e., quality, time to market, customer satisfaction, employee satisfaction)

  • The How:

    • Establish a set of leading and lagging indicators that tie back to team goals.

    • Utilize Lean Agile Intelligence measures as a leading indicator to ensure you are on the right path to achieve those goals.

• The What: Different Metrics are experimented with, and learnings are shared with others
  
  
  • The How: Metrics are there to help teams improve. Start with the outcomes you want to achieve, then relate a set of balanced metrics to those outcomes. Over time add, remove, or tweak metrics to achieve the outcomes as you see fit. Metrics should be actionable. In his book “Actionable Agile Metrics for Predictability” Dan Vacanti details the metrics that drive improvement conversations. Flow metrics help us ask the right questions to identify improvement areas. Frequent reviewing with high visibility leads to faster feedback and decision-making.

  • The How: The team can experiment with tracking metrics such as “Net Promoter Score®, or NPS®. NPS measures customer experience and predicts business growth. This proven metric transformed the business world and now provides the core measurement for customer experience management programs worldwide.

    • NPS is one type of customer satisfaction metric in a survey format. Measuring customer usage using quantifiable data is critical to understanding how customers use your product, where they are struggling, and where to apply more areas of focus. For a more in-depth blog post about this topic, visit here.

    • Calculate your NPS using the answer to a key question, using a 0-10 scale: How likely is it that you would recommend [brand] to a friend or colleague?

      Respondents are grouped as follows:

      • Promoters (score 9-10) are loyal enthusiasts who will keep buying and referring others, fueling growth.

      • Passives (score 7-8) are satisfied but unenthusiastic customers are vulnerable to competitive offerings.

      • Detractors (score 0-6) are unhappy customers who can damage your brand and impede growth through negative word-of-mouth.

      Subtracting the percentage of Detractors from the percentage of Promoters yields the Net Promoter Score, which can range from a low of -100 (if every customer is a Detractor) to a high of 100 (if every customer is a Promoter).” - Source, NetPromoter.com

•  

Source - Goodhart's Law

 

Optimizing

Teams or an Organization “optimizing” knowledge sharing of the Measures practice learnings across the enterprise should focus on the following improvements:

• The What: Metrics for team goals and incentives are objective and aim at measuring the impact of deliverables

  • The How: Metrics should be used judiciously and in a way that aligns with the Agile principles of collaboration, self-organization, and continuous improvement. Here are some metrics that teams can use to set team goals and incentives that are objective and aimed at measuring the impact of deliverables:

    1. Outcome-Based Metrics: Agile teams can use outcome-based metrics to measure the impact of deliverables on the desired business outcomes or customer value. For example, if the team is working on a software product, outcome-based metrics could include customer satisfaction scores, revenue generated from the product, number of users or customers acquired, or customer retention rates. 

    2. Key Performance Indicators (KPIs): KPIs can be quantitative measures that are specific, measurable, achievable, relevant, and time-bound (SMART). For example, if the team's goal is to improve software quality, KPIs could include metrics such as defect density, code coverage, or cycle time for bug fixes. KPIs provide objective and measurable targets that the team can strive to achieve, and incentives can be tied to meeting or exceeding these targets.

    3. Lean Metrics: Agile teams can use lean metrics, such as lead time, cycle time, or throughput, to measure the efficiency and effectiveness of their workflow. These metrics provide insights into how quickly work is flowing through the team's process, how much work is in progress, and how long it takes to deliver value to customers. 

    4. Quality Metrics: Quality metrics could include defect counts, defect severity or priority, test coverage, or customer-reported issues. These metrics provide objective measures of the quality of the team's deliverables and can be used to set goals and incentives around improving quality, reducing defects, or increasing customer satisfaction.

    5. Team Health Metrics: These metrics could include team morale, team member engagement, feedback scores from team members, or team retrospectives. Team health metrics provide insights into how the team is functioning as a unit and can be used to set goals and incentives around team collaboration, communication, and overall well-being.

    6. Customer Feedback Metrics: Customer feedback metrics provide insights into how well the team's deliverables are meeting customer needs and expectations and can be used to set goals and incentives around customer satisfaction, retention, and loyalty.

       

*   *   *   *   *   *

"While absolute numbers are easy to game and suspect to Goodhart’s Law, trends can be used to drive rich improvement conversations between teams."

*   *   *   *   *   *

 

Conclusion

In conclusion, metrics are not just numbers to track but a means to drive conversations and improvements. By following the tips and techniques outlined in this post, teams can develop a balanced metric framework that promotes accountability, identifies areas for improvement, and drives performance. Whether you are just starting to track metrics or are emerging beyond the foundational techniques, there is always room for improvement. By staying focused on the right metrics and leveraging advanced techniques like flow metrics, predictability measures, and customer-centric measures, any team can achieve its goals and improve its performance. If you want to gain a comprehensive understanding of your team’s current process status, we recommend taking advantage of our free agile assessment for Product Team Agility or for DevOps Team Agility.