How to use engineering metrics for the success of engineers and teams
Measuring the right things ensures that we inspire the right actions!
DevStats (sponsored)
With DevStats, you can improve your process, identify bottlenecks, and ship better products, faster.
Insights based on all 3 main frameworks: DORA, SPACE and DevEx
Deliver more business impact and make data-driven decisions
Align your software development with business outcomes
If you haven’t yet, I highly recommend checking DevStats out! No credit card is required to start your trial.
Let’s get back to this week’s thought.
Intro
When it comes to metrics, this is VERY important to keep in mind:
We inspire the actions of individuals by the way we measure their success.
If you check and measure the right things, people will naturally be inclined to put in the effort that is aligned with that.
Checking the wrong things will inspire the individuals to “game” the system and the outcome is going to be a culture of competing and not helping each other.
This can cause quite a big problem inside an organization and can ultimately lead to downfall.
There has been quite a negative reputation around metrics in the last year
Especially from the publishing of this article by McKinsey, where they stated that you can accurately measure developer productivity by employing specific objective measures.
I’ve responded with how I measure developer productivity in this article: How I measure developer productivity (paid article). To do a quick recap, here are the main directions of what I look for:
Team productivity > Developer productivity
Helping others > Completing your own tasks
Building the RIGHT things > Amount of things being built
Specifically, this is what I look for when checking developer productivity:
Are they focusing on building the RIGHT things and challenging requirements.
How much are they helping others.
How are they contributing to the success of the whole team/organization.
What improvements have they implemented and ensure they get adopted by other engineers.
You can read more about how I measure and check for these specific things in detail in the article.
What metrics do I use for teams
There are 2 specific things I like to do, it depends a lot on how many teams I manage and also the number of people that are in the org.
As mentioned in the article, I like to check how much business value a specific team delivers in a certain period. And I like to use OKRs for that.
I specifically like to map the team-level OKRs to org-wide objectives. These 3 things are very important when defining OKRs:
clearly define the objective,
let the team decide on key results,
have regular check-ins to see if we are on track (I suggest monthly).
This ensures clarity on the most important things → delivering business value.
For mid to large-size companies (5+ engineering teams) I always recommend having some additional metrics specific to engineering org. The bigger the company, the more important it is to have some more tangible info.
The reason is that it really helps to see specific org-wide level deviations and it gives you information that you would normally miss or you would expect it to be on the right level.
A good example from my experience is PR Cycle Time and Code Review Size. I thought that we were doing well, but then I started to use DevStats for my team and I saw that we could make improvements in that front.
My recommendations are DORA, SPACE, DevEx and also other system metrics like uptime/downtime, errors, database load, API response time, etc.
No need to incorporate all different metrics in DORA, SPACE or DevEx. You can start with the ones that make the most sense for your org and the teams and add/remove them if needed.
I normally go for 1 or 2 specific metrics at a time and check them together with the teams to see what improvements we can make.
I like what
, ex-Engineering Director at Google mentioned:At Google, they used the DORA’s metric Lead Time for Changes (submit2prod) as a success metric.
DORA metrics
DORA metrics evaluate key aspects of software delivery to identify issues and discover opportunities for improvement:
Deployment Frequency – How often you release a new version or update for your project.
Lead Time for Changes – The time from code commit to when it's deployed to production.
Change Failure Rate – The percentage of deployments requiring rollbacks due to bugs or incomplete features.
Time to Restore Service – How long it takes to restore service to a stable state after an incident caused by a change in production.
SPACE metrics
SPACE outlines five key dimensions for assessing productivity more accurately:
Satisfaction and wellbeing: this measures how fulfilled developers feel with aspects like their work, team, tools, or company culture.
Performance: focuses on the outcomes produced by a system or process.
Activity: tracks the number of actions or outputs completed during the course of work.
Communication and collaboration: evaluates how effectively individuals and teams communicate and work together.
Efficiency and flow: refers to the ability to make progress or complete tasks with minimal interruptions or delays, either individually or within a system.
DevEx metrics
DevEx encompasses how developers feel about, think about, and value their work.
The framework distills developer experience to its three core dimensions: feedback loops, cognitive load, and flow state.
Feedback loops
Fast feedback loops allow developers to complete their work quickly with minimal friction. Slow feedback loops, by contrast, interrupt the development process, leading to frustration and delays as developers wait or decide to switch tasks.
Cognitive load
When cognitive load remains high as a result of problems such as poorly documented code or systems, developers need to devote extra time and effort to complete tasks and avoid mistakes.
Flow state
Frequent experiences of flow state at work lead to higher productivity, innovation, and employee development. Similarly, studies have shown that developers who enjoy their work perform better and produce higher-quality products.
Whenever you’re incorporating metrics, always start by creating a great culture with the right values first
It’s never a good idea to incorporate metrics in the background and use them for your interpretation only.
For every project or company I work with, I make sure that it’s very clear what our core values are and what the overall expectation is.
These are the core values, which I created in Zorion and they were located at the top of the first page of the Technology Notion workspace, so everyone can see them anytime.
When it comes to metrics, it helps so that people know and realize that you have the right intentions.
It’s hard to go wrong if you have the attitude of “How can metrics help us to be successful” and not “How can we use the metrics to blame”.
Let’s get more into specific things about what to do and what not.
What to do and what not to do when it comes to metrics
It’s crucial that you interpret metrics the RIGHT way and not have them for the wrong reasons.
Here is what you SHOULDN’T do:
❌ Don’t blame people because of metrics
❌ Don’t assess productivity solely based on metrics
❌ Don’t try to incorporate all of them at once, focus on the ones that really matter and add/adjust.
Here is what you SHOULD do:
✅ Use metrics for learning - yours and the team’s learning + understanding deviations
✅ Be transparent regarding how they are used and how they can help
✅ The focus on metrics should be on teams and not on individuals
Last words
We’ve gone through a lot of stuff today and it’s easy to get overloaded with too much info on all the specific frameworks.
My advice is to keep it as simple as possible.
Start with some of the most important 1-3 metrics for your organization and then adjust based on what you see helping and what not.
We are not over yet!
Senior Engineer to Lead: Grow and thrive in the role (announcement)
It’s getting close to the start of the November’s cohort-based course! It’s going to start on November 12. I am very excited for it!
Here is what students are saying:
For the next 3 days, you can enroll to the course with 25% off. Instead of paying $600, you pay $450. Use code: ENG25 or visit the link below!
Looking forward to seeing you there.
Liked this article? Make sure to 💙 click the like button.
Feedback or addition? Make sure to 💬 comment.
Know someone that would find this helpful? Make sure to 🔁 share this post.
Whenever you are ready, here is how I can help you further
Join the Cohort course Senior Engineer to Lead: Grow and thrive in the role here.
Interested in sponsoring this newsletter? Check the sponsorship options here.
Take a look at the cool swag in the Engineering Leadership Store here.
Want to work with me? You can see all the options here.
Get in touch
You can find me on LinkedIn or Twitter.
If you wish to make a request on particular topic you would like to read, you can send me an email to info@gregorojstersek.com.
This newsletter is funded by paid subscriptions from readers like yourself.
If you aren’t already, consider becoming a paid subscriber to receive the full experience!
You are more than welcome to find whatever interests you here and try it out in your particular case. Let me know how it went! Topics are normally about all things engineering related, leadership, management, developing scalable products, building teams etc.
For smaller teams, any metric that revolves around adding value is what matters the most, and metrics like flow state and well being of people are secondary.
The path you’ve suggested is pretty straightforward and efficient to implement (these teams often consider that evaluating integration performance can be difficult and time-consuming, so they overlook it)
Loved this! Specially your core values list at Zorion!