I'm with you in that I believe business impact is a massive one to measure. If anything, delivery of value to the business (and ultimately end users) is what really matters here. Business value can include things like:
- cost saving (e.g. optimising infra usage/cost)
- revenue growth (e.g. building features that drive customer acquisition or retention)
- reducing tech debt (e.g. making the systems more maintainable)
- improving operational efficiency (e.g. creating automations or enhancing tooling to reduce waste)
As you say, there's no one size fits all and that's the problem. Even with the above in mind, how would one quantify someone being more or less productive if they tackled tonnes of tech debt vs. shaving $1m off infra costs?
As for using PRs as a measure, I won't open that can of worms just yet.
I am also very curious how using PRs as a measure will be received by engineers. A couple of leaders I have spoken to who have implemented DX4 have said that people didn't mind too much about it. But there's a whole comms piece that is important to do well with this framework.
Thanks for putting this review together, it’s very helpful.
I think that the most important place to start is by asking the most important question: “What problem are you trying to solve?” As you mentioned at the start, different people will have different answers.
The problem is that whatever system you end up using, is the system that everyone will latch on to. So, how you frame and communicate the metrics is extremely important.
In my mind, one of the key pillars is different than the others: Business Impact.
All the other pillars are upstream of Business Impact.
- Slow to deliver? Then slow to create business impact.
- Low quality? Then less business impact.
- Doesn’t provide individual growth? Then less growth and less business impact.
- Low care? Lower retention and morale, hence less business impact.
And on the flip side, you can have a fast, high quality dev team that works on challenge projects that they love. And they create absolutely no business impact.
All great points Gilad, thank you for taking the time to writhing this. I do believe that all pillars I mentioned should be somehow captured in a productivity framework. Many of them lack basic parts (ie DORA).
I also think that it's a miss that we're not able as an industry to define what's productivity consistently. Or be able to assess if a given team is productive or not. In my opinion we rely too much on judgement when there are so many metrics at our disposal. We just haven't found the right formula.
The comms part of ALL of this which you mentioned is without a doubt extremely important!
Hi, thanks for your perspective. It's nice summary, I spot the same issue in the existing frameworks, especially lack of enough emphasis on the business impact. Btw spent 10 years in software development, nobody ever even shown me nor explained any business metrics.
I am following DX (and their release of DX4) for a while too, and I find it still not complete.
As you wrote yourself at the end, when you want to measure business impact, you take (as qualitative) product owner and developers opinion, and (as quantitative) % of time spent on features.
None of these two present direct impact on the business outcome.
In order to truly measure the teams by their business impact the organizational change must happen.
Teams have to be organized toward business model’s parameters, like acquisition, convertiom, retention, recommendation and so on.
only then we can directly measure them against these needles.
I think currently ots not possible because most of the time teams are organized in siloes (e.g. backend, frontend, DBA, DevOps(infrastructure)) and they have no direct impact on the parameters mentioned above.
On the technical side it has been discovered long time ago, and it's the Conway's law. We use inverted conways maneuvre to migrate from monolithical architecture to microservices by reorganizing the teams topology first.
I just wonder if the same maneuvre should be utilized when trying to organize the teams towards the business objectives/domains
Great points! I absolutely agree that teams need to align with key business metrics, not just functions like backend or DevOps - for many good reasons.
Applying an inverse Conway’s maneuver to business objectives is an interesting idea which I hadn't come across! Thank you for sharing it.
Interesting read, thanks for sharing.
I'm with you in that I believe business impact is a massive one to measure. If anything, delivery of value to the business (and ultimately end users) is what really matters here. Business value can include things like:
- cost saving (e.g. optimising infra usage/cost)
- revenue growth (e.g. building features that drive customer acquisition or retention)
- reducing tech debt (e.g. making the systems more maintainable)
- improving operational efficiency (e.g. creating automations or enhancing tooling to reduce waste)
As you say, there's no one size fits all and that's the problem. Even with the above in mind, how would one quantify someone being more or less productive if they tackled tonnes of tech debt vs. shaving $1m off infra costs?
As for using PRs as a measure, I won't open that can of worms just yet.
I am also very curious how using PRs as a measure will be received by engineers. A couple of leaders I have spoken to who have implemented DX4 have said that people didn't mind too much about it. But there's a whole comms piece that is important to do well with this framework.
We've done a full circle !
Thanks for putting this review together, it’s very helpful.
I think that the most important place to start is by asking the most important question: “What problem are you trying to solve?” As you mentioned at the start, different people will have different answers.
The problem is that whatever system you end up using, is the system that everyone will latch on to. So, how you frame and communicate the metrics is extremely important.
In my mind, one of the key pillars is different than the others: Business Impact.
All the other pillars are upstream of Business Impact.
- Slow to deliver? Then slow to create business impact.
- Low quality? Then less business impact.
- Doesn’t provide individual growth? Then less growth and less business impact.
- Low care? Lower retention and morale, hence less business impact.
And on the flip side, you can have a fast, high quality dev team that works on challenge projects that they love. And they create absolutely no business impact.
All great points Gilad, thank you for taking the time to writhing this. I do believe that all pillars I mentioned should be somehow captured in a productivity framework. Many of them lack basic parts (ie DORA).
I also think that it's a miss that we're not able as an industry to define what's productivity consistently. Or be able to assess if a given team is productive or not. In my opinion we rely too much on judgement when there are so many metrics at our disposal. We just haven't found the right formula.
The comms part of ALL of this which you mentioned is without a doubt extremely important!
Hi, thanks for your perspective. It's nice summary, I spot the same issue in the existing frameworks, especially lack of enough emphasis on the business impact. Btw spent 10 years in software development, nobody ever even shown me nor explained any business metrics.
I am following DX (and their release of DX4) for a while too, and I find it still not complete.
As you wrote yourself at the end, when you want to measure business impact, you take (as qualitative) product owner and developers opinion, and (as quantitative) % of time spent on features.
None of these two present direct impact on the business outcome.
In order to truly measure the teams by their business impact the organizational change must happen.
Teams have to be organized toward business model’s parameters, like acquisition, convertiom, retention, recommendation and so on.
only then we can directly measure them against these needles.
I described the idea here:
https://open.substack.com/pub/maxpiechota/p/customer-success-driven-metrics-for?utm_source=share&utm_medium=android&r=2jrq6c
I think currently ots not possible because most of the time teams are organized in siloes (e.g. backend, frontend, DBA, DevOps(infrastructure)) and they have no direct impact on the parameters mentioned above.
On the technical side it has been discovered long time ago, and it's the Conway's law. We use inverted conways maneuvre to migrate from monolithical architecture to microservices by reorganizing the teams topology first.
I just wonder if the same maneuvre should be utilized when trying to organize the teams towards the business objectives/domains
Great points! I absolutely agree that teams need to align with key business metrics, not just functions like backend or DevOps - for many good reasons.
Applying an inverse Conway’s maneuver to business objectives is an interesting idea which I hadn't come across! Thank you for sharing it.