Aren't we overdoing it? I mean the measurement? Sure, we can measure the utilization of the AI tools, how much code gets generated, etc.
Risking controversy: it's largely useless.
Measuring utilization was a red herring, even when we were measuring human work. Ask any Lean Management or Theory of Constraints folks, and they (we?) will rant about it as long as you want.
Generated code would only be interesting if the act of generation were unassisted. If a developer spent an hour writing the production-ready code and another developer spent the same hour delivering the same code but used the time to prompt, re-prompt, and review code, is there really a difference?
And it just so happens that we have (or should have) a good compound metric that allows us to integrate the vast majority of these nuances into one dimension.
*How much value are we delivering over time as compared to when we haven't used AI tools?*
In that aspect, it may actually be interesting to measure tool utilization one way or another to have a reference dimension.
But that's it. How much more value are we delivering?
And for the companies that are clueless about the actual value they create, they may use a less useful, albeit much easier to answer, question:
*How has our big-picture throughput changed?*
In other words, how many more value-adding features are we delivering?
Because, with the AI tools, I can easily optimize *some* aspects of my work by 100%. However, if it creates more work for others down the line, the aggregated gain may not be nearly as impressive.
A simple example is generating a large chunk of code quickly and shifting the cognitive load of ensuring it works well and doesn't break anything to a person conducting a code review. I just got super-fast. And the fact that the team delivers at the same (or slower) pace, well, who cares?
So, you want to know how much better your new shiny AI tools made you? Ask product people.
Aren't we overdoing it? I mean the measurement? Sure, we can measure the utilization of the AI tools, how much code gets generated, etc.
Risking controversy: it's largely useless.
Measuring utilization was a red herring, even when we were measuring human work. Ask any Lean Management or Theory of Constraints folks, and they (we?) will rant about it as long as you want.
Generated code would only be interesting if the act of generation were unassisted. If a developer spent an hour writing the production-ready code and another developer spent the same hour delivering the same code but used the time to prompt, re-prompt, and review code, is there really a difference?
And it just so happens that we have (or should have) a good compound metric that allows us to integrate the vast majority of these nuances into one dimension.
*How much value are we delivering over time as compared to when we haven't used AI tools?*
In that aspect, it may actually be interesting to measure tool utilization one way or another to have a reference dimension.
But that's it. How much more value are we delivering?
And for the companies that are clueless about the actual value they create, they may use a less useful, albeit much easier to answer, question:
*How has our big-picture throughput changed?*
In other words, how many more value-adding features are we delivering?
Because, with the AI tools, I can easily optimize *some* aspects of my work by 100%. However, if it creates more work for others down the line, the aggregated gain may not be nearly as impressive.
A simple example is generating a large chunk of code quickly and shifting the cognitive load of ensuring it works well and doesn't break anything to a person conducting a code review. I just got super-fast. And the fact that the team delivers at the same (or slower) pace, well, who cares?
So, you want to know how much better your new shiny AI tools made you? Ask product people.