Metrics and KPIs: Managing upwards & measuring success

What are some metrics developers run into?

Some may argue that hours of work is not a metric, yet some companies still measure how much time people spend on coding. Essentially if you work less than 8 hours a day, you might have a few things to discuss with your manager. What’s the problem with this metric? There are many scientific studies that show there are different kinds of work that can be done in 8 hours. For instance, you can do coding for 8 hours a day but you can’t do architecture planning for 8 hours a day—your mind would just explode. If your company uses hours of work as a metric, they’re not catering to different kinds of work. There are days when you might work 2 hours and spend the rest of the time thinking, so this metric is absurd.

There are also metrics like number of bugs, number of times a feature returned to development from testing, number of commits in a time period, and number of features done in a time period. All of these metrics provoke unwanted behavior that either makes it slower to ship code or makes it hard for people to focus on doing their best work in the most efficient way. None of these metrics are meaningful in and of themselves.

Another metric is % of features done “on time.” If you want developers to estimate how much time a certain feature might need, the unwanted behavior is that they will start adding more and more time and buffer to their estimation to make sure they ship on time. Similarly with number of comments in merge requests, the consequence is that the developers are seen as incompetent because they didn’t spend enough time writing readable code. Sometimes, developers will game this by talking to others by submitting merge requests.

What are some consequences of metrics?

Every metric has consequences. Some of them are hard to measure, or even impossible. So what happens if we had a metric of the number of features per month per developer, and it starts conflicts within the team? For instance, someone may not want to take on bigger features because they would finish fewer features per month. In this case, how would you measure the negative impacts on the team morale? Instead of having a team working on something to satisfy the customer, and make the customer happier, you would have started a conflict in the team and you can’t measure these conflicts.

What’s even worse, it’s not only impacting individuals, but it’s impacting the whole team. In psychology, group dynamics refers to a system of behaviors and psychological processes that occur within a social group. With conflicts and competitions, you won’t be able to assemble a team, a powerful one, to satisfy the requirements of our customers. The more metrics you introduce, and the more conflicts are introduced, the negative impacts will multiply. Balancing out one metric with another doesn’t work because you can’t predict the outcome and consequences each new metric might introduce.

Metrics and KPIs blindfold us. Whenever we see numbers, we tend to stick to them. It’s hard for us to unsee them. What happens often is that teams may be reaching their KPIs, but what’s not seen is the teams may be burning out. When developers on a team burnout, people will start to leave. The key to success is to provide a good work environment where employees feel they're closely aligned with the end users and strategic goals.

What are the alternatives to metrics?

If your company claims that they have hired highly intellectual and self-motivated engineers, there is no reason why you need to control them with arbitrary KPIs and metrics. Move from “control by objectives” to “providing the vision and transparency,” developers will start to self organize. When people have freedom to express their proper knowledge and motivation, and provide customers with something that benefits the customer, they’re happier. It’s better to focus on the customer's value. This brings up another problem: developers are often very disconnected from the result of our work. The morale and motivation can drop very easily if development teams are not talking to the customers, and are not sitting close to the customers. Making sure developers understand what the users are benefiting from the product they build will make developers happier, more productive, and have a higher sense of ownership.

If you’re a team lead, maybe you can run an experiment of having no KPIs and performance reviews. It’s not only much less stressful for developers, it also prevents developers from gaming the system and trying to match their performance to the expected result on the performance review you have provided. A much better way to approach management is to have the manager help with removing any impediments. By observing the team and conducting retrospectives, you can get people used to the inspect and adopt process. You can then move from retros to ad-hoc investigations.

This allows you to move into the Kaizen method, where there’s constant improvement, whenever you see a blocker, you stop and you improve. You can start with statistically significant blockers and prioritize the problems over features and products you’re releasing.