Measures which matter

About one month ago I was following a couple of colleagues discussing on Twitter about measures and specifically about measuring productivity. It was a nice exchange of views on whether development of a new product is production or creativity and what to measure in either case.

I personally think that SW development is most about learning: make an assumption, do the cheapest experiment to validate that assumption and evaluate the results by means of customer feedback. That’s why Agile makes sense.

Now I do not want to come into specifics (I do not know the background of the discussion), but I was thinking: I cannot give THE right answer to the question, but I would put on the table some basic principles about how a good metric should look like.

1. Measure one level up – Whatever you want to measure you might better look at the next level: look at teams if you want to measure individuals, look at the development organization if you want to measure teams, look at the e2e flow if you want to measure the development organization
2. Few, Simple and Visible – Full transparency makes the chosen metric powerful. Don’t over measure.
3. Drive right behaviors – Don’t measure outcomes, but put functional behaviors in place and the outcomes will take care of themselves. You get what you measure.

Some time ago we had an issue about high pressure on teams to deliver (actually from both inside and outside), so that many people (and many influencers in the organization) did not consider important improving as individuals and teams. So management decided to measure teams’ improvement ability.

I’m scared already only by thinking at possible solutions a traditional and bureaucratic organization could have thought about in the attempt of achieving this goal.

We simply tried to apply the 3 principles above and setup a metric called:

Organizational improvement factor = % Total Time dedicated to Improvement actions vs. Total available time. Here is the rationale behind:

1. Measure one level up – We wanted to measure teams, so we decided to measure the development organization: it is not finger pointing to any individual or team, is collaborative and everybody feels accountable for the overall value.
2. Few, Simple and Visible – A unique metric which is simple, updated at each sprint and fully visible on a white board in a common space.
3. Drive right behaviors – Behaviors are a function of people and their environment: this metric resulted to be a good way of influencing the environment and then the teams’ behavior. Everybody feels now that improvements have a recognized dignity.

The same applies when talking about measuring stuff like development productivity or efficiency.

1. What is the next level up which makes sense to measure?
2. How can we make it simple and transparent?
3. What good behaviors should we look for?

In that case I would rather measure organizational effectiveness:

Are we doing the right things and only what is strictly needed? 

How is our Return on Investment?

You will realize that you need to consider delivered value (not only costs) and discover that effectiveness eats efficiency at breakfast every morning: if you drop 50% of the scope because it is not needed,  manage to deliver faster and get money earlier, there will be no cost efficiency program or productivity improvement whatsoever that will give you comparable results.

However, being a team or an organization a complex system of human beings, I would like to leave you with a quote from master W.E. Deming:

97% of what matters in an organization can’t be measured. Only maybe 3% can be measured. But when you go into most organizations and look at what people are doing, they’re spending all their time focusing on what they can measure and none of their time on what really matters – what they can’t measure. Why would we do this? We’re spending all our time measuring what doesn’t matter.