Measuring The Damage

This piece originally appeared in the IPA Review.

Before coming to the IPA, I had a small role in a Victorian Education Department team implementing Gonski school reforms. The department developed school performance targets and a reforms package that would, among other things, help schools identify their strengths and weaknesses. To what extent would the reforms move schools closer to the targets, the minister’s office asked? So we then had to somehow score the reforms against the targets, and estimate how much progress would be provided by funding.

The folly of such exercises is well-captured in Jerry Z. Muller’s The Tyranny of Metrics, released last year. Muller, an historian at the Catholic University of America, argues that metric fixation has overrun bureaucracies, public and private, distorting their behaviour and ultimately frustrating their purposes. Metric fixation replaces experience and discretion with institutional targets and measures, argues that all inputs and outputs should be reported (transparent), and connects rewards and penalties to performance against metrics. The result is that institutions pursue only their most obvious and measurable tasks, leading to the corruption of their internal information flows and ultimately to waste and inefficiency. Muller observes that “measurement may become counterproductive when it tries to measure the unmeasurable and quantify the unquantifiable”.

Not all metrics are useless. Muller notes accurate measurement against readily identifiable ends is desirable when possible. The trick is to distinguish between good and bad uses. To this end, he dedicates the bulk of the book to case studies taken from across society, demonstrating the problem is not limited to the public sphere. Businesses and charities have been just as charmed by made-up numbers and phony rigour as schools, universities, medicine, policing, and the military.

Continue reading