Gathering metrics is an essential part of the CIO’s business. Without a consistent and accurate stream of data to analyze and act on, IT can rapidly degrade into the black hole of cost overrun, missed deadlines, and unforeseen issues. But Stephen Mann knows that not all metrics are created equal. In order to truly gain the advantage, metrics have to be timely, accurate, and important for the business. His list of issues includes:
- Measuring what’s easy, not what’s important
- Focusing only on IT metrics (and not business)
- No structure for or context between metrics
- One dimensional view
- Metric hierarchy isn’t clear
- Too much emphasis on I&O benchmarks
- Reporting is poorly designed/delivered
- Overlooking behavioral aspects of metrics
- No evaluation of metrics and their need over time
- Metrics and performance misunderstood.
In the infrastructure and operations world, the metrics must, for example, be the most valuable possible – not the ones that are the easiest to get: I&O organizations shouldn’t spend more time collecting and reporting metrics than the value we get from them, but that still isn’t an excuse to just measure the easy stuff. The availability of system reports and metrics again comes into play, with little or no effort needed to suck performance-related information out of the ITSM tool or tools. Consider why you report each and every metric in your current reporting pack and assess the value they provide versus the effort required to report them. Not only will you find metrics that you report on just because you can (“they were there already”), you will also find metrics that are “expensive” to provide but provide little or no value (“they seemed like a good idea at the time”). I&O must also be careful to collect data that applies to business objectives — not IT activity performance. Chances are your CEO doesn’t care what internal marks you’ve achieved – but they will care about what business strategy you assisted, what strategic SLA you reached, and how your work is positively impacting the entire organization.