Common presumed best practices for determining return on investment (ROI), advocated by almost all apparent authorities, in reality often undermine ROI’s very purposes.
ROI is supposed to provide a valid and reliably supportable objective basis for making decisions: the quantified dollar benefits of an approach versus its quantified dollar costs. However, the customarily recommended practice of listing but not quantifying the dollar value of “intangible” benefits leaves a gaping loophole that can render even seemingly conscientious ROI analysis a deceptive sham.
The dirty little secret of ROI is that voluminous documentation and calculations often are merely a smokescreen obscuring the fact that in most organizations the proponent’s favored outcome almost always prevails. Although perhaps not consciously recognized, calling the exercise “justification” indicates a mindset where there’s not even a pretense of objectivity, since measurements are chosen to justify the proponent’s predefined answer.
Even when the analysis seems impartial, failing to quantify intangibles in reality often causes the game to be fixed, albeit without conscious intent or awareness. That is, if the ROI calculations support the preferred outcome, then the calculations are cited as the basis for deciding in favor of it. And if the objective calculations don’t support the desired answer, the unquantified intangibles provide subjective justification rationale for disregarding the unfavorable ROI calculations.
Because the intangibles have not been quantified, there often is virtually no limit on how much the proponent can claim that the intangible benefits override the quantified ROI calculations, no matter how bad a deal the tangibles calculations portray.
Aberrations in the Absence of Wiggle Room
Sometimes proponents go beyond even the broad leeway provided by unquantified intangibles. There’s an old joke that lawyers tell that ends with the punchline, “That’s easy for you—you’ve got evidence.” Lawyer wisdom goes on to say that if you don’t have evidence, plead the law; and if you don’t have the law, well then you’re into primetime TV lawyer show plotlines…
While working with a major international consulting firm, I had analyzed some options for a client and wrote explicitly in our report, “Do not do X.” Because our client wanted to do X, we assiduously had quantified the immense downside dollar risk of doing X and did not offer the merest hint of any possible intangible reasons to the contrary. Imagine my surprise when I heard our client present our report to the company Board with the words, “In accordance with our consultant’s recommendations, we will do X.”
Identifying REAL Value
Ultimately, this and several other ROI concerns stem from a commonly experienced difficulty accurately identifying the REAL value the ROI calculations should be addressing.
Salespeople frequently encounter the issue, which becomes evident when prospects are not persuaded by their product’s ROI claims. “Your price is too high,” means the customer doesn’t see the value. Of course, salespeople’s general reputation for exaggeration invites some skepticism. More often than recognized, though, sellers are failing to connect with prospective buyers on a very fundamental level—the benefits the seller emphasizes may not be what the customer values most, and may not value at all.
For instance, we were asked for assistance by a sales organization that couldn’t figure out how to demonstrate their product’s ROI. Many customers require vendors to show ROI, but as so often happens, this vendor’s customers didn’t accept their ROI claims. They already had identified more than 75 benefits of their product and seemed to be adding to their list daily, but they were stumped about how to turn all those benefits into ROI in a way that customers found convincing. Not only would 75 claimed benefits overwhelm anyone, but it’s entirely possible the vendor still was missing the boat.
The issue is by no means limited to traditional vendors and arm’s length sales situations. Whether one realizes it or not, demands for ROI evidence almost always occur when someone is trying to “sell” some proposed solution. The selling aspect is easier to see when the seller works for an external vendor’s sales department and is selling an identifiable product or service with a defined price. But just as much selling can go on when all the players are internal, backing for an idea is being sought, and money is not explicitly changing hands.
Product Focus Pitfall
For external vendors, some of the problem is easy to trace to typical traditional sales techniques and training. Many sales people view their role pretty much as, “I’ve got a whole lot of these products, why don’t you buy some…” Very often, sales training mainly concentrates on their product’s features and ways to convey those features attractively to their prospects.
There’s a presumption that the prospect will want the features and buy the product. Why then do so many prospects persist in not buying products with wonderful features? Maybe the salesperson didn’t describe the features in a sufficiently attractive manner. So the salesperson goes back to sales training to learn more about the product features and more magic words for describing the product’s features in ways customers can’t resist, except frequently the prospects generally continue to resist buying.
Perhaps regardless of its apparent attractiveness, the product doesn’t provide the prospective customer any perceived value. If so, the reason would seem to be either that the customer has no need for such a product or that the salesperson has not adequately communicated how the product meets the prospect’s needs.
Consequently, some enlightened sales training suggests focusing first on finding out what the customer wants and then dazzling them with the product’s features and ROI. While an improvement over selling features in a vacuum, this approach usually is subject to a significant seldom-recognized trap that still prevents prospects from seeing enough value to buy the product.
You see, asking the seemingly reasonable “What do you want?” actually invites presumed solution product design answers. Salespeople have a way of unintentionally and unconsciously steering such product descriptions toward designs that match the products they are selling, often using the briefest customer reference to some familiar-sounding feature to shut off the customer and launch back into their product’s features spiel.
Again, though, it’s not just vendor salespeople who fall prey to this pitfall, and it doesn’t only afflict Information Technology (IT) solutions. Business analysts, marketing specialists, project managers, engineers, developers of all sorts, and solution proponents in general repeatedly land in the same trap and seldom recognize it. They’re all usually focusing on design solutions. It’s only that, unlike external vendors, internal folks may not have so predefined a product to make the customer’s design fit to. But very often they’ve got just as much of an idea of the answer they want and will squeeze everything to conform to it.
REAL Business Requirements Provide Value
Although such customer statements commonly are called “customer/user requirements,” in fact, they actually represent requirements of some product, system, or software that is presumably one of the possible ways how to satisfy the presumed REAL business/user requirements whats that provide value when met.
The product how provides value if and only if it actually satisfies the REAL business requirements whats. The more attention that is paid to the product solution hows, the less likely it is that the REAL business requirements whats are identified adequately.
ROI that is meaningful to the customer has to be grounded in providing what the customer values, which is the customer’s REAL business requirements. Customers do not require your product/system/idea solution. The solution has value to the customer if and only if it satisfies the customer’s REAL business requirements.
REAL business requirements exist within the business environment and therefore must be discovered. While many well-known requirements techniques are applicable, they need to be positioned within a markedly different mindset. The starting point is simply being aware of the difference between REAL business requirements whats and product/system/software requirements hows.
One needs to learn how to discover, as opposed to the specifying and product feature-fitting that tend to be more familiar to many people. A focus on value is critical, as is genuinely listening and hearing from the customer perspective, rather than the lip service that (despite denials and protestations to the contrary) many sales and marketing people, developers, and analysts often are perceived as more customarily paying to customers.
The Problem Pyramid™ is a difficult but powerful tool for reliably identifying REAL business requirements and the value obtained by satisfying them. (For more on discovery methods and using the Problem Pyramid™, see my Artech House book, Discovering REAL Business Requirements for Software Project Success.)
The Problem Pyramid™ defines a disciplined systematic set of six steps (shown by numbered boxes) that must be followed in sequence, starting with (1) defining the REAL problem, opportunity, or challenge that when solved, taken, or addressed will provide REAL value. Surprisingly many projects are destined for failure because the REAL problem is not defined correctly.
Two sets of measures are critical for defining the REAL problem. (2) Current measures of how things are now are actually what tell us that we’ve got a problem. (3) Goal measures, when accomplished, tell us the problem has been solved, the opportunity has been taken, or the challenge has been met.
Part of the Problem Pyramid™ discipline involves guidelines for making sure that each step along the way is correct. These guidelines pay special attention to the measures. If one cannot identify measures, or if the measures do not fit the problem, either the problem or the measures (or both) are incorrect. The difference between the goal and current measures is the benefit, or value, to be obtained. If accomplishing the goal measures doesn’t provide REAL value, the measures and/or problem are not defined correctly.
There is no point in proceeding until the problem and its measures are defined correctly. That’s where the value comes from. Many ROI efforts fail from the start by skipping these vital three steps or getting them wrong.
One doesn’t solve a problem directly. Rather, (4) one identifies the as-is process causes of the problem that are producing the current measures. Then (5) one defines the business deliverable whats should-be process, which when accomplished will address the causes and reasonably lead to achieving the goal measures, thereby providing the value. These are the REAL business requirements that customers value and are essential to meaningfully determining ROI.
Notice that there’s no point in specifying (6) how to accomplish the (5) whats until the whats have been discovered accurately and completely. Most projects and ROI fail because people start with the (6) product, system, or approach how that they want to sell, create, or adopt.
ROI Value Modeling™
ROI analysis involves comparing two or more (6) how alternatives. Ordinarily, one of those alternatives is business as usual (BAU), which often is called “no change.” An ROI Value Statement™ describes the rationale explaining how the (6) alternative’s hows satisfy the (5) REAL business requirements whats, thereby reasonably achieving (3) goal measures to provide value. An ROI Value Map™ calculates ROI by tying the costs of implementing the (6) how alternative to the (3) value provided.
So what’s all this REAL business requirements stuff have to do with our original points about quantifying intangibles? The obvious answer is that the Problem Pyramid™ makes explicit and demands quantifying in dollars the (2) current and (3) goal measures that define value.
While seemingly intangible benefits such as improved customer satisfaction or better management information can be measured directly, for example with surveys, the Problem Pyramid™ discipline also demands objectively measuring their impacts on revenues and expenses. While a proposed solution indeed could have quantifiable intangible benefits, amazingly often quantifying presumed intangible benefits may reveal they don’t actually have significant dollar value. For example, commonly-cited intangibles include more satisfied customers or better management information. Yet, in fact, many satisfied customers never buy again and never tell anyone else. It’s nice they are satisfied, but it may not have REAL dollar value. Similarly, too often better management information is either not useful or not used. The Problem Pyramid™ discipline can spot and help prevent undertaking projects that seem like good ideas but whose claimed intangible benefits really won’t pay off.
It’s not just the measures where the Problem Pyramid™ and ROI Value Modeling™ quantify intangibles. Very often, (5) REAL business requirements include things ordinarily considered intangible. However, because REAL business requirements are business deliverable whats, they must be defined as measurably observable deliverables.
Most ROI analyses treat risk, flexibility, and opportunity as intangibles. These need to be quantified too, sometimes as (5) REAL business requirements deliverable whats and perhaps more often as (6) how alternatives. For instance, greater risk may be reflected by adding skills, resources, and time to an alternative’s implementation.
ROI that relates explicitly to the REAL business requirements customers value most and leaves no unquantified loopholes provides a reliable basis for persuasively guiding meaningful decision-making.
Robin Goldsmith will be presenting a free webinar with ITMPI on October 4! Sign up here: You Don’t Need No Stinking Test Cases?