Join Newsletter
Forgot
password?
Register
Trusted Business Advisors, Expert Technology Analysts

Taneja Blog

Taneja Blog

Do ROI Calculators Produce Real ROI?

As both a vendor product marketer and now an analyst, I've often been asked to help produce an "official" ROI (or the full TCO) calculator for some product. I used to love pulling out Excel and chaining together pages of cascading formulas.  But I'm getting older and wiser.  Now I see that ROI calculators are by and large just big rat holes. In fact I was asked again this week and, instead of quickly replying "yes, if you have enough money" and spinning out some rehashed spreadsheet (like some other IT analyst firms), I spent some time thinking about why the time and money spent producing detailed ROI calculators is usually a wasted investment, if not a wasted opportunity (to do better).

First, no one has ever really built a realistic ROI calculator - at least not one that has been "proven" with real world experimentation across controlled groups of actual living customers. Of course you can't really experiment - a given customer can't take both action a and opposing action b at the same time and see the difference. Even if one tries to run experiments between departments there are simply too many variables to control in real life. Companies are so different from each other when it comes to how "ROI" things are accounted for and measured internally that it's impossible to really baseline across industries (much less establish a control group). And if two companies valued similar things, had similar cultures, and even accounted for both hard and soft benefits in the same way - their measurements still wouldn't line up.  And besides all that, people will always be gaming whatever rules get set down (just ask any financial accounting auditor!)- working towards the KPI metric versus actually creating the most value.

The first good step toward ROI documentation would be to interview or survey lots of customers to establish some baselines, understand the variances and distributions, and create some correlations. But there is no point in trying to convince smart people that some two or three cherry picked anecdotes define an ROI equation that represents reality for all. And by the way, applying arbitrary risk-adjusting discount factors to cherry picked evidence isn't any more believable (it's just smoke and hocus-pocus - "uh, seems complex so it must be real"). Your real baseline data or surveys here need to be large, or even better, based on that big data you've collected from trusted call home and supply chain sources. (However I will come back around to talking about how 3rd party validated use cases do provide real and valued information for your prospects...)

I shy away from promising clients a full, realistic ROI model assuming that they mean for a reasonable fee (the sky is the limit if you are willing to pay unreasonably). The respectable ROI modeler has to understand all the relevant factors qualitatively, believably quantify them (which is hard work) and show differentiating comparison to the current baseline/competition/alternatives. To do this defensibly requires broad market surveys, lots of field interviews, domain knowledge, deep competitive analysis, and even then some good gut-driven guess work.  And all that assumes that the vendor even wants an accurate model - most vendors I know really want to, well, exaggerate a bit when it comes to making ROI arguments.

And if all that is accomplished, and a working ROI calculator emerges that seems to pass the first round of "believability" sniff testing, most prospects will still only "look" at an ROI spreadsheet as a checkbox thing late in the purchasing cycle (when some finance guy asks for it). As marketing content, it isn't a good time or place to introduce or make your convincing value propositions shine.

Besides, we know from first-hand experience that real IT decision-makers don't believe in any vendor's ROI model at face value (good for them), and most finance folks only consider hard CAPEX factors (which eliminates most of your good stuff including predicted OPEX savings, estimated risk reduction, associated staff leverage and productivity gains). So even if you make a detailed researched tool, the ROI calculator is still a thankless, checkbox piece of content. Bottom line - You still may have to have one, but don't put a lot of $$ into it.

Here are some practical ROI content suggestions for all you product marketers -

  1. As a checkbox item you might as well produce a simple ROI calculator internally - first use Excel, and build it so the prospect can easily change every "assumption" even as you provide smart default values based on validations/surveys/etc.  I'd suggest presenting a simpler ROI calculation, not a comprehensive one - no more than 2-3 input factors per key messaging value.  If you haven't nailed down what makes you valuable, then back up and start with that task... 
  2. Then if you are fancy, have your smart web person make a browser based version for your web site (ask for interactive data-driven visualization using something based on d3.js). If you are all very clever, you can use it to sample what people might find most valuable about your solution. If you have lots of existing customers and relevant call home data, you can even present actual ROI experience to-date as a comparison/baseline or as default settings.
  3. Spend your limited coin on analyst services that directly document and validate your biggest value propositions. I'd much rather help spend your money to produce content that drills down into, or validates with lab or field testing, unique aspects of your ROI thesis than produce a barely read hunk of unbelievable drivel that of course will show 10,000% return. Besides, 3rd party validated reports can be used much earlier in the sales cycle, and usually produce key "facts" that can then be used in your own ROI spreadsheet downstream.
  4. Work hard (yourself) to quantify, believably, the often larger soft benefits and potential upside "returns".  For example, what is the value of not having a hacker steal all the data and bankrupt a company in any given year? What is the benefit of making the right IT resource investment (versus buying poor/bad/misaligned resources) at the right time? What is the cost savings for not having to hire extra customer support to explain poor performance? Do use actual real-world statistics (like for failure rates, floods, etc) but first talk to someone math-smart about the proper use of probabilities and expected likelihood/outcomes.
  5. You win bonus points (and clients will respect you) if your ROI model enables quick comparison to alternatives like doing nothing or adopting the competitor (and not just variations based on how much of your solution is purchased this quarter). I personally won't believe an ROI tool that doesn't allow for a negative return under some circumstance of allowed inputs. Where are the breakeven points?  Where does the expected return become truly compelling (annotate any nonlinear inflection points on your ROI curves!)?

As a professional curiosity, let me know what you decide to pursue for proving ROI (and how it actually turns out).

Bookmark and Share
  • Premiered: 08/24/16
  • Author: Mike Matchett
Topic(s): ROI TCO

Comments

There are no comments to display. Scroll down to leave your own!

 

Leave a Comment

You must be logged in to comment. Click here to log in or register if you don't have an account.