Do you want to be approximately right or precisely wrong?

Credit Goes To Writer:

Mark Pettler

More variables or detail = more assumptions = more scope for error

In order to develop an accurate commercial model it is assumed that going into more detail or including more variables will result in much higher accuracy. However, more detail or more variables results in more assumptions. Each assumption will have a degree of uncertainty and as the number of assumptions grow so does the potential error of the model or calculation. By considering the worst and best case for your assumptions or a range in the case of a variable then the model can be developed to give a range of likely outcomes.

Reconciling bottom up to top down

As the size and complexity of your analytical model grows so does the scope for calculation errors in addition to errors from incorrect assumptions. By comparing your detailed model against a simpler top down model with a small number of broad assumptions you can sense check the results and start questioning some of the detailed assumptions or calculations. For example, when I led a bid for major advertising contract, our sales model showed a growth which was not consistent with the likely growth of the overall advertising market spend and realistic increase in market share. It turned out there were some calculation errors and some very optimistic assumptions in the detailed bottom up sales model.

If it sounds too good to be true then it probably isn’t.

If an analytical model shows surprising good improvements then you have to ask; “What are we planning to do so differently to deliver this improvement?” Unless you’re planning to do something spectacular then you need to check the validity of individual assumptions. When you start building more and more detail in business models then what seem like modest improvement assumptions of even a fraction of a percent can accumulate to double figure totals.

a and b and c and d does not equal a + b + c + d

When a model contains multiple initiatives, benefits are often added and assumed to be all fully incremental. In reality this is often not the case for a number of reasons:

  • Interactions and cannibalisation. For example introducing a new product might steal some sales from existing products or a regional sales initiative might divert spend on national accounts.
  • Multiple initiatives can compete for the same resource resulting in partial success. Most successful businesses focus on a small number of strategic initiatives which increases confidence of successful delivery of all projects.
  • The total opportunity is finite. For example multiple sales initiatives may add up to a sales increase that is unachievable because the overall market or individual customers just don’t have the increased spend available. I remember in my Mars days, I was asked to approve a number of projects to reduce waste on drinks production lines. When I added the total projected waste reductions from all the proposed projects they were double the current waste levels.
  • In conclusion, I think there is a fine balancing act between developing:
    • A model that has sufficient detail to be credible and insightful for business decision making
    • A model that is simple to understand and has a small number of validated assumptions

    There is ultimately a trade-off between robustness and precision. Perhaps the best approach is to build two models; detailed and simple and consider both when making important business decisions.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s