Before we created the Autocase family of TBL-CBA software, our team of economists, along with engineers and architects conducted numerous custom assessments as consultants. Our collective knowledge is now reflected in Autocase, but, we have to admit, there were some moments along the way when we were caught off-guard and ran into some tricky traps: practices, policies and thinking that might have derailed us. We hope the following 10 will guide those interested in TBL-CBA but we also suspect that they will resonate with our fellow practitioners.
Over the last few weeks we have posted the top 10 traps when running TBL-CBA. We’ve collected these below in one post. Click here if you would like an infographic of the Top Ten TBL-CBA Traps.
1. Re-Inventing the Wheel
Cost-benefit analysis is a time-tested methodology. It was invented over a century ago, was standardized after World War II, and has been refined since. So, there’s no excuse not to use, as a starting point, existing national government guidance on methodologies and data (the US, Canadian, UK, Australia, and European Commission’s documents are particularly robust). These guidelines will also assist your efforts on how to be “MECE” – Mutually Exclusive and Comprehensively Exhaustive (see Traps #2 and #3).
2. Forgetting Certain (Increasingly Vocal) Stakeholders
When conducting a Triple Bottom Line – Cost Benefit Analysis (TBL-CBA), a common trap is thinking too narrowly about the project. By definition, a triple bottom line approach avoids thinking only in financial terms – the narrowest thinking of all. Assigning a dollar value to environmental or social metrics that lack a simple market price is the big step that helps to broaden the view.
Broadening the view, however, does not mean losing the perspective of each project stakeholder. Infrastructure and building projects have always been complex to manage, but now project managers are being asked to manage an even wider scope of concerns because project footprints have grown and project stakeholders have become more sensitive and sophisticated about the direct and spill-over effects for them. This is where Multiple Account CBA comes to the rescue – it breaks out negative and positive impacts by stakeholder group, from neighbors to local governments to those downstream. It goes a long way towards countering NIMBYism and avoiding project delays.
3. Double Counting
By avoiding the trap of being too narrow, people often fall into this trap. In particular, benefits are more likely to be double counted than costs. Take, for example an infrastructure or building project that lowers the cost of a product for consumers. There will also be a rise in the profits of producers and other middle men involved in delivering the product to consumers because it costs less to deliver. The cost savings is passed on from producer to consumer, so it is one and the same. Similarly, if, out of the net benefit of a project, tax is paid to the government, this should be counted as a transfer not as a loss.
Another frequently double-counted benefit is “property price increase” resulting from a project. Many cost-benefit analyses of property price increases from better transit access, for example, will often erroneously include both the capitalized future transportation benefits and the non-transportation use benefits of liveability.
So, be inclusive and avoid thinking narrowly about stakeholders, but not so inclusive that you double count. By side-stepping both traps, you’ll have lived up to one of the founding principles of cost-benefit analysis: be “MECE” – Mutually Exclusive and Comprehensively Exhaustive. (See trap # 1)
4. Letting the Loudest Voice Win
While the TBL-CBA analysis often points out trade-offs between people, planet and profits and highlights distributional effects across affected stakeholder groups, don’t fall into the trap of implicitly weighting one group or sector more than another. Cost-benefit analysis does the weighting for you. For example, a quantity of water used or saved, when multiplied by the social cost of water (which accounts for regional scarcity) will show up as more material if there is a lot of water used. Likewise, if a project uses minimal energy, even when multiplied by the social cost of carbon emissions, energy likely won’t be a material factor.
One of the most compelling attributes of TBL-CBA is its objectivity.
5. Asking “Are We Doing the Right Project?” but Forgetting “Are We Doing the Project Right?”
Once the project has been greenlighted, a million and one design decisions remain to be made by engineers, architects, sustainability professionals, planners, project manager, and designers that determine what exactly gets built and how. TBL-CBA should be used for these small decisions, not just the make or break choices, to keep the evolving design in line with the investment thesis. TBL-CBA should be used early and often throughout the planning, design, procurement, and construction phases as the project changes. For example. after a decision has been made to build a bus rapid transit line – where it will stop, what the stations will look like, and how it will be integrated with other transportation networks – TBL-CBA can be used to decide whether to collect water at the bus depot with a green roof or a cistern, and whether it makes economic sense to reuse the water for washing the buses.
6. Assuming it is Difficult and Expensive
With standardized inputs, methodology and outputs, cost-effective automation of TBL-CBA is possible. While there will be people who will feel better paying $50,000-250,000 for a bespoke TBL-CBA study, don’t let anyone talk you into a custom study until they have told you what they will be adding that hasn’t already been done once before.
7. Tolerating False Precision
The results are in – the Net Present Value is $101.234567 million (exactly) – let’s go! But wait…what if the risk associated with that number is +/- $200 million? How do you feel now? Is the project still a go? Robust TBL-CBA methodologies employ risk analysis so that all inputs are described by a probability curve derived from the low-expected-high values across the economics literature for a given input. Doing so means that you can offer confidence intervals (think, levels of certainty) attached to all the results. So, everything, including uncertainty, is on the table for the client.
8. Muddying the Waters between Impacts and Benefits
Impact analysis captures the overall output and income resulting from a project i.e. “400 local jobs created” or “an increase of $3M in GDP”. But this is not cost-benefit analysis. Impact analysis is often a marketing exercise intended to curry favor with those whose stamp of approval is needed for the project to move ahead. One particularly egregious example is job creation. Jobs created as part of a project are costs to the project, not benefits in a societal cost-benefit framework. To count as a societal benefit, these workers would have to be paid more than they would otherwise or be unable to be employed elsewhere were the project not to go ahead. In this case, there is no net benefit unless there is a labor shortage so a project attracts new entrants to the profession. When trades are induced to work on a project because of higher wages, it is only this incremental wage that is a benefit. If a project hires from a pool of unemployed workers, their wages over what they would have received from any unemployment insurance scheme could be considered a benefit. But in general, the job itself is not a benefit.
Impact analysis = marketing hype. Cost-benefit analysis = sound economics.
9. Ignoring the Vast Collection of Ecosystem Services Research
A lot of smart people have spent a lot of time over the last two decades putting values on ecosystem goods and services (e.g. clean air, fresh water, habitat protection). There has been an explosion of research and growing consensus and convergence in these values.
Graph Source: Cumulative total of ecosystem services valuation studies sourced from EVRI from 1960 to 2008. Rudolf de Groot, Luke Brander, Sander van der Ploeg, Robert Costanza, Florence Bernard, Leon Braat, Mike Christie, Neville Crossman, Andrea Ghermandi, Lars Hein, Salman Hussain, Pushpam Kumar, Alistair McVittie, Rosimeiry Portela, Luis C. Rodriguez, Patrick ten Brink, Pieter van Beukering, Global estimates of the value of ecosystems and their services in monetary units, Ecosystem Services, Volume 1, Issue 1, July 2012, Pages 50-61, ISSN 2212-0416, https://www.sciencedirect.com/science/article/pii/S2212041612000101?via%3Dihub. (//www.sciencedirect.com/science/article/pii/S2212041612000101) Source: modified from M. Christie, I. Fazey, R. Cooper, T. Hyde, A. Deri, L. Hughes, G. Bush, L. Brander, A. Nahman, W. de Lange, B. Reyers “An Evaluation of Economic and Non-economic Techniques for Assessing the Importance of Biodiversity to People in Developing Countries” Defra, London (2008)
Embarking on original research is always a good thing but it can be an unnecessary expense. Far better, we think, to stand on the shoulders of giants, learn from their research, scour all the research, but be selective and harness only the best and most relevant results for your purpose. Even this is very laborious if you do it correctly and do it frequently, but it’s certainly more attainable than sending your staff out to the field for years at a time.
10. Mixing Nominal and Real
All the costs, benefits and discount rate should be either in nominal (current year) dollars or real (constant) dollars. Don’t mix real dollars and nominal discount rates or vice versa. Most economists recommend being real and staying real (man). That said, the preference for real/constant dollars is more convention than recommendation. In financial analysis, when cash flow is the primary concern, costs and benefits are usually estimated in nominal dollars with forecasts of specific price indices used for the inputs. Cost benefit analysis, on the other hand, usually shies away from forecasting individual prices and assumes that they all increase at the general rate of inflation. So, while CBA uses real and financial analysis nominal, make sure your discount rate matches your preference.
These are our favorites. Which ones resonated the most? What did we miss?
Click here if you would like an infographic of the Top Ten TBL-CBA Traps.