Black Boxes. And that is where the term “black box” comes into play. In the planning context, a black box process is one in which there is an information or communications gap, leaving agency staff unable to explain the logic and sources of analysis that they used. This puts them at risk of being unable to justify their plans and decisions to government leaders, legislators or the general public. The tools or model that they cannot explain is typically labeled as a black box. But in reality, almost any party may be to blame. For instance, it may be that the model was deficient in layout, labels or documentation. Or it may be that the vendor or consultant provided insufficient training for users. Or agency staff may have been too busy to read documentation or receive training. Or it may be that the agency staff understood the tools or models, but were not equipped to explain it well to other interested parties. Whatever the reason or excuse, the bottom line is that their fears may be justified.
Are spreadsheets the answer? Well, one response is to say, “what we really need is a simple spreadsheet that all can understand, instead of relying on some highfalutin, fancy model.” However, creating homegrown spreadsheets does not necessarily guarantee any greater clarity of use; most people have seen examples of poorly labeled spreadsheet workbooks with convoluted layouts that are neither clear nor simple. And spreadsheets, like any software tool, can be over-simplified or overly complicated. So it is not the medium, rather it is what the tool accomplishes and how it can be explained, that ultimately matters.
Economic Impacts. Now let’s understand how infrastructure investments or policies affect the economy. Using the transport example, the story to tell has a sequence of steps:
- A project leads to some resulting transport changes (e.g., travel times, costs or access).
- Direct Effect – There are changes in costs & access for households & businesses, shifting spending & sales patterns that directly affect the growth of some economic sectors.
- Wider Effect – The direct effects on business growth also affects supplier orders and worker spending, leading to broader impacts on other sectors of the economy.
- Time Effect – All of these effects continue over time but shift as the economy evolves.
Step 1 can be estimated by use of travel models or engineering estimates. The other steps can be estimated through economic impact models such as TREDIS and REMI, both of which contain cost and access elasticities (coefficients) for step 2, input-output tables for step 3 and dynamic forecasting for step 4.
Wrong-Headed Shortcuts. But there is a cheap shortcut approach. One can save a lot of time and money by simply using an input-output model to calculate direct and wider economic effects. And one can search the internet to find examples where that has been done to calculate economic impacts of changes in both transportation costs and energy costs. Some have characterized this approach as simple and transparent, noting that it avoids more sophisticated models that they call “black boxes.” So what is wrong with that?
Well, first, it requires a major leap of faith. You see, input-output models such as IMPLAN only address step 3. That makes them ideal for analyzing the wider effects of a business openings, expansions or closings. But they do not address step 2, which requires a separate analysis of how businesses respond when faced with cost and access changes. The way out of this problem, commonly used to enable the simple approach, is to just “assume” that when business costs fall for any given industry, that its output (sales) will grow proportionately.
Unfortunately, that is a bogus assumption. To see why, consider two situations. The first is a project that halves the cost of producing toothpaste. How much more would people brush their teeth and buy toothpaste if its cost was halved. Would toothpaste sales really double? Now consider what might occur if the cost of the latest widescreen TV or tablet computer dropped in half. Might sales more than double? The bottom line is that economists know some products and services have highly elastic demand (very sensitive to costs) while others have just the opposite characteristic.
And that is why the folks who provide IMPLAN, the nation’s most widely used and respected input-output model, appropriately instruct users that their system cannot model transportation cost changes unless the user has some outside way of determining how costs lead to direct changes in business output (sales). Those using the shortcut approach are thus ignoring warnings about proper analysis methods.
And worst of all, reports using the shortcut approach have a common feature, which is that they avoid explaining the logic of how they got from cost savings to economic impact. And that makes this type of analysis the ultimate “black box.”
Towards Clarity of Transparency. There is a widespread need for economic impact analyses to be presented in a clearer manner, in which the logical sequence of outcomes and results is explained as a logical story. Tools such as TREDIS and REMI actually do provide information on intermediate results that can be used to tell a logical story about how economic impacts occur. But it is still not always easy for users to know how to communicate this information to interested parties. The new TREDIS 4.0 is introducing reports for “impact tracing” and identification of “key drivers of impact,” to make it easier to tell the impact story. But if you want to really succeed at demystifying the black box and presenting a logical case for your decisions and results, you must first consider what’s really important to your audience, and then use words & concepts that resonate. We’ll have a blog on that in the future.