Managing Revision Risk

The ongoing potential for data revisions to create chaos in the best-laid plans of analysis is no trivial matter. It’s a perennial challenge and one that requires constant attention. But it needn’t be fatal in the essential task of reading the numbers for clues about where the economy’s headed. There are no complete solutions, but there are techniques to keep this hazard from turning an otherwise reasonably designed forecast into trash.

One approach I use begins by recognizing that there can be a lot more noise in the short term numbers compared with longer-term data. For instance, if you’re analyzing private-sector non-farm payrolls for insight on the business cycle, the potential for getting whipsawed by revisions for, say, 3-month percentage changes can be quite high. That doesn't mean we should ignore shorter-term comparisons, but it's a mistake to rely on them alone.

Accordingly, we can manage revision risk to a degree by focusing on year-over-year changes as well, a technique that also minimizes seasonal distortions. If you’re comparing the latest data point to its counterpart from a year ago, the earlier number is less likely to be revised after 12 months. Even if it is, history shows that the differences between initial estimates and revisions tend to be less volatile through time vs. looking at shorter-term comparisons.

As an illustration, consider private nonfarm payrolls on rolling 3-month percentage-change basis. Let’s compare those changes in terms of the initially reported data (via the ALFRED database at the St. Louis Fed) and the revised data (as reported by FRED).

As you can see in the chart above, the difference between the initial estimate and its revision can be rather dramatic at times. In early 2010, for instance, the initial data was wildly misleading relative to the revised data that was reported later. Such differences tend to less extreme and therefore less troublesome in year-over-year comparisons, as the second chart below illustrates. Yes, revisions can be a minimal problem for short-term comparisons at times, but you never know when a big revision is going to hit you over the head.

The higher level of revision risk in the shorter term applies to many data series, which is why it's important to look at year-over-year changes to keep this problem from spinning out of control. It's not a perfect solution, however. Revisions still bedevil annual comparisons. That inspires adding another layer of defense by combining a carefully selected mix of indicators in order to keep a lid on the noise from any one set of numbers.

A third treatment for managing revision risk is looking at the aggregate of annual changes for a broad set of indicators through the filter of a diffusion index. In other words, focus on the binary signal of the indicators in terms of the annual trend: is the indicator rising or falling? Combining these signals into a single diffusion index, which is the basis for the Capital Spectator Economic Trend Index, provides a valuable signal for assessing the business cycle overall and estimating recession risk in particular.

To be sure, perfection still eludes us. It always does in matters of macro analysis, which is why it's critical to evaluate the business cycle from several perspectives, with different methodologies and different indicators. In sum, developing a solid read on the business cycle, which can only be estimated, takes a fair amount of work.

That said, the various risks that cloud our capacity for looking ahead aren't absolute, at least not always. There are usually partial solutions to consider. That's not always clear when some analysts talk of a given risk in the interest of rationalizing their forecast du jour. Yes, revisions can be a problem—a big problem if you're clueless. Fortunately, there are ways to deal with revisions to keep them from turning an otherwise reasonable forecast into garbage.