There is something wrong with the monthly Payroll Employment Report.
Economists were very surprised by the massive jump in jobs reported in May. They quickly changed their viewpoints about the economy as did important policymakers.
The reasons are simple:
- Employment is the most important economic indicator – the foundation for income, spending, and government revenue.
- The payroll employment report is viewed as the best and most important employment indicator.
- Those analyzing and reporting the data have earned a reputation for honesty and professionalism.
This indicator is so important that we must get it right – or at least be aware that something might be wrong. In this analysis I will explain an arcane element of the methodology that I believe to be the culprit. The result may well be a significant overstatement of the job gains.
Because the reasons are technical, it is a challenge to explain. I have decided to emphasize descriptive material and charts, but everything is thoroughly sourced. This is what I have included in the analysis:
- Background on the overall U.S. employment structure and dynamics.
- The BLS process for the monthly report, including strengths and weaknesses.
- A description of the problem.
- The effect of revisions.
- What to watch for in upcoming NFP reports.
The U.S. Employment Market
Let us start with some key statistics about the employment market. The table below will help keep our analysis in the right context. The total number of jobs at the end of 2019 was nearly 152 million. Private employment was over 129 million. Contrast this with the net change in jobs over the quarter – less than 600K, or about 200K per month with a sampling error of more than +/- 100K. If a report is 50K jobs away from expectations, it is viewed as a big event. We are attempting to measure something that is very small when compared to the overall size of the job universe.
Now turn to the number of jobs created and lost. Most people, even experts, do not realize that job creation was nearly 8 million over the quarter. The problem is that job destruction was 7 million. The number in the headlines is always the net change in total jobs, concealing the turmoil underneath this bit of surf.
Sources: Q4 2019 Business Employment Dynamics and CES reports
Just to emphasize this size comparison, here it is in graphic form.
The most interesting elements are the sources of job creation and destruction, so let us turn to those.
The table shows that expansion of current businesses was the larger contributor to new jobs while new business formation was less important. Here is the graphical comparison of net effects over Q4 2019.
Observant readers will note that the total of the sources does not match the overall change. The BLS has many different reports, each from a specific source. While there is a benchmarking process to link them, it takes time. There is also a difference in basic definitions.
The data I have emphasized so far comes from the Business Employment Dynamics series, drawn from the Quarterly Census of Employment and Wages. It is the closest thing we have to an actual count since it is taken from state employment office data. Q4 2019 is the most recent report, released last week.
The BLS Process for the Payroll Employment Report
The reported payroll employment number is not an actual count of changes nor is it a measure of “new” or “lost” jobs. It is a calculation of the net change in monthly estimates of the total number of jobs. The BLS estimates the total number of payroll jobs, and they do it every month. May’s estimate of 132,912,000 jobs was compared to April’s 130,403,000. The numbers include millions of jobs gained and lost each month, all part of the much smaller net change.
The Estimation Process
The Census Bureau, cooperating with the Bureau of Labor Statistics conducts a survey of a carefully developed sample of business establishments. It includes about 145,000 businesses representing nearly 700,000 different worksites. The sample is updated annually, and a supplemental sample of new businesses is selected midway through the year.
Among other questions, they ask for the total number of jobs in the pay period including the 12th of the month. It does not matter whether someone is actively working. In contrast to the Household survey, used to determine the unemployment rate, it is a count of jobs, not people.
Some businesses do not immediately respond to the survey. The BLS uses great methods to improve the response rate which has increased dramatically over time. In the past three years 70% or more respond in time for the first report. About 92% have replied by a month later, the first revision. The third and final version includes response rates of nearly 94%.
The Problem
What about the missing 6%? Are these just businesses whose managers who did not respond on time? Or might they be companies that are out of business? The difference is crucial to the resulting job count. The BLS makes the assumption that the non-respondents are like those who did respond. I have described this as the imputation assumption, since it imputes characteristics from a known group to the unknown group. The BLS assumption is that newly formed businesses replace dying businesses that did not respond. That assumption has proved to be accurate for many years. The U.S. economy is amazingly resilient and stimulates new business formation even in the worst of times. The technical note accompanying last week’s report includes the following statement:
The first component excludes employment losses from business deaths from sample-based estimation in order to offset the missing employment gains from business births. This is incorporated into the sample-based estimation procedure by simply not reflecting sample units going out of business,but imputing to them the same employment trend as the other firms in the sample. This procedure accounts for most of the net birth/death employment.
After this imputation step the BLS applies the much-maligned Birth-Death adjustment. The target of critics for more than a decade, it is much less significant than the imputation assumption.
Under current circumstances, the imputation assumption is implausible in the extreme. The BLS recognizes the issue. In their most recent methodological update from July 2, 2020 they report as follows:
The widespread disruption to labor markets due to the COVID-19 pandemic and the potential impact to the birth-death model have prompted BLS to both revisit research conducted in the aftermath of the Great Recession (2008-2009) and incorporate new ideas to account for changes in the number of business openings and closings. Two areas of research have been implemented to improve the accuracy of our birth-death model in the CES estimates. These adjustments will better reflect the net effect of the contribution of business births and deaths to the estimates. These two methodological changes are the following:
A portion of both reported zeros and returns from zero in the current month from the sample were used in estimation to better account for the fact that business births and deaths will not offset.
Current sample growth rates were included in the net birth-death forecasting model to better account for the changing relationships between business openings and closings.
Technically-minded readers should feel free to study the changes in detail. Here is a taste:
Secondly, BLS adjusted the portion of business births and deaths that cannot be accounted for using our sample data by including more recent information. Net birth-death forecasts are normally modeled using an auto-regressive integrated moving average (ARIMA) based on 5 years of historical birth-death residual values that end 9 months before the forecast of the current month. Instead of using only historical data—data that would not accurately account for how the labor market has changed due to COVID-19—a regression variable that includes data up to the current month was included in the model. The regression variable is the CES sample-based ratio of over-the-month change, known as the sample link, for each of the major industry sectors. Each major industry sector sample link was used for the basic-level industries only within that sector.
So instead of the imputation plus a birth-death adjustment, the BLS will (temporarily) attempt to use regression results from individual industries to estimate the “gap” from the normal method. Here is the table of these adjustments as applied to June.
The Effect of Revisions
Won’t the current problem be fixed by the upcoming revisions? Not very quickly. The first two revisions mostly reflect additional responses along with some additional seasonal adjustment. The business deaths we are concerned about will remain among the non-respondents. Eventually we will learn the truth. The Quarterly Census of Employment and Wages presents a count based upon state unemployment insurance programs. It is close to an actual count of private sector jobs and is the basis for benchmark revisions in the nonfarm payroll series. Unfortunately, compiling this report takes a long time. It will be six months before we see the Q2 2020 data.
Conclusions, So Far
I applaud the BLS for recognizing this problem and for taking action. I know from my past work and discussions with them that they had a firm grasp on the issue. Changing the methodology is a politically sensitive matter, even when it is necessary.
Despite this effort, I am very concerned about the result. Here are some reasons, which I will cover more completely in future segments of this analysis.
- For outside analysts to be confident of the application of regressors to the adjustment, we need to review the research.
- Regression equations based upon past data are not helpful when there is no similar prior period.
- The seasonal adjustment process used is not built for violent changes, as we are seeing now.
- The reported results do not pass a “sniff test.” I am listing my concerns as the basis for further discussion. More to come.
- The sector rebounds are implausible, especially the rebound in hospitality.
- The results are inconsistent with weekly jobless claims and continuing claims. These actual counts only cover job losses but present a tall order to find offsetting job gains.
- News reports can easily trap you in anecdotes, but there have been plenty of bankruptcy stories in retail and restaurant permanent closings on YELP. I have seen little about new business formation, especially in these sectors.
- Financing a new business is a big challenge in the current environment.
- The ADP report, using different methodology generated far different results (before revision to conform to the BLS result).
What to Watch
I may be the only one saying this, but the most important piece of information to watch in Friday’s report is the response rate! May’s first release was 69.4% and June’s was 63.1%, below the recent rate of 70%. May’s first revision was 85.5%, below the recent 90% level.
This may not seem like a great discrepancy but refer to the original table. If the number of missing establishments reflects a proportionate number of jobs, a 4% miss could be over 5 million jobs.