Tuesday, October 22, 2013

Why Counterfactual Thinking Is Important

I recently made the case that many observers are not thinking properly about the Fed's Quantitative Easing (QE) programs. Using the analogy of George Bailey's life in the film It's a Wonderful Life, I argued that the critics who question the efficacy of the QE programs are doing the wrong counterfactual. Today, Barry Ritholtz makes the same point:
One of the analytical errors I seem to constantly come across is what I call the non-result result. It goes something like this: If you do X, and there is no measurable change, X is therefore ineffective.
The problem with this analysis is the lack of a control group, If you are testing a new medication to reduce tumors, you want to see what happened to the group that did not get the tested therapy. Perhaps their tumors grew and metastasized. Hence, no increase in tumor mass or spreading is considered a very positive outcome.

This seems to get loss in the debate over QE. The debate — either ignorantly or disingenuously — makes claims such as “Look how few jobs have been created, and look how high unemployment is.”
Understanding this logic, and lacking a control group, we must employ a counter-factual. The question one should be asking is “How many less jobs would have been created?;  How much higher would unemployment be?”
This idea is nicely summarized by the QE counterfactual produced by Poltiical Calculations. It shows what would have happened to nominal GDP had there been no QE3. It is not a pretty sight:

I too ran a QE counterfactual in my George Bailey post. There I considered what would have happened to employment, the stock market, PCE core inflation, and the repo interest rate conditional on (1) the Fed not increasing its share of marketable treasuries starting in late 2010 and (2) the Eurozone crisis, China slowdown, and fiscal policy shocks still occurring as they did. The implications for the economy were the same as in the figure above. 

These counterfactual exercises serve as a nice complement to the on-going quasi-natural experiments that indicate monetary policy at the ZLB can still pack a punch. Yes, monetary policy is far from perfect. But it is also far from impotent as claimed by some QE skeptics.


  1. The fact that some people dismiss QE without the counterfactual is worth noting; the fact that "serious economists" are contending that even large amounts of QE will hardly budge GDP or inflation is astounding. Yi Wen, St. Louis Fed, just published a paper to this effect; some NY Fed researchers did the same back in 2012. (There is even talk of disinflation in these papers).
    This opens up an incredible vista, if true: Paying down serious amounts of national debt, without the inflation penalty.
    Wen tosses about numbers in the trillions; even if QE were quadrupled almost nothing would happen.
    Great Ball of Fire.
    We could pay off, say, half the national debt in the next year or two, scot-free.

  2. David, very nice article. I also liked the link to Noah's "derp" post: good stuff.

    O/T: I'm asking all my favorite econ bloggers about this: George Selgin thinks this new popular video:

    is tin foil hat material:

    Your take? (A minute or two of viewing should be sufficient)

    1. Tom, if only we had videos this well produced from our view point. It gets some things right and a lot wrong. I would like to create one like this on monetary policy, safe assets, interest rates and how they all intersect.

  3. This comment has been removed by the author.

  4. One of the many problems in using counterfactuals in this sense is that there is only one data point. In order to establish causation you need many data points to calculate confidence intervals. Please understand that next time the same conditions arise, due to the lack of established causality, the outcome may be different. Thus, any attempt to justify QE on the basis of counterfactuals is naive unless causality is established within reasonable bounds. Please note that I also believe that QE was very helpful in averting deflation but beliefs worth little unless they are justified. And justification is difficult in the case of counterfactual thinking unless there is statistical significance, which your study lacks because it deals with a single event. Thank you.