Optimal Monetary Policy When the Economy Must Be Learned

The optimal policy of Adrian, Laxton, and Obstfeld’s edited volume, Advancing the Frontiers of Monetary Policy (International Monetary Fund, 2018), minimizes a quadratic loss against a calibrated model. When the model of the economy must be learned from data, learning enters at four levels: the economy’s agents, the econometrician, a residual correction, and the policymaker. Each is measured on real-time data for the United States, three inputs excepted (Section 2), and the policymaker’s rule is compared with the realized federal funds rate.

On what data is each learner measured?

Every observation the model reads at a decision point is one the FOMC could have read before its meeting, with three exceptions: the neutral-rate path is the ex post Holston–Laubach–Williams estimate, real-time from 2016:Q1 for the rolling agent of Section 8; core personal consumption expenditures (PCE) inflation is read from one vintage of the price index; and the term premium is read from one release. The data hold 146 quarterly decision points from 1990:Q1 through 2026:Q2, assembled from 30 series, each held with every release as a separate vintage. The estimation window runs from 1996:Q1 through 2026:Q2: 122 quarters with 836 observation rows, an observation row being one observed quantity in one quarter. Of its quarters, 36 are at the ELB, in two ELB episodes, 2009:Q1 through 2015:Q4 and 2020:Q2 through 2022:Q1. The agents are trained from, and compared at, the 118 decision points from 1997:Q1 through 2026:Q2.

What is optimal policy, and who learns what?

The volume’s optimal policy minimizes, at each decision, a loss quadratic in the deviation of four-quarter inflation from its target, the output gap, and the change in the policy rate; equation (1) sums it from one quarter ahead. The volume’s DM1 policy, which minimizes the volume’s dual-mandate loss at its baseline weights, is a path: the sequence of rates that minimizes the loss from the current state, re-optimized as the state moves. The baseline agent is a rule: an IFB rule whose four coefficients are learned. The agent learns by reinforcement learning in the sense of a policy-gradient method: gradient descent on the loss of simulated episodes in the estimated model, which is solved exactly at every quarter, the gradient taken with the set of quarters at the ELB held fixed. In the volume’s experiment from the 2014:Q4 state, run in the paper’s reproduction of the volume’s model, the volume’s IFB rule lifts off, raising the rate above the ELB, in 2015:Q1. The DM1 path holds the rate at the ELB through 2015:Q4, and its rate in 2016:Q1 stands 0.5 bp above the ELB. The baseline agent holds the rate at the ELB for five quarters and lifts off in 2016:Q2. Their losses under equation (1), summed over the sixteen quarters from 2015:Q1 through 2018:Q4 plus the loss along each policy’s perfect-foresight path from the final state, are 14.44, 9.95, and 10.83: the baseline agent’s loss lies between the DM1 path’s and the volume’s IFB rule’s.

Equation (1), the loss; Figure 1, the four learners; Figure 2, the volume’s experiment on the learned rule.

In what estimated economy is the rule learned?

The world model is a reduced form of the volume’s semi-structural economy: an IS equation, a hybrid Phillips curve, and an IFB policy equation with the ELB, equations (2) through (4), with model-consistent expectations, estimated by maximum likelihood on the real-time data. In a Monte Carlo study on simulated economies with an ELB episode, the estimates of the anchored specification’s weight lie on both sides of its true value, and the study does not show the recovery of the sum of the two IS weights or of the policy equation’s inflation and gap responses. The likelihood determines the scale of the estimated policy equation only weakly, and the paper normalizes the equation’s inflation response, its response to the inflation forecast’s deviation from target on top of a one-for-one response, to 3.0, the only local minimum of the profile at which no parameter lies at a bound. Under the normalization the likelihood has more than one local optimum. Of 44 starting values, 40 are random, and four are optima of the likelihood found outside the random search, one of them the optimum of the likelihood profile at the normalized value. The deepest optimum, the baseline estimate, is reached from two of the four and from none of the 40 random starting values. Its objective, the negative log-likelihood per observation row, lower being better, is 0.1445, and its likelihood exceeds that of the alternative estimate, the optimum reached from the data-driven starting value. The baseline estimate places a weight of 0.946 on expected inflation in the Phillips curve, with a Phillips slope of 0.0596. The Federal Reserve Board staff’s Tealbook projections have the lower root-mean-square error (RMSE) on every line of Table 2; for four-quarter inflation four quarters ahead, the model’s RMSE over the line’s 83 quarters is 1.32 times the staff’s. The paper’s test, two-sided at the 5 percent level, distinguishes the difference in squared error from zero on one line alone, that of four-quarter inflation four quarters ahead over its 75 consecutive quarters through 2018:Q3.

Equations (2) through (4), the IS equation, the Phillips curve, and the policy equation; Table 1, the baseline estimate; Figure 3, the profile in the inflation response and the optima of the search; Table 2, the model against the staff’s Tealbook projections.

Do the data select model-consistent expectations against their alternatives?

Three families of departures, a fixed anchor, cognitive discounting, and an anchor learned from experience, each nest model-consistent expectations at a parameter value of one. Each family, estimated from the baseline and the alternative estimates as starting values, reaches its deeper optimum at the baseline estimate’s parameters, with its own parameter at the upper bound of one. Holding the anchored family’s weight w at 0.8, with the anchor’s level measured from the long-run survey expectation, lowers the log-likelihood by 166.1; holding the cognitive discount m at 0.8 lowers it by 224.6. The long-run survey expectation is the Survey of Professional Forecasters’ ten-year expectation of consumer price index (CPI) inflation, expressed in core PCE units (the survey less the sample’s mean wedge between headline CPI and core PCE inflation). The learned anchor’s level, the learned level, is a constant-gain mean of realized inflation. With the survey taken as an observation of it, its gain is estimated at 0.030 per quarter, a half-life of about 23 quarters, and the survey’s measurement error about it has an estimated standard deviation of 0.30 pp, larger than the survey’s own standard deviation of 0.21 pp over the sample; the survey lies more than 0.21 pp below the learned level in 43 of the 122 quarters, all of them before 2010:Q1 or from 2023:Q1 on. The survey varies less than the learned level does, although from 2020:Q3 through 2022:Q4 the survey rises by 0.92 pp against 0.68 pp for the learned level. The expectations that enter the IS and Phillips equations place no weight on the learned level. In the Monte Carlo study, at one true value away from model-consistent expectations, the estimates of the learned anchor’s gain lie on both sides of its true value, and four of the five estimates of the cognitive discount lie above its true value.

Table 3, the three expectations families; Figure 4, the likelihood profiles; Figure 5, the long-run survey expectation and the learned level.

What do the IS and Phillips equations leave in their residuals, and is it signal?

A Gaussian process component is fitted to each filtered shock, the residual of the IS or the Phillips equation as the model’s Kalman filter recovers it. An ELB episode makes the filtered shocks depend on the state even at the true parameters, because the observation of the policy rate is censored at the ELB, so the null distribution against which a component is judged is simulated on economies with an ELB episode. Against it, the Phillips component shows no evidence of signal beyond what the ELB episode induces. The IS component shows evidence of signal the IS equation does not capture; Section 9 states it as a limitation of that equation. Each component is capped where it enters a simulation; the capped components, judged against simulated economies whose components are capped in the same way, give the same two results. The baseline agent’s training environment does not include the components.

In which environment is the rule trained, and how is one rule selected?

The policy agent is the rule of equation (4) without its shock, its four coefficients trained on simulated episodes that start from the real-time states of the 118 decision points, each episode at a draw of four coefficients of the IS and Phillips equations around the baseline estimate, without the residual components. From the volume’s 2014:Q4 state, with no shocks, the loss under equation (1) summed over 24 quarters in the estimated model is lower for the baseline agent than for the volume’s IFB rule on 100 of the 100 draws around the estimate, by 2.89 on average. An agent trained with the residual components in its environment has a higher loss than the IFB rule on every one of those draws, by 3.72 on average, in the estimated model without the components. With the components in the model, at the one state and estimate evaluated, the agent trained with them has the lower loss of the two agents. The baseline agent is selected among eight training runs as the checkpoint with the lowest loss on the validation set. It is an inertial rule, with a smoothing coefficient of 0.736, an inflation response of 3.353, the long-run response to the inflation forecast being one plus this coefficient, an output-gap response of 3.159, and an intercept of −0.580 pp. The runs’ ranking on the validation set is nearly the reverse of their ranking on the test set, a second set of starting states and draws, which the selection of the reported checkpoint does not use. Across the eight runs’ checkpoints the inflation response runs from 3.215 to 3.698 while the test losses differ by 0.7 percent of the lowest, so the coefficients are read as one selection among rules of nearly equal loss.

Figure 6, the performance of the trained rules across draws around the estimate; Figure 7, the search over training runs.

How do the baseline agent’s prescriptions compare with the realized federal funds rate?

Table 4, the comparison by period; Figure 8, the baseline agent and the realized federal funds rate; Table 5, the baseline agent, the volume’s IFB rule, and three published rules; Table 6, the frozen estimate beside the baseline estimate; Table 7, the frozen, rolling, and baseline agents against the realized federal funds rate; Figure 9, the frozen, rolling, and baseline agents after 2015:Q4; Table 8, the exchange of rule, parameters, and filtered gap; Table 9, the counterfactual outcomes.

Does the baseline agent’s mean difference depend on the loss weights or on the training run?

Under four variants of the loss weights, the weight $\beta$ on the squared output gap (baseline 1.0) and the weight $\gamma$ on the squared change in the policy rate (baseline 0.50) each halved and raised by half, the mean difference from the realized federal funds rate over the 118 decision points runs from +0.580 pp at $\beta$ = 0.5 to +0.906 pp at $\gamma$ = 0.25. Every variant lies within 0.19 pp of the baseline +0.723 pp. Across the five trainings at the baseline weights, the mean difference runs from +0.723 pp to +0.755 pp, a range of 3.2 bp; the section states the paper’s limitations.

Table 10, the loss weights; Table 11, the repetitions of the training.

What, then, is optimal policy when the economy must be learned?

Optimal monetary policy when the economy must be learned is, in the paper, a rule whose optimality is conditional on the four learners’ estimates: the expectations specification, the estimated model, the residual components, and the selected checkpoint. The baseline agent, the rule trained under the loss of equation (1) in the estimated model, prescribes higher rates than the realized federal funds rate on average in every period of Table 4, a mean difference the paper’s test distinguishes from zero over all quarters and not in every period; in the two ELB episodes the difference cannot be negative. Those higher rates are a property of the conditional rule and not an evaluation of monetary policy as the FOMC conducted it. The paper’s test, two-sided at the 5 percent level, does not distinguish the mean of the frozen agent’s prescription less the baseline agent’s from zero over 2016:Q1 through 2026:Q2 or through 2020:Q4, and it distinguishes that mean from zero from 2021:Q1; the same pair of results holds wherever those quarters are divided from 2019:Q3 through 2025:Q2, so that a change cannot be dated at 2020:Q4, and the paper reports no test of the change. From 2021:Q1 the difference between the frozen and the baseline agent’s prescriptions is accounted for by the output gaps the two estimates infer in quarters in which no observation of the gap enters either filter.

Appendix A, Conditions of the Monte Carlo Study, the 18 conditions O1 through O18 on simulated economies and the parameter bounds of the estimator; Appendix B, Five Questions on the Estimates and the Agents, on the baseline estimate’s Phillips curve, the frozen agent’s responses, the rolling agent’s mean differences, and the rolling estimate’s weight on expected inflation; Appendix C, Mean Differences by Treatment, Paired Differences, and Losses under the CBO Measure, the baseline agent’s mean difference by period under four treatments of the state, the differences between two agents’ prescriptions at the same decision points, and the losses under the CBO measure of the output gap.