Treasury staff insight

MEI Special Topic: Illustrating Forecast Uncertainty

Posted
Topics:
Publication type:
Publication category:

Staff and teams are writing in their individual capacity and the views are not necessarily a "Treasury" view. Please read our disclaimer.

Figure 1: Real Expenditure GDP.

The Treasury publishes economic forecasts at least twice during the year. These forecasts are communicated as a central track which represents our view of the most likely path the economy will take over a forecast horizon. However, forecast uncertainty is ever present.

There are four main reasons for forecast uncertainty:

  • Firstly, the forecast is a conditional forecast. Our assessment of the economic outlook is based on current economic conditions and policy settings. If there is a change in fiscal policy or an unpredictable economic event happens, we need to revise our forecasts. For this reason, the actual outcome could be quite different from the forecasts.
  • Secondly, economic forecasting is a blend of both art and science.  Although the forecasts are generated by a set of quantitative relationships and identities, assumptions and judgements always come into play.
  • Thirdly, the economy is made up of a large number of different individuals whose behaviour and motivations vary over time, meaning that past relationships may not be a good indicator to the future.
  • The fourth area is that there are measurement errors associated with most economic variables.

Scenarios are one way to illustrate uncertainty…

In order to illustrate the risks to the forecasts, the Treasury usually provides two alternative scenarios which illustrate how outcomes could differ from the central forecasts. For example, in the Half Year Economic and Fiscal Update 2016 (HYEFU), one of the scenarios was about the stronger growth in the latter half of 2016 accelerating in 2017. It has turned out that the nominal GDP outturn is more in line with this scenario than the central forecast even though some of the underlying story described in the scenario did not eventuate. The main reason for providing scenario analysis is to illustrate that the economic outlook is uncertain, which policy makers need to take into account when making decisions.

…but another is through the use of fan charts

This special topic examines another approach for presenting the inherent uncertainty of economic forecasts. The uncertainty about the future outcomes of a variable is described by a fan chart that quantifies the probability of a particular observation falling below or exceeding certain values.  The approach was first employed by the Bank of England[1]. The New Zealand Treasury has used fan charts for describing uncertainty associated with its tax revenue forecasts since the 2010 Half Year Update. The methodology used in this note is similar to that used in producing tax revenue fan charts.  A full summary of the methodology can be found in Treasury Working Paper 10/08.

Past forecast errors are used as the basis for determining the likely range of possible outcomes. The greater the historical errors, the greater the extent of uncertainty surrounding any point estimate. The data used in this special topic are based on the Treasury’s Budget and Half Year Update forecasts made over the period from 2001 to 2016. In total, there are 31 forecasts.  In addition a key assumption is made that the resulting forecast errors are normally distributed.  This assumption means that the width of the fans either side of the central forecast are symmetric.  It also means that two thirds of observations are expected to fall within one standard deviation of the central forecast and 90 percent within two standard deviations.  The greater the historical errors then the larger the resulting standard deviations of the errors and hence the wider the fan charts[2].  

Presented below are a set of fan charts for a number of key economic variables, namely real GDP, 90-day interest rates, the Trade Weighted Index exchange rate, the unemployment rate, inflation, and nominal GDP.  In this set of graphs, the outer limit of the fan chart represents the 90th percentile of the 2016 HYEFU forecasts. In other words, we would expect that in only 10 (5 below and 5 above) out of 100 occasions the actual outcomes will fall outside the coloured area of the fan chart. The 70th percentile are represented by the inner dark grey band. Each pair of lighter colour area represent another 10th percentile.

Uncertainty associated with forecasts of real GDP

Figure 1 shows a fan chart for real expenditure GDP. The width of the fan increases further into the forecast period, highlighting that the accuracy of predictions deteriorates as the length of the forecast horizon increases. In other words, unsurprisingly, the further away from the present the more uncertain the future is.

Figure 1: Real Expenditure GDP

 

Figure 1: Real Expenditure GDP.
Sources: Statistics New Zealand, the Treasury

Furthermore, the distribution of risks is not uniform and the greatest proportion of occurrences lie close to the central forecast.  For example, the fan chart suggests that there is a 70% chance that the actual outturn will be within the band of ± 4% of the central forecasts in 2020q2 and there is 90% chance that the actual outturn will be within the band of ± 6% of the central forecasts for the same quarter.  That is to increase the number of observations covered by the band by 29% (ie from 70 to 90 percent), we need to increase the width of the band by 50% (from 4% to 6%).

Finally, it is also interesting to note that both the scenarios that were presented in the HYEFU lie well within the 70th percentile. It suggests that both the scenarios in the update were not “black swan” events (eg. the GFC) that by their nature are unpredictable and fall outside the normal range of possibilities, but instead illustrate scenarios within normal bounds.

Uncertainty associated with forecasts of interest rates

Figure 2: 90-day Bill Rate

 

Figure 2: 90-day Bill Rate.
Sources: Reserve Bank, the Treasury

Figure 2 illustrates the uncertainty associated with the interest rate track. These forecasts assumed that the Reserve Bank would hold its benchmark rate at a record low for a sustained period of time with the next move not predicted until the end of 2018. Figure 2 shows that by the end of 2017, there is a 30% chance that the 90-day bill rate will have risen well above 3.5% or have fallen below 0.5%. However, there is a limit as to how low nominal interest rates can go when the interest rate reaches zero. Therefore, when the forecasted path for interest rate is low, a truncated fan chart may be appropriate.

Uncertainty is particularly high for forecasts of the exchange rate

Figure 3: Trade Weighted Index

 

Figure 3: Trade Weighted Index.
Sources: Reserve Bank, the Treasury

Figure 3 shows that there is a high chance that the exchange rate path will turn out to be very different from the forecast track in the HYEFU. Forecast errors for the exchange rate are relatively large compared with other key macro variables, highlighting that our ability to forecast the exchange rate is not as good as it is for other economic variables (for which there is still considerable uncertainty).

Uncertainty associated with forecasts of the unemployment rate

For the unemployment rate forecasts, the 70% percentile interval ranged from 5.8% to 3.8% for the one-year-ahead forecasts (see Figure 4). To put it another way, at the time the forecasts were published it is estimated that there was a 15% chance that the unemployment rate would increase to 5.8% or higher within a year and there was also the same probability that the unemployment rate would decrease to 3.8% or lower within a year.

Figure 4: Unemployment Rate

 

Figure 4: Unemployment Rate.
Sources: Statistics New Zealand, the Treasury

Inflation

Apart from real output, another key macro variable that has a significant influence on nominal GDP is the consumers price index (CPI). Figure 5 shows that the magnitude of forecast errors is similar to that of real expenditure GDP for forecast horizons of more than 2 years. For example, there is a 10% chance that the level of CPI is 5% higher or lower in the second quarter of 2020 than was forecast in HYEFU.

Figure 5: Consumer Price Index

 

Figure 5: Consumer Price Index.
Sources: Statistics New Zealand, the Treasury

Nominal GDP

The final figure displays a fan chart for nominal GDP. The overall level of uncertainty is slightly higher than that for real output, reflecting that past forecast errors for both inflation and output are not highly correlated. Furthermore, some other macro variables such as the terms of trade also play a vital role in determining the level of nominal GDP. There is a 10% chance that the level of nominal GDP will be 7% higher or lower than expected in 2020Q2.

Figure 6: Nominal GDP

 

Figure 6: Nominal GDP.
Sources: Statistics New Zealand, the Treasury

In summary, we have presented a set of fan charts to provide quantitative assessments of uncertainty surrounding the HYEFU 2016 forecasts. This additional information helps to give a visual picture of the potential uncertainty associated with the outlook, which is relevant for policy decisions.

Notes

  • [1] Inflation Report February 1996, Bank of England
  • [2] For example, the periods used to construct the fans include extreme events, such as the global financial crisis (GFC) and the Canterbury earthquakes.

 

Feedback

Treasury Staff Insights: Rangitaki Mailbox
Email: [email protected]


The Treasury welcomes feedback on this article. If you intend to email comments or questions on this article please note these provisions.

 

Treasury Staff Insights: Rangitaki

See Treasury Staff Insights: Rangitaki for other articles in this series.