Macroeconomic forecasting is an exercise enveloped in significant uncertainty but of critical importance in informing policy-makers on the most likely economic scenario over the short-to-medium term.

In examining the statistical forecasting performance in GDP growth, as produced by the Finance Ministry between 2005 and 2011, the magnitude of forecast errors will be largely dependent on the particular vintage of data used as a benchmark. The target growth rates vary, depending on the vintage of data used as baseline and also as official data is continuously revised by statistics authorities.

While the most recent revised data tends to be more reliable than the first release, using the latest revised data as the benchmark to analyse forecast error will add a further degree of uncertainty to the analysis – revisions of historical data will impact on the degree of accuracy of forecast estimates both ex-ante and ex-post. The ex-ante error may arise due to changes in historical data on which the forecast figures were estimated; the ex-post forecast error may emerge as the base on which the forecast figures were estimated in the first place would have changed.

Defining forecast errors as forecast figures less the actual figures reported implies that a positive mean forecast error represents an over-prediction of the rate of GDP growth. A negative value implies an under-prediction. Based on this definition, real GDP forecast estimates were on average less than 0.2 percentage points lower than the rates as published in the fist vintage of data.

There was a deviation of around -1.5 percentage points in 2005 while in 2008 GDP forecast was over-estimated by around 1.2 percentage points. Overall, data does not suggest any systematic bias in GDP forecast when measured in real terms. On the other hand, a positive bias seems present in the forecast of nominal GDP with forecasts tending to be higher than actual outturns.

However, were one to exclude outliers for 2005 and 2009, the bias in the nominal GDP growth has consistently declined. On the other hand, removing the same outlier years, there appears to be a small bias indicating a tendency to under-predict real GDP growth by around 0.3.percentage points.

Displaying the forecast errors in terms of distribution (Chart 1) helps to visualise the concentration and spread of the forecast error. The forecast errors are grouped into eight classes, ranging from minus three and over to plus three and over. The distribution points towards a generally even spread of error on the upside and downside classes.

In the distribution, the frequency of forecast error at zero is taken as an average of the two adjacent classes, given that, in practice, it is almost impossible that forecast error is exactly equal to zero.

Given that macroeconomic forecasting is an international practice, an international comparison of forecast errors helps to put into the right perspective any degree of bias that may exist in forecast produced locally.

Chart 2: International Comparison of Forecast Errors. (Data Source: Frankel, J. (2011) “Over-optimism in forecasts by official budget agencies and its implications”, Oxford Review of Economic Policy Vol. 27 (4))

Chart 2 illustrates the degree of forecast errors in one-year ahead GDP growth of 25 different economies produced by their respective national authorities over a number of years. Malta is one of the few countries which, on average, tend to slightly under predict a one-year ahead real GDP growth. These results, produced in an international publication, are in line with the result and conclusion drawn from Chart 1.

It is more opportune to recognise the degree of uncertainty surrounding forecasts figures and recognise the validity of reporting estimates in terms of probability outcomes. One way to do this is to make use of fan charts, which can be described as graphical representations of forecast probabilities. The fan chart emphasises the inevitable uncertainty around the outlook for the economy. This could reflect uncertainty about the future economic environment: for example, the outlook for world GDP, prices of oil and other commodities and political developments around the globe. It could also reflect uncertainty about the structure of the economy itself.

Chart 3: Fan chart

Presenting GDP forecasts in terms of a fan chart would enable an illustration of the probabilistic realisation of GDP growth if today’s economic circumstances were experimentally repeated over several times.

Future economic developments are conditioned by a significant degree of uncertainty. Obtaining reliable forecasts depends on the ability to take a balanced view of possible economic developments and the ability to appreciate and evaluate the surrounding uncertainties.

An assessment of past performance of forecast errors for Malta indicates that, on average, forecast errors are relatively small and with a fairly even spread . GDP forecast figures do not seem to exhibit any significant and systematic trend in over-predicting or under-predicting real GDP growth over the short-term horizon. Some indications of overshooting with regards nominal GDP were observed but there is some evidence that this has declined over the last few years.

This article is an adaptation of an analysis of the topic included in the Economic Bulletin of June 2012, published by the Economic Policy Department within the Ministry of Finance, the Economy and Investment.