Clark-West adj MSPE test
To evaluate statistical significance of the out-of-sample forecasts provided by the
various methodologies, we implement the Clark and West (2007) adjusted mean square
prediction error (MSPE) test. The null hypothesis of the test is that the forecasting
accuracy between a benchmark and an alternative forecasting model is equal. As
benchmark we consider the RW model. For the test are used standard normal critical
values, as suggested by the authors. P-values of the test are reported in Table 6.
Clark and West adjusted Mean Square Prediction Error test results
Panel A: Monthly data
Panel B: Daily data
Bias
P-value
Bias
P-value
Model
EUR/USD
ARIMA
GARCH
AR-SVR
MARS-SVR
EEMD-AR-SVR
EEMD-MARS-SVR
USD/JPY
ARIMA
GARCH
AR-SVR
MARS-SVR
EEMD-AR-SVR
EEMD-MARS-SVR
AUD/NOK
ARIMA
GARCH
AR-SVR
MARS-SVR
EEMD-AR-SVR
EEMD-MARS-SVR
NZD/BRL
ARIMA
GARCH
AR-SVR
MARS-SVR
EEMD-AR-SVR
EEMD-MARS-SVR
ZAR/PHP
ARIMA
GARCH
AR-SVR
MARS-SVR
EEMD-AR-SVR
EEMD-MARS-SVR
0.002
-0.003
0.009
0.103
0.005
-0.024
0.234
0.387
0.169
0.221
0.029**
0.099*
0.000
0.000
0.000
0.000
0.000
0.000
0.407
0.441
0.413
0.413
0.000***
0.000***
-2.724
-1.035
-3.141
-14.819
-1.316
0.771
0.153
0.112
0.416
0.832
0.135
0.008***
-0.068
-0.063
-0.173
-0.174
-0.070
-0.075
0.247
0.228
0.220
0.220
0.000***
0.000***
0.084
0.062
0.165
0.255
0.049
0.593
0.698
0.883
0.892
0.890
0.107
0.913
0.004
0.004
0.008
0.008
0.005
0.000
0.056*
0.098*
0.099*
0.099*
0.000***
0.000***
0.012
0.022
0.004
-0.030
0.006
0.032
0.002***
0.000***
0.000***
0.099*
0.000***
0.141
0.000
0.000
0.000
0.000
0.000
0.000
0.093*
0.178
0.095*
0.095*
0.000***
0.000***
-0.012
0.023
-0.013
-0.111
0.001
0.086
0.131
0.441
0.144
0.046**
0.104
0.295
0.002
0.001
0.000
0.000
0.002
-0.002
0.220
0.244
0.351
0.351
0.000***
0.000***
Note: *,** and *** report rejection of the null hypothesis of equal forecasting ability with
the RW model on 10 %, 5% and 1% level of significance.
The results of the Clark-West adj-MSPE test on monthly data suggest that no model
consistently outperforms the benchmark or that in other words the results of Table 4
regarding the EEMD-AR-SVR model are not statistically significant. . Nevertheless,
we should note that the relative small number of thirty observations and consequently
little degrees of freedom used for the out-of-sample forecasting might explain this
inconsistency, as the small size of the sample lowers the statistical power of the test.
On the other hand on daily basis, the test provides strong statistical validation (1% level
of significance) on the forecasting superiority of the EEMD-MARS-SVR and EEMDAR-SVR over the benchmark model.

Th P03 14 Random Noise Attenuation by a Hybrid

PDF handout - G. William Schwert