Clark-West adj MSPE test To evaluate statistical significance of the out-of-sample forecasts provided by the various methodologies, we implement the Clark and West (2007) adjusted mean square prediction error (MSPE) test. The null hypothesis of the test is that the forecasting accuracy between a benchmark and an alternative forecasting model is equal. As benchmark we consider the RW model. For the test are used standard normal critical values, as suggested by the authors. P-values of the test are reported in Table 6. Clark and West adjusted Mean Square Prediction Error test results Panel A: Monthly data Panel B: Daily data Bias P-value Bias P-value Model EUR/USD ARIMA GARCH AR-SVR MARS-SVR EEMD-AR-SVR EEMD-MARS-SVR USD/JPY ARIMA GARCH AR-SVR MARS-SVR EEMD-AR-SVR EEMD-MARS-SVR AUD/NOK ARIMA GARCH AR-SVR MARS-SVR EEMD-AR-SVR EEMD-MARS-SVR NZD/BRL ARIMA GARCH AR-SVR MARS-SVR EEMD-AR-SVR EEMD-MARS-SVR ZAR/PHP ARIMA GARCH AR-SVR MARS-SVR EEMD-AR-SVR EEMD-MARS-SVR 0.002 -0.003 0.009 0.103 0.005 -0.024 0.234 0.387 0.169 0.221 0.029** 0.099* 0.000 0.000 0.000 0.000 0.000 0.000 0.407 0.441 0.413 0.413 0.000*** 0.000*** -2.724 -1.035 -3.141 -14.819 -1.316 0.771 0.153 0.112 0.416 0.832 0.135 0.008*** -0.068 -0.063 -0.173 -0.174 -0.070 -0.075 0.247 0.228 0.220 0.220 0.000*** 0.000*** 0.084 0.062 0.165 0.255 0.049 0.593 0.698 0.883 0.892 0.890 0.107 0.913 0.004 0.004 0.008 0.008 0.005 0.000 0.056* 0.098* 0.099* 0.099* 0.000*** 0.000*** 0.012 0.022 0.004 -0.030 0.006 0.032 0.002*** 0.000*** 0.000*** 0.099* 0.000*** 0.141 0.000 0.000 0.000 0.000 0.000 0.000 0.093* 0.178 0.095* 0.095* 0.000*** 0.000*** -0.012 0.023 -0.013 -0.111 0.001 0.086 0.131 0.441 0.144 0.046** 0.104 0.295 0.002 0.001 0.000 0.000 0.002 -0.002 0.220 0.244 0.351 0.351 0.000*** 0.000*** Note: *,** and *** report rejection of the null hypothesis of equal forecasting ability with the RW model on 10 %, 5% and 1% level of significance. The results of the Clark-West adj-MSPE test on monthly data suggest that no model consistently outperforms the benchmark or that in other words the results of Table 4 regarding the EEMD-AR-SVR model are not statistically significant. . Nevertheless, we should note that the relative small number of thirty observations and consequently little degrees of freedom used for the out-of-sample forecasting might explain this inconsistency, as the small size of the sample lowers the statistical power of the test. On the other hand on daily basis, the test provides strong statistical validation (1% level of significance) on the forecasting superiority of the EEMD-MARS-SVR and EEMDAR-SVR over the benchmark model.
© Copyright 2024 ExpyDoc