组合预测是集成学习的重要组成分支, 也是提升预测或分类精度的有效手段之一. 预测模型的整体偏差度量方法应用成熟广泛, 例如在组合预测中, 集成模型的样本内均方误差是传统权重优化方法的主要目标损失. 然而由于"过拟合"风险的存在, 训练误差的最小化并非最小化泛化误差的充分条件. 因此为了增加权重优化主体的多样性、减少权重过拟合风险和控制模型尾部损失等, 本文定义了用以刻画组合模型极端偏差的度量指标. 在此基础上, 设计了一种全新的权衡整体和极端偏差的目标损失函数, 并构建了基于粒子群优化算法的最优权值求解方法. 在黄金和原油价格数据上的仿真实验结果表明, 本文所提出的组合预测方法能够有效对抗传统方法的过拟合问题, 与简单平均、最优权重法等基准模型相比, 能够较好地提升组合预测模型的泛化性能, 降低模型预测误差.
Abstract
Forecast combination is an important branch of ensemble learning and it is also an effective tool to improve the forecasting accuracy. The global bias estimation method of the forecaster has a mature application in various aspects. For instance, ensemble model's in-sample mean square error is a main objective of traditional weight optimization approaches in forecast combination. However, due to the risk of "overfitting", the minimization of the training error does not necessarily imply a corresponding minimization of the generalization error. Therefore, to increase diversity of optimization objectives, reduce risk of weight overfitting, and control tail error of ensemble model, this paper defines a metric to capture extreme bias of combined forecaster. Furthermore, a novel objective function is proposed which can make a trade-off between the global and extreme bias to achieve optimal weights. Specifically, the particle swarm optimization method is introduced to achieve the optimal combination weights. The experimental results on gold price and oil price data demonstrate that the proposed forecast combination approach can efficiently reduce overfitting risk and improve the generalization ability, outperforming simple averaging, optimal weight method and other benchmark models.
关键词
组合预测 /
极端偏差 /
偏差权衡优化 /
粒子群优化算法
{{custom_keyword}} /
Key words
forecast combination /
extreme bias /
bias trade-off /
particle swarm optimization
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 周志华. 集成学习:基础与算法[M]. 北京:电子工业出版社, 2020.Zhou Z H. Ensemble methods:Foundations and algorithms[M]. Beijing:Publishing House of Electronics Industry, 2020.
[2] Kang Y, Cao W, Petropoulos F, et al. Forecast with forecasts:Diversity matters[J]. European Journal of Operational Research, 2022, 301(1):180-190.
[3] Lamberson P J, Page S E. Optimal forecasting groups[J]. Management Science, 2012, 58(4):805-810.
[4] Atiya A F. Why does forecast combination work so well?[J]. International Journal of Forecasting, 2020, 36(1):197-200.
[5] Ganaie M A, Hu M, Malik A K, et al. Ensemble deep learning:A review[J]. Engineering Applications of Artificial Intelligence, 2022, 115:105151.
[6] Aiolfi M, Timmermann A. Persistence in forecasting performance and conditional combination strategies[J]. Journal of Econometrics, 2006, 135(1-2):31-53.
[7] Breiman L. Statistical modeling:The two cultures[J]. Statistical Science, 2001, 16(3):199-215.
[8] Makridakis S, Spiliotis E, Assimakopoulos V. The M4 Competition:Results, findings, conclusion and way forward[J]. International Journal of Forecasting, 2018, 34(4):802-808.
[9] Bojer C S, Meldgaard J P. Kaggle forecasting competitions:An overlooked learning opportunity[J]. International Journal of Forecasting, 2021, 37(2):587-603.
[10] Kourentzes N, Barrow D, Petropoulos F. Another look at forecast selection and combination:Evidence from forecast pooling[J]. International Journal of Production Economics, 2018, 209:226-235.
[11] Agnew C E. Bayesian consensus forecasts of macroeconomic variables[J]. Journal of Forecasting, 2010, 4(4):363-376.
[12] Kourentzes N, Barrow D K, Crone S F. Neural network ensemble operators for time series forecasting[J]. Expert Systems with Applications, 2014, 41(9):4235-4244.
[13] Barrow D K, Kourentzes N. Distributions of forecasting errors of forecast combinations:Implications for inventory management[J]. International Journal of Production Economics, 2016, 177:24-33.
[14] Elliott G, Timmermann A. Handbook of economic forecasting[M]. Elsevier, 2013.
[15] Stock J H, Watson M W. Combination forecasts of output growth in a seven-country data set[J]. Journal of Forecasting, 2004, 23(6):405-430.
[16] 张新雨, 邹国华. 模型平均方法及其在预测中的应用[J]. 统计研究, 2011(6):97-102.Zhang X Y, Zou G H. Model averaging method and its application in forecast[J]. Statistical Research, 2011(6):97-102.
[17] Matsypura D, Thompson R, Vasnev A L. Optimal selection of expert forecasts with integer programming[J]. Omega, 2018, 78(1):165-175.
[18] Montero-Manso P, Athanasopoulos G, Talagala T S, et al. FFORMA:Feature-based forecast model averaging[J]. International Journal of Forecasting, 2020, 36(1):86-92.
[19] Burnham K P, Anderson D R. Model selection and multimodel inference:A practical information-theoretic approach[M]. Springer, 2002.
[20] Bates J M, Granger C. The combination of forecasts[J]. Journal of the Operational Research Society, 1969, 20(4):451-468.
[21] Yang Y H. Combining forecasting procedures:Some theoretical results[J]. Econometric Theory, 2004, 20(1):176-222.
[22] Aiolfi M, Timmermann A. Persistence in forecasting performance and conditional combination strategies[J]. Journal of Econometrics, 2006, 135(1-2):31-53.
[23] Geweke J, Amisano G. Optimal prediction pools[J]. Journal of Econometrics, 2008, 164(1):130-141.
[24] Smith J, Wallis K F. A simple explanation of the forecast combination puzzle[J]. Oxford Bulletin of Economics and Statistics, 2009, 71(3):331-355.
[25] Claeskens G, Magnus J R, Vasnev A L, et al. The forecast combination puzzle:A simple theoretical explanation[J]. International Journal of Forecasting, 2016, 32(3):754-762.
[26] Blanc S M, Setzer T. When to choose the simple average in forecast combination[J]. Journal of Business Research, 2016, 69(10):3951-3962.
[27] Blanc S M, Setzer T. Bias-variance trade-off and shrinkage of weights in forecast combination[J]. Management Science, 2020, 66(12):5720-5737.
[28] Li D, Jiang F, Chen M, et al. Multi-step-ahead wind speed forecasting based on a hybrid decomposition method and temporal convolutional networks[J]. Energy, 2022, 238:121981.
[29] Liu J, Wang P, Chen H, et al. A combination forecasting model based on hybrid interval multi-scale decomposition:Application to interval-valued carbon price forecasting[J]. Expert Systems with Applications, 2022, 191:116267.
[30] Wang J, Hao Z, Tao H, et al. A multi-granularity heterogeneous combination approach to crude oil price forecasting[J]. Energy Economics, 2020, 91:104790.
[31] Markowitz H. Portfolio selection[J]. The Journal of Finance, 1952, 7(1):77-91.
[32] 冯明宇. VaR和CTE估计的经验似然方法[D]. 杭州:浙江大学, 2013.Feng M Y. A method for estimating VaR and CTE via empirical likelihood[D]. Hangzhou:Zhejiang University, 2013.
[33] Acerbi C, Nordio C, Sirtori C. Expected shortfall as a tool for financial risk management[J]. Quantitative Finance, 2001, 31(2):379-388.
[34] 韩天雄. 非寿险精算[M]. 北京:中国财政经济出版社, 2010.Han T X. Actuarial aspects of non-life insurance[M]. Beijing:China Financial & Economic Publishing House, 2010.
[35] Kennedy J, Eberhart R. Particle swarm optimization[C]//ICNN'95-International Conference on Neural Networks, IEEE, 1995, 4:1942-1948.
[36] Shami T M, El-Saleh A A, Alswaitti M, et al. Particle swarm optimization:A comprehensive survey[J]. IEEE Access, 2022, 10:10031-10061.
[37] 周浩, 张逸飞, 王震, 等. 一种改进的原油价格组合预测优化策略研究[J]. 系统工程理论与实践, 2021, 41(10):2660-2668.Zhou H, Zhang Y F, Wang Z, et al. An improved research on optimization strategy of crude oil price forecast combination[J]. Systems Engineering—Theory & Practice, 2021, 41(10):2660-2668.
[38] Wang L L, Liu L W, Wang Z, et al. Strategic behavior and optimization in an unobservable constant retrial queue with balking and set-up time[J]. Journal of Systems Science and Information, 2020, 8(3):273-290.
[39] Zhang Y, Wang J, Yu L, et al. An extreme bias-penalized forecast combination approach to commodity price forecasting[J]. Information Sciences, 2022, 615:774-793.
[40] Pesaran M H, Timmermann A. A simple nonparametric test of predictive performance[J]. Journal of Business & Economic Statistics, 1992, 10(4):461-465.
[41] Hansen P R, Lunde A, Nason J M. The model confidence set[J]. Econometrica, 2011, 79(2):453-497.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金（72271229，71988101，71771208）；国家能源集团2021年度十大软课题《能源系统模型构建与中国能源展望研究》（GJNY-21-141）
{{custom_fund}}
PDF(846 KB)
