Abstract: Accurate prediction of drilling efficiency is critical for developing the earth-rock excavation schedule. The single machine learning (ML) prediction models usually suffer from problems including parameter sensitivity and overfitting. In addition, the influence of environmental and operational factors is often ignored. In response, a novel stacking-based ensemble learning method taking into account the combined effects of those factors is proposed. Through multiple comparison tests, four models, eXtreme gradient boosting (XGBoost), random forest (RF), back propagation neural network (BPNN) as the base learners, and support vector regression (SVR) as the meta-learner, are selected for stacking. Furthermore, an improved cuckoo search optimization (ICSO) algorithm is developed for hyper-parameter optimization of the ensemble model. The application to a real-world project demonstrates that the proposed method outperforms the popular single ML method XGBoost and the ensemble model optimized by particle swarm optimization (PSO), with 16.43% and 4.88% improvements of mean absolute percentage error (MAPE), respectively.

土方开挖过程中钻进效率预测的Stacking集成学习模型

[1]AbbasAK, RushdiS, AlsabaM, et al., 2019. Drilling rate of penetration prediction of high-angled wells using artificial neural networks. Journal of Energy Resources Technology, 141(11):112904.

[2]AbbaspourH, DrebenstedtC, BadroddinM, et al., 2018. Optimized design of drilling and blasting operations in open pit mines under technical and economic uncertainties by system dynamic modelling. International Journal of Mining Science and Technology, 28(6):839-848.

[3]Abu BakarMZ, ButtIA, MajeedY, 2018. Penetration rate and specific energy prediction of rotary-percussive drills using drill cuttings and engineering properties of selected rock units. Journal of Mining Science, 54(2):270-284.

[4]AkünME, KarpuzC, 2005. Drillability studies of surface-set diamond drilling in Zonguldak region sandstones from Turkey. International Journal of Rock Mechanics and Mining Sciences, 42(3):473-479.

[5]BreimanL, 2001. Random forests. Machine Learning, 45(1):5-32.

[6]BuiDT, NhuVH, HoangND, 2018. Prediction of soil compression coefficient for urban housing project using novel integration machine learning approach of swarm intelligence and multi-layer perceptron neural network. Advanced Engineering Informatics, 38:593-604.

[7]CankurtS, SubasiA, 2022. Tourism demand forecasting using stacking ensemble model with adaptive fuzzy combiner. Soft Computing, 26(7):3455-3467.

[8]ChenKL, JiangJC, ZhengFD, et al., 2018. A novel data-driven approach for residential electricity consumption prediction based on ensemble learning. Energy, 150:49-60.

[9]ChenWL, WangXL, CaiZJ, et al., 2021. DP-GMM clustering-based ensemble learning prediction methodology for dam deformation considering spatiotemporal differentiation. Knowledge-Based Systems, 222:106964.

[10]CuiSZ, YinYQ, WangDJ, et al., 2021. A stacking-based ensemble learning method for earthquake casualty prediction. Applied Soft Computing, 101:107038.

[11]DarborM, FaramarziL, SharifzadehM, 2019. Performance assessment of rotary drilling using non-linear multiple regression analysis and multilayer perceptron neural network. Bulletin of Engineering Geology and the Environment, 78(3):1501-1513.

[12]ElkatatnyS, 2018. Application of artificial intelligence techniques to estimate the static Poisson’s ratio based on wireline log data. Journal of Energy Resources Technology, 140(7):072905.

[13]GalarM, FernandezA, BarrenecheaE, et al., 2012. A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4):463-484.

[14]GanC, CaoWH, WuM, et al., 2019. Prediction of drilling rate of penetration (ROP) using hybrid support vector regression: a case study on the Shennongjia area, central China. Journal of Petroleum Science and Engineering, 181:106200.

[15]GuoYY, WangX, XiaoPC, et al., 2020. An ensemble learning framework for convolutional neural network based on multiple classifiers. Soft Computing, 24(5):3727-3735.

[16]HaghighiF, OmranpourH, 2021. Stacking ensemble model of deep learning and its application to Persian/Arabic handwritten digits recognition. Knowledge-Based Systems, 220:106940.

[17]HustrulidWA, KuchtaM, MartinR, 2013. Open Pit Mine Planning and Design, 3rd Edition. Taylor & Francis, Boca Raton, Florida, USA.

[18]KahramanS, 2002. Correlation of TBM and drilling machine performances with rock brittleness. Engineering Geology, 65(4):269-283.

[19]KahramanS, BilginN, FeridunogluC, 2003. Dominant rock properties affecting the penetration rate of percussive drills. International Journal of Rock Mechanics and Mining Sciences, 40(5):711-723.

[20]KaushikA, KaurP, ChoudharyN, et al., 2022. Stacking regularization in analogy-based software effort estimation. Soft Computing, 26(3):1197-1216.

[21]KazemzadehMR, AmjadianA, AmraeeT, 2020. A hybrid data mining driven algorithm for long term electric peak load and energy demand forecasting. Energy, 204:117948.

[22]KoopialipoorM, TootoonchiH, ArmaghaniDJ, et al., 2019. Application of deep neural networks in predicting the penetration rate of tunnel boring machines. Bulletin of Engineering Geology and the Environment, 78(8):‍6347-6360.

[23]LiH, WangXS, DingSF, 2018. Research and development of neural network ensembles: a survey. Artificial Intelligence Review, 49(4):455-479.

[24]LiLL, CenZY, TsengML, et al., 2021. Improving short-term wind power prediction using hybrid improved cuckoo search arithmetic-support vector regression machine. Journal of Cleaner Production, 279:123739.

[25]LiSP, 2018. Research on Excavation Simulation of a Pumped Storage Power Station Based on Excavation and Filling Balance. MS Thesis, Tianjin University, Tianjin, China(in Chinese).

[26]LiZX, WuDZ, HuC, et al., 2019. An ensemble learning-based prognostic approach with degradation-dependent weights for remaining useful life prediction. Reliability Engineering & System Safety, 184:110-122.

[27]LvF, WangJJ, CuiB, et al., 2020. An improved extreme gradient boosting approach to vehicle speed prediction for construction simulation of earthwork. Automation in Construction, 119:103351.

[28]MaoCY, LinRR, ToweyD, et al., 2021. Trustworthiness prediction of cloud services based on selective neural network ensemble learning. Expert Systems with Applications, 168:114390.

[29]Mendes-MoreiraJ, SoaresC, JorgeAM, et al., 2012. Ensemble approaches for regression: a survey. ACM Computing Surveys, 45(1):10.

[30]MengXJ, ChangJX, WangXB, et al., 2019. Multi-objective hydropower station operation using an improved cuckoo search algorithm. Energy, 168:425-439.

[31]MustafaAB, AbbasAK, AlsabaM, et al., 2021. Improving drilling performance through optimizing controllable drilling parameters. Journal of Petroleum Exploration and Production, 11(3):1223-1232.

[32]PaulA, BhowmikS, PanuaR, et al., 2018. Artificial neural network-based prediction of performances-exhaust emissions of diesohol piloted dual fuel diesel engine under varying compressed natural gas flowrates. Journal of Energy Resources Technology, 140(11):112201.

[33]PavlyukevichI, 2007. Lévy flights, non-local search and simulated annealing. Journal of Computational Physics, 226(2):1830-1844.

[34]PengH, ZengZG, DengCS, et al., 2021. Multi-strategy serial cuckoo search algorithm for global optimization. Knowledge-Based Systems, 214:106729.

[35]Pernía-EspinozaA, Fernandez-CenicerosJ, AntonanzasJ, et al., 2018. Stacking ensemble with parsimonious base models to improve generalization capability in the characterization of steel bolted components. Applied Soft Computing, 70:737-750.

[36]QiCC, FourieA, ChenQS, 2018. Neural network and particle swarm optimization for predicting the unconfined compressive strength of cemented paste backfill. Construction and Building Materials, 159:473-478.

[37]RenQB, LiMC, SongLG, et al., 2020. An optimized combination prediction model for concrete dam deformation considering quantitative evaluation and hysteresis correction. Advanced Engineering Informatics, 46:101154.

[38]SaeidiO, TorabiSR, AtaeiM, et al., 2014. A stochastic penetration rate model for rotary drilling in surface mines. International Journal of Rock Mechanics and Mining Sciences, 68:55-65.

[39]SalimiA, RostamiJ, MoormannC, et al., 2016. Application of non-linear regression analysis and artificial intelligence algorithms for performance prediction of hard rock TBMs. Tunnelling and Underground Space Technology, 58:236-246.

[40]ShiX, LiuG, GongXL, et al., 2016. An efficient approach for real-time prediction of rate of penetration in offshore drilling. Mathematical Problems in Engineering, 2016:3575380.

[41]WangH, ZhangYM, MaoJX, et al., 2020. A probabilistic approach for short-term prediction of wind gust speed using ensemble learning. Journal of Wind Engineering and Industrial Aerodynamics, 202:104198.

[42]WangL, ZengY, ChenT, 2015. Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Systems with Applications, 42(2):855-863.

[43]WangN, ZhaoSY, CuiSZ, et al., 2021. A hybrid ensemble learning method for the identification of gang-related arson cases. Knowledge-Based Systems, 218:106875.

[44]WangRC, WuS, 2019. Neural network model based prediction of fragmentation of blasting using the Levenberg-Marquardt algorithm. Journal of Hydroelectric Engineering, 38(7):100-109 (in Chinese).

[45]WangT, LiZJ, YanYJ, et al., 2007. A survey of fuzzy decision tree classifier methodology. Proceedings of the Second International Conference of Fuzzy Information and Engineering, p.959-968.

[46]YanT, ShenSL, ZhouAN, et al., 2022. Prediction of geological characteristics from shield operational parameters by integrating grid search and K-fold cross validation into stacking classification algorithm. Journal of Rock Mechanics and Geotechnical Engineering, 14(4):1292-1303.

[47]YangXS, DebS, 2014. Cuckoo search: recent advances and applications. Neural Computing and Applications, 24(1):169-174.

[48]ZhangXH, ZhuQX, HeYL, et al., 2018. A novel robust ensemble model integrated extreme learning machine with multi-activation functions for energy modeling and analysis: application to petrochemical industry. Energy, 162:593-602.