Abstract: With the reduction in manufacturing and launch costs of low Earth orbit satellites and the advantages of large coverage and high data transmission rates, satellites have become an important part of data transmission in air-ground networks. However, due to the factors such as geographical location and people’s living habits, the differences in user’ demand for multimedia data will result in unbalanced network traffic, which may lead to network congestion and affect data transmission. In addition, in traditional satellite network transmission, the convergence of network information acquisition is slow and global network information cannot be collected in a fine-grained manner, which is not conducive to calculating optimal routes. The service quality requirements cannot be satisfied when multiple service requests are made. Based on the above, in this paper artificial intelligence technology is applied to the satellite network, and a software-defined network is used to obtain the global network information, perceive network traffic, develop comprehensive decisions online through reinforcement learning, and update the optimal routing strategy in real time. Simulation results show that the proposed reinforcement learning algorithm has good convergence performance and strong generalizability. Compared with traditional routing, the throughput is 8% higher, and the proposed method has load balancing characteristics.

一种基于业务需求与流量感知的卫星网络多路径路由算法

[1]Arfeen A, Uddin R, 2020. Quality of experience-based optimization of satellite Internet-at-sea using WAN accelerators. Int J Satell Commun Netw, 38(6):527-556.

[2]Bujari A, Luglio M, Palazzi CE, et al., 2020. A virtual PEP for web optimization over a satellite-terrestrial backhaul. IEEE Commun Mag, 58(10):42-48.

[3]Chen Q, Yang L, Guo DK, et al., 2022. LEO satellite networks: when do all shortest distance paths belong to minimum hop path set. IEEE Trans Aerosp Electron Syst, 58(4):3730-3734.

[4]Gao K, Xu CQ, Qin JR, et al., 2019. QoS-driven path selection for MPTCP: a scalable SDN-assisted approach. Proc IEEE Wireless Communications and Networking Conf, p.1-6.

[5]Han C, Huo LY, Tong XH, et al., 2020. Spatial anti-jamming scheme for Internet of satellites based on the deep reinforcement learning and Stackelberg game. IEEE Trans Veh Technol, 69(5):5331-5342.

[6]Huo LW, Jiang DD, Zhu XN, et al., 2022. A SDN-based fine-grained measurement and modeling approach to vehicular communication network traffic. Int J Commun Syst, 35(12):e4092.

[7]Jia ZY, Sheng M, Li JD, et al., 2020. LEO-satellite-assisted UAV: joint trajectory and data collection for Internet of Remote Things in 6G aerial access networks. IEEE Int Things J, 8(12):9814-9826.

[8]Kuhn N, Michel F, Thomas L, et al., 2020. QUIC: opportunities and threats in SATCOM. Proc 10^th Advanced Satellite Multimedia Systems Conf and the 16^th Signal Processing for Space Communications Workshop, p.1-7.

[9]Langley A, Riddoch A, Wilk A, et al., 2017. The QUIC transport protocol: design and Internet-scale deployment. Proc Conf of the ACM Special Interest Group on Data Communication, p.183-196.

[10]Li X, Tang FL, Zhu YM, et al., 2022. Processing-while-transmitting: cost-minimized transmission in SDN-based STINs. IEEE/ACM Trans Netw, 30(1):243-256.

[11]Liu D, Zhang JK, Cui JJ, et al., 2022. Deep learning aided routing for space-air-ground integrated networks relying on real satellite, flight, and shipping data. IEEE Wirel Commun, 29(2):177-184.

[12]Liu JH, Zhao BK, Xin Q, et al., 2021. DRL-ER: an intelligent energy-aware routing protocol with guaranteed delay bounds in satellite mega-constellations. IEEE Trans Netw Sci Eng, 8(4):2872-2884.

[13]Liu LT, Shen YL, Zeng SG, et al., 2021. FO-Sketch: a fast oblivious sketch for secure network measurement service in the cloud. Electronics, 10(16):2020.

[14]Liu ZG, Zhu J, Zhang JM, et al., 2020. Routing algorithm design of satellite network architecture based on SDN and ICN. Int J Satell Commun Netw, 38(1):1-15.

[15]Mogensen RS, Markmoller C, Madsen TK, et al., 2019. Selective redundant MP-QUIC for 5G mission critical wireless applications. Proc IEEE 89^th Vehicular Technology Conf, p.1-5.

[16]Murua J, Reviriego P, 2020. Faking elephant flows on the count min sketch. IEEE Netw Lett, 2(4):199-202.

[17]Oroojlooyjadid A, Nazari M, Snyder LV, et al., 2022. A deep Q-network for the beer game: deep reinforcement learning for inventory optimization. Manuf Ser Oper Manag, 24(1):285-304.

[18]Rabitsch A, Hurtig P, Brunstrom A, 2018. A stream-aware multipath QUIC scheduler for heterogeneous paths. Proc Workshop on the Evolution, Performance, and Interoperability of QUIC, p.29-35.

[19]Shi H, Zhang L, Zuo XT, et al., 2021. Multipath deadline-aware transport proxy for space network. IEEE Int Comput, 25(6):51-57.

[20]Tang L, Huang Q, Lee PPC, 2019. MV-Sketch: a fast and compact invertible sketch for heavy flow detection in network data streams. Proc IEEE INFOCOM Conf on Computer Communications, p.2026-2034.

[21]Wang F, Jiang DD, Qi S, et al., 2021. An Adaboost based link planning scheme in space-air-ground integrated networks. Mob Netw Appl, 26(2):669-680.

[22]Wu Q, Chen X, Zhou Z, et al., 2021. Deep reinforcement learning with spatio-temporal traffic forecasting for data-driven base station sleep control. IEEE/ACM Trans Netw, 29(2):935-948.

[23]Xu JP, Ai B, 2021. Deep reinforcement learning for handover-aware MPTCP congestion control in space-ground integrated network of railways. IEEE Wirel Commun, 28(6):200-207.

[24]Ya D, Bin Q, Wei N, 2021. DW-Sketch: a sketch-based scheme for realizing multi-network measurement tasks. Proc 2^nd Int Conf on Computer Communication and Network Security, p.191-195.

[25]Yang SY, Li HW, Wu Q, 2018. Performance analysis of QUIC protocol in integrated satellites and terrestrial networks. Proc 14^th Int Wireless Communications & Mobile Computing Conf, p.1425-1430.

[26]Yang WJ, Shu SJ, Cai L, et al., 2021. MM-QUIC: mobility-aware multipath QUIC for satellite networks. Proc 17^th Int Conf on Mobility, Sensing and Networking, p.608-615.

[27]Yu ML, Jose L, Miao R, 2013. Software defined traffic measurement with OpenSketch. Proc 10^th USENIX Conf on Networked Systems Design and Implementation, p.29-42.

[28]Zhang R, Liu J, Yang D, et al., 2020. A survey on satellite networks based on software-defined networking. Front Data Comput, 2(3):3-17.