Full Text:   <130>

CLC number: TP183; TN919.72

On-line Access: 2020-09-09

Received: 2019-08-07

Revision Accepted: 2019-12-08

Crosschecked: 2020-08-05

Cited: 0

Clicked: 218

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Yi-ning Chen

https://orcid.org/0000-0002-3435-2851

Guang-hua Song

https://orcid.org/0000-0003-3330-4978

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2020 Vol.21 No.9 P.1308-1320

http://doi.org/10.1631/FITEE.1900401


A traffic-aware Q-network enhanced routing protocol based on GPSR for unmanned aerial vehicle ad-hoc networks


Author(s):  Yi-ning Chen, Ni-qi Lyu, Guang-hua Song, Bo-wei Yang, Xiao-hong Jiang

Affiliation(s):  School of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310027, China; more

Corresponding email(s):   ch19930611@zju.edu.cn, lvniqi@gmail.com, ghsong@zju.edu.cn, boweiy@zju.edu.cn, jiangxh@zju.edu.cn

Key Words:  Traffic balancing, Reinforcement learning, Geographic routing, Q-network


Yi-ning Chen, Ni-qi Lyu, Guang-hua Song, Bo-wei Yang, Xiao-hong Jiang. A traffic-aware Q-network enhanced routing protocol based on GPSR for unmanned aerial vehicle ad-hoc networks[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(9): 1308-1320.

@article{title="A traffic-aware Q-network enhanced routing protocol based on GPSR for unmanned aerial vehicle ad-hoc networks",
author="Yi-ning Chen, Ni-qi Lyu, Guang-hua Song, Bo-wei Yang, Xiao-hong Jiang",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="21",
number="9",
pages="1308-1320",
year="2020",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.1900401"
}

%0 Journal Article
%T A traffic-aware Q-network enhanced routing protocol based on GPSR for unmanned aerial vehicle ad-hoc networks
%A Yi-ning Chen
%A Ni-qi Lyu
%A Guang-hua Song
%A Bo-wei Yang
%A Xiao-hong Jiang
%J Frontiers of Information Technology & Electronic Engineering
%V 21
%N 9
%P 1308-1320
%@ 2095-9184
%D 2020
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.1900401

TY - JOUR
T1 - A traffic-aware Q-network enhanced routing protocol based on GPSR for unmanned aerial vehicle ad-hoc networks
A1 - Yi-ning Chen
A1 - Ni-qi Lyu
A1 - Guang-hua Song
A1 - Bo-wei Yang
A1 - Xiao-hong Jiang
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 21
IS - 9
SP - 1308
EP - 1320
%@ 2095-9184
Y1 - 2020
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.1900401


Abstract: 
In dense traffic unmanned aerial vehicle (UAV) ad-hoc networks, traffic congestion can cause increased delay and packet loss, which limit the performance of the networks; therefore, a traffic balancing strategy is required to control the traffic. In this study, we propose TQNGPSR, a traffic-aware q-network enhanced geographic routing protocol based on greedy perimeter stateless routing (GPSR), for UAV ad-hoc networks. The protocol enforces a traffic balancing strategy using the congestion information of neighbors, and evaluates the quality of a wireless link by the q-network algorithm, which is a reinforcement learning algorithm. Based on the evaluation of each wireless link, the protocol makes routing decisions in multiple available choices to reduce delay and decrease packet loss. We simulate the performance of TQNGPSR and compare it with AODV, OLSR, GPSR, and QNGPSR. Simulation results show that TQNGPSR obtains higher packet delivery ratios and lower end-to-end delays than GPSR and QNGPSR. In high node density scenarios, it also outperforms AODV and OLSR in terms of the packet delivery ratio, end-to-end delay, and throughput.

基于GPSR和Q网络的流量感知无人机ad-hoc网络路由协议

陈弈宁1,吕倪祺1,宋广华1,杨波威1,姜晓红2
1浙江大学航空航天学院,中国杭州市,310027
2浙江大学计算机科学与技术学院,中国杭州市,310027

摘要:在流量密集的无人机ad-hoc网络中,流量拥塞会增加网络时延和丢包,大大限制网络性能。因此,需要一个流量平衡策略控制流量。本文提出TQNGPSR,一个基于GPSR和Q网络的流量感知无人机ad-hoc网络路由协议。该协议利用邻居节点的拥塞信息实现流量平衡,并用一种强化学习算法-Q网络算法-评价当前节点每条无线链接的质量。基于对这些链接的评估,该协议可在多个选择中做出合理决定,降低网络时延和丢包率。在仿真环境中测试TQNGPSR、AODV、OLSR、GPSR和QNGPSR。结果表明,相比于GPSR和QNGPSR,TQNGPSR有更高包到达率和更低端到端时延。在高节点密度场景中,TQNGPSR在包到达率、端到端时延和吞吐量上优于AODV和OLSR。

关键词:流量平衡;强化学习;地理信息路由;Q网络

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Abadi M, Barham P, Chen JM, et al., 2016. TensorFlow: a system for large-scale machine learning. Proc 12th USENIX Conf on Operating Systems Design and Implementation, p.265-283.

[2]Basagni S, Chlamtac I, Syrotiuk VR, et al., 1998. A distance routing effect algorithm for mobility (DREAM). Proc 4th Annual ACM/IEEE Int Conf on Mobile Computing and Networking, p.76-84.

[3]Bekmezci I, Sahingoz OK, Temel Ş, 2013. Flying ad-hoc networks (FANETs): a survey. Ad Hoc Netw, 11(3):1254-1270.

[4]Bolch G, Greiner S, de Meer H, et al., 2006. Queueing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications (2nd Ed.). John Wiley & Sons, New York, USA.

[5]Boyan JA, Littman ML, 1994. Packet routing in dynamically changing networks: a reinforcement learning approach. Proc 7th Int Conf on Neural Information Processing Systems, p.671-678.

[6]Coutinho N, Matos R, Marques C, et al., 2015. Dynamic dual-reinforcement-learning routing strategies for quality of experience-aware wireless mesh networking. Comput Netw, 88:269-285.

[7]Farahnakian F, Ebrahimi M, Daneshtalab M, et al., 2011. Q-learning based congestion-aware routing algorithm for on-chip network. Proc 2nd Int Conf on Networked Embedded Systems for Enterprise Applications, p.1-7.

[8]Jung WS, Yim J, Ko YB, 2017. QGeo: Q-learning-based geographic ad hoc routing protocol for unmanned robotic networks. IEEE Commun Lett, 21(10):2258-2261.

[9]Karp B, Kung HT, 2000. GPSR: greedy perimeter stateless routing for wireless networks. Proc 6th Annual Int Conf on Mobile Computing and Networking, p.243-254.

[10]Kenta T, Takeshi M, Shinya K, et al., 2006. Experimental evaluation of an on-demand multipath routing protocol for video transmission in mobile ad hoc networks. J Zhejiang Univ-Sci A, 7(S1):145-150.

[11]Ko YB, Vaidya NH, 2000. Location-aided routing (LAR) in mobile ad hoc networks. Wirel Netw, 6(4):307-321.

[12]Li RL, Li F, Li X, et al., 2014. QGrid: Q-learning based routing protocol for vehicular ad hoc networks. Proc 33rd Int Performance Computing and Communications Conf, p.1-8.

[13]Lin SC, Wang P, Luo M, 2016. Control traffic balancing in software defined networks. Comput Netw, 106:260-271.

[14]Lyu N, Song GH, Yang BW, et al., 2019. QNGPSR: a Q-network enhanced geographic ad-hoc routing protocol based on GPSR. Proc 88th Vehicular Technology Conf, p.1-6.

[15]Ma X, Xu Y, Sun GQ, et al., 2013. State-chain sequential feedback reinforcement learning for path planning of autonomous mobile robots. J Zhejiang Univ-Sci C (Comput & Electron), 14(3):167-178.

[16]Mnih V, Kavukcuoglu K, Silver D, et al., 2015. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533.

[17]Peng J, Williams RJ, 1996. Incremental multi-step Q-learning. Mach Learn, 22(1-3):283-290.

[18]Shi RH, Deng YY, 2008. An improved scheme for reducing the latency of AODV in ad hoc networks. Proc 9th Int Conf for Young Computer Scientists, p.594-598.

[19]Sutton RS, Barto AG, 1998. Reinforcement Learning: an Introduction. MIT Press, Cambridge, p.1.

[20]Watkins CJCH, Dayan P, 1992. Q-learning. Mach Learn, 8(3-4):279-292.

[21]Wu C, Ohzahata S, Kato T, 2012. Routing in VANETs: a fuzzy constraint Q-learning approach. Proc Global Communications Conf, p.195-200.

[22]Wu C, Ohzahata S, Kato T, 2013. Flexible, portable, and practicable solution for routing in VANETs: a fuzzy constraint Q-learning approach. IEEE Trans Veh Technol, 62(9):4251-4263.

[23]Xu DH, Chiang M, Rexford J, 2011. Link-state routing with hop-by-hop forwarding can achieve optimal traffic engineering. IEEE/ACM Trans Netw, 19(6):1717-1730.

[24]Zhan HW, Zhou Y, 2008. Comparison and analysis AODV and OLSR routing protocols in ad hoc network. Proc 4th Int Conf on Wireless Communications, Networking and Mobile Computing, p.1-4.

[25]Zhang JJ, Xi K, Luo M, et al., 2014. Load balancing for multiple traffic matrices using SDN hybrid routing. Proc 15th Int Conf on High Performance Switching and Routing, p.44-49.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - Journal of Zhejiang University-SCIENCE