Abstract: Hydraulic legged robots have potential for high-dynamic motion due to their large power-to-weight ratios. However, it is challenging to ensure both stability and continuity in the motion of such robots. In this study, we propose a jumping motion control framework based on deep reinforcement learning that enables hydraulic limb leg units to perform stable and continuous jumping motions. First, to accurately represent the performance of a physical prototype, a quasi-realistic model incorporating physical feasibility constraints is constructed. This model is informed by analysis of the relevant fluid dynamics, and incorporates a trajectory generator and a motion tracking controller. To achieve stable and continuous jumping performance, a deep reinforcement learning algorithm is developed, which jointly optimizes the trajectory generator and the motion tracking controller. Through validation on the physical prototype, we demonstrate that the proposed method reduces the maximum deviation and the average deviation by over 47% and 60%, respectively, and improves landing compliance by up to 7.7% compared to a baseline optimization algorithm, the non-dominated sorting genetic algorithm (NSGA-II). The proposed control framework may serve as a reference for high-dynamic motion control of legged robots and multi-objective optimization across several decision variables.

基于强化学习的液压足式机器人稳定连续的垂直跳跃控制方法

[1]AhnD, ChoBK, 2022. Online jumping motion generation via model predictive control. IEEE Transactions on Industrial Electronics, 69(5):4957-4965.

[2]BaKX, ChenCH, MaGL, et al., 2024. A compensation strategy of end-effector pose precision based on the virtual constraints for serial robots with RDOFs. Fundamental Research, in press.

[3]BaKX, SheJB, XuB, et al., 2025. Matrix sensitivity-based adaptive iterative feedback control of leg hydraulic drive system of legged robot. Control Engineering Practice, 165:106557.

[4]BjelonicM, SankarPK, BellicosoCD, et al., 2020. Rolling in the deep-hybrid locomotion for wheeled-legged robots using online trajectory optimization. IEEE Robotics and Automation Letters, 5(2):3626-3633.

[5]BoaventuraT, Medrano-CerdaGA, SeminiC, et al., 2013. Stability and performance of the compliance controller of the quadruped robot HyQ. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, p.1458-1464.

[6]ChoiS, JiG, ParkJ, et al., 2023. Learning quadrupedal locomotion on deformable terrain. Science Robotics, 8(74):eade2256.

[7]DuanP, YuZN, GaoKZ, et al., 2024. Solving the multi-objective path planning problem for mobile robot using an improved NSGA-II algorithm. Swarm and Evolutionary Computation, 87:101576.

[8]EgliP, HutterM, 2022. A general approach for the automation of hydraulic excavator arms using reinforcement learning. IEEE Robotics and Automation Letters, 7(2):5679-5686.

[9]ExarchosI, JiangYF, YuWH, et al., 2021. Policy transfer via kinematic domain randomization and adaptation. Proceedings of the IEEE International Conference on Robotics and Automation, p.45-51.

[10]FangLZ, ZhangK, LuZY, et al., 2025. Adaptive robust joint force control for rapid motion of hydraulic limb leg units. Control Engineering Practice, 165:106596.

[11]GaoHB, LiuYF, DingL, et al., 2019. Low impact force and energy consumption motion planning for hexapod robot with passive compliant ankles. Journal of Intelligent & Robotic Systems, 94(2):349-370.

[12]GuY, YuanCZ, 2020. Adaptive robust trajectory tracking control of fully actuated bipedal robotic walking. Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics, p.1310-1315.

[13]HanL, ZhuQX, ShengJP, et al., 2024. Lifelike agility and play in quadrupedal robots using reinforcement learning and generative pre-trained models. Nature Machine Intelligence, 6(7):787-798.

[14]HanYY, LiuGP, LuZY, et al., 2023. A stability locomotion-control strategy for quadruped robots with center-of-mass dynamic planning. Journal of Zhejiang University-SCIENCE A, 24(6):516-530.

[15]HoellerD, RudinN, SakoD, et al., 2024. Anymal parkour: learning agile navigation for quadrupedal robots. Science Robotics, 9(88):eadi7566.

[16]HwangboJ, LeeJ, DosovitskiyA, et al., 2019. Learning agile and dynamic motor skills for legged robots. Science Robotics, 4(26):eaau5872.

[17]JaiswalS, SopanenJ, MikkolaA, 2021. Efficiency comparison of various friction models of a hydraulic cylinder in the framework of multibody system dynamics. Nonlinear Dynamics, 104(4):3497-3515.

[18]LiX, YuHY, ZongHZ, et al., 2024. Light weight design and integrated method for manufacturing hydraulic wheel-legged robots. Journal of Zhejiang University-SCIENCE A, 25(9):701-715.

[19]MaHP, ZhangYJ, SunSY, et al., 2023. A comprehensive survey on NSGA-II for multi-objective optimization and applications. Artificial Intelligence Review, 56(12):15217-15270.

[20]NansaiS, RojasN, ElaraMR, et al., 2015. A novel approach to gait synchronization and transition for reconfigurable walking platforms. Digital Communications and Networks, 1(2):141-151.

[21]SchulmanJ, WolskiF, DhariwalP, et al., 2017. Proximal policy optimization algorithms. arXiv:1707.06347.

[22]SeminiC, BarasuolV, GoldsmithJ, et al., 2017. Design of the hydraulically actuated, torque-controlled quadruped robot HyQ2Max. IEEE/ASME Transactions on Mechatronics, 22(2):635-646.

[23]ShaoX, FanYQ, ShaoJP, et al., 2023. Improved active disturbance rejection control with the optimization algorithm for the leg joint control of a hydraulic quadruped robot. Measurement and Control, 56(7-8):1359-1376.

[24]SpinelliFA, EgliP, NubertJ, et al., 2024. Reinforcement learning control for autonomous hydraulic material handling machines with underactuated tools. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, p.12694-12701.

[25]SunYR, HuaZS, LiYB, et al., 2021. Modeling and analysis on low energy consumption foot trajectory for hydraulic actuated quadruped robot. International Journal of Advanced Robotic Systems, 2021(11):1-12.

[26]XiangPJ, YanL, LiuXS, et al., 2025. Structural topology design for electromagnetic performance enhancement of permanent-magnet machines. Chinese Journal of Mechanical Engineering, 38(1):26.

[27]XieZM, DaXY, van de PanneM, et al., 2021. Dynamics randomization revisited: a case study for quadrupedal locomotion. Proceedings of the IEEE International Conference on Robotics and Automation, p.4955-4961.

[28]YaoB, 2009. Desired compensation adaptive robust control. Journal of Dynamic Systems, Measurement, and Control, 131(6):061001.

[29]YaoZK, XuFY, JiangGP, et al., 2024. Data-driven control of hydraulic manipulators by reinforcement learning. IEEE/ASME Transactions on Mechatronics, 29(4):2673-2684.

[30]ZhangJH, LiuJY, ZongHZ, et al., 2025. Bridging the gap to bionic motion: challenges in legged robot limb unit design, modeling, and control. Cyborg and Bionic Systems, 6:0365.

[31]ZhangK, ZhangJH, ZongHZ, et al., 2025. High dynamic position control for a typical hydraulic quadruped robot leg based on virtual decomposition control. IEEE/ASME Transactions on Mechatronics, 30(4):2473-2484.

[32]ZhuJ, PayneJJ, JohnsonAM, 2024. Convergent iLQR for safe trajectory planning and control of legged robots. Proceedings of the IEEE International Conference on Robotics and Automation, p.8051-8057.

[33]ZongHZ, ZhangJH, JiangL, et al., 2024. Bionic lightweight design of limb leg units for hydraulic quadruped robots by additive manufacturing and topology optimization. Bio-Design and Manufacturing, 7(1):1-13.

[34]ZongHZ, LouB, YuanHH, et al., 2025. Integrating kinematic and dynamic factors with generative design for high-performance additive manufacturing structures. Virtual and Physical Prototyping, 20(1):e2501383.