Journal of Zhejiang University SCIENCE  A

Accepted manuscript available online (unedited version)


An optimization scheduling strategy for electric-heat-hydrogen integrated energy systems based on memory-enhanced deep reinforcement learning


Author(s):  Zhongli BAI1, 2, Qiang GAO1, 2, 3, Hongzhi ZHANG1, 2, Junjie LIU1, 2, Yuehui JI1, 2, Yu SONG1, 2, Xu CHENG1, 4

Affiliation(s):  1Tianjin Key Laboratory of New Energy Power Conversion, Transmission, and Intelligent Control, Tianjin University of Technology, Tianjin 300384, China 2School of Electrical Engineering and Automation, Tianjin University of Technology, Tianjin 300384, China 3Maritime College, Tianjin University of Technology, Tianjin 300384, China 4School of Computer Science and Engineering, Tianjin University of Technology, Tianjin 300384, China

Corresponding email(s):  Yu SONG, jasonsongrain@hotmail.com Xu CHENG xu.cheng@ieee.org

Key Words:  Integrated energy system; Electric-heat-hydrogen; LSTM-SAC; Reinforcement learning; Scheduling strategy


Share this article to： More <<< Previous Paper|Next Paper >>>

Zhongli BAI1,2, Qiang GAO1,2,3, Hongzhi ZHANG1,2, Junjie LIU1,2, Yuehui JI1,2, Yu SONG1,2, Xu CHENG1,4. An optimization scheduling strategy for electric-heat-hydrogen integrated energy systems based on memory-enhanced deep reinforcement learning[J]. Journal of Zhejiang University Science A,in press.Frontiers of Information Technology & Electronic Engineering,in press.https://doi.org/10.1631/jzus.A2500598

@article{title="An optimization scheduling strategy for electric-heat-hydrogen integrated energy systems based on memory-enhanced deep reinforcement learning",
author="Zhongli BAI1,2, Qiang GAO1,2,3, Hongzhi ZHANG1,2, Junjie LIU1,2, Yuehui JI1,2, Yu SONG1,2, Xu CHENG1,4",
journal="Journal of Zhejiang University Science A",
year="in press",
publisher="Zhejiang University Press & Springer",
doi="https://doi.org/10.1631/jzus.A2500598"
}

%0 Journal Article
%T An optimization scheduling strategy for electric-heat-hydrogen integrated energy systems based on memory-enhanced deep reinforcement learning
%A Zhongli BAI1
%A 2
%A Qiang GAO1
%A 2
%A 3
%A Hongzhi ZHANG1
%A 2
%A Junjie LIU1
%A 2
%A Yuehui JI1
%A 2
%A Yu SONG1
%A 2
%A Xu CHENG1
%A 4
%J Journal of Zhejiang University SCIENCE A
%P
%@ 1673-565X
%D in press
%I Zhejiang University Press & Springer
doi="https://doi.org/10.1631/jzus.A2500598"

TY - JOUR
T1 - An optimization scheduling strategy for electric-heat-hydrogen integrated energy systems based on memory-enhanced deep reinforcement learning
A1 - Zhongli BAI1
A1 - 2
A1 - Qiang GAO1
A1 - 2
A1 - 3
A1 - Hongzhi ZHANG1
A1 - 2
A1 - Junjie LIU1
A1 - 2
A1 - Yuehui JI1
A1 - 2
A1 - Yu SONG1
A1 - 2
A1 - Xu CHENG1
A1 - 4
J0 - Journal of Zhejiang University Science A
SP -
EP -
%@ 1673-565X
Y1 - in press
PB - Zhejiang University Press & Springer
ER -
doi="https://doi.org/10.1631/jzus.A2500598"


Abstract: High wind-solar penetration drives integrated energy systems (IES) to act as cross-vector buffers that absorb surplus electricity via hybrid storage, or convert it into heat and hydrogen for cross-medium energy peak shaving. However, conventional mixed-integer linear programming (MILP) solvers and static incentive schemes often struggle with the high-dimensional, strongly coupled, non-convex, and time-varying nature of electric-heat-hydrogen scheduling. Thus, we propose an end-to-end optimization framework for a renewable electric-heat-hydrogen IES, and evaluate it through offline dispatch computations and cost settlement on a 24-hour simulated case study. The proposed framework incorporates a coupled power-state of charge (SOC) dual penalty mechanism to ensure consistent storage operation over time. Additionally, it includes a bidirectional incentive-based demand response (B-IDR) mapping that is differentiable and can capture asymmetric feedback in response to price fluctuations. Furthermore, a long-term short-term memory (LSTM)-augmented maximum-entropy Soft Actor-Critic (SAC) scheduler is utilized for stable and efficient control in this continuous and high-dimensional setting. Comparisons with other methods under identical settings show that the proposed method achieves lower operating costs and carbon emissions, as well as improved training stability.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Full Text:  <295>

CLC number: 

On-line Access: 2026-03-03

Received: 2025-11-14

Revision Accepted: 2026-01-28

Crosschecked: 0000-00-00

Cited: 0

Clicked: 389

Citations:  Bibtex RefMan EndNote GB/T7714

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2026 Journal of Zhejiang University-SCIENCE