Full Text:   <1549>

Summary:  <467>

CLC number: TP273.1

On-line Access: 2014-01-07

Received: 2013-05-27

Revision Accepted: 2013-11-26

Crosschecked: 2013-12-19

Cited: 1

Clicked: 3357

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
1. Reference List
Open peer comments

Journal of Zhejiang University SCIENCE C 2014 Vol.15 No.1 P.43-50


Adaptive dynamic programming for linear impulse systems

Author(s):  Xiao-hua Wang, Juan-juan Yu, Yao Huang, Hua Wang, Zhong-hua Miao

Affiliation(s):  School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200072, China; more

Corresponding email(s):   x.wang@shu.edu.cn, zhhmiao@shu.edu.cn

Key Words:  Adaptive dynamic programming (ADP), Impulse system, Optimal control, Neural network

Xiao-hua Wang, Juan-juan Yu, Yao Huang, Hua Wang, Zhong-hua Miao. Adaptive dynamic programming for linear impulse systems[J]. Journal of Zhejiang University Science C, 2014, 15(1): 43-50.

@article{title="Adaptive dynamic programming for linear impulse systems",
author="Xiao-hua Wang, Juan-juan Yu, Yao Huang, Hua Wang, Zhong-hua Miao",
journal="Journal of Zhejiang University Science C",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Adaptive dynamic programming for linear impulse systems
%A Xiao-hua Wang
%A Juan-juan Yu
%A Yao Huang
%A Hua Wang
%A Zhong-hua Miao
%J Journal of Zhejiang University SCIENCE C
%V 15
%N 1
%P 43-50
%@ 1869-1951
%D 2014
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.C1300145

T1 - Adaptive dynamic programming for linear impulse systems
A1 - Xiao-hua Wang
A1 - Juan-juan Yu
A1 - Yao Huang
A1 - Hua Wang
A1 - Zhong-hua Miao
J0 - Journal of Zhejiang University Science C
VL - 15
IS - 1
SP - 43
EP - 50
%@ 1869-1951
Y1 - 2014
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.C1300145

We investigate the optimization of linear impulse systems with the reinforcement learning based adaptive dynamic programming (ADP) method. For linear impulse systems, the optimal objective function is shown to be a quadric form of the pre-impulse states. The ADP method provides solutions that iteratively converge to the optimal objective function. If an initial guess of the pre-impulse objective function is selected as a quadratic form of the pre-impulse states, the objective function iteratively converges to the optimal one through ADP. Though direct use of the quadratic objective function of the states within the ADP method is theoretically possible, the numerical singularity problem may occur due to the matrix inversion therein when the system dimensionality increases. A neural network based ADP method can circumvent this problem. A neural network with polynomial activation functions is selected to approximate the pre-impulse objective function and trained iteratively using the ADP method to achieve optimal control. After a successful training, optimal impulse control can be derived. Simulations are presented for illustrative purposes.


创新要点:目前自适应动态规划方法研究局限于连续和离散系统,对脉冲系统研究较少。 本文研究了线性脉冲系统的最优控制问题,运用自适应动态规划思路,完成了脉冲系统相关理论证明,证实了方法的收敛性。通过神经网络逼近最优目标函数,当迭代稳定后,神经网络获得稳定参数,同时获得最优脉冲控制率。


Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1]Ahmed, N.U., 2003. Existence of optimal controls for a general class of impulsive systems on Banach spaces. SIAM J. Control Optim., 42(2):669-685.

[2]Bainov, D.D., Simeonov, P.S., 1995. Impulsive Differential Equations: Asymptotic Properties of the Solutions. World Scientific, Singapore.

[3]Balakrishnan, S.N., Ding, J., Lewis, F.L., 2008. Issues on stability of ADP feedback controllers for dynamical systems. IEEE Tran. Syst. Man Cybern. B, 38(4):913-917.

[4]Bertsekas, D.P., 2011. Approximate policy iteration: a survey and some new methods. J. Control Theory Appl., 9(3):310-335.

[5]Dierks, T., Jagannathan, S., 2011. Online optimal control of nonlinear discrete-time systems using approximate dynamic programming. J. Control Theory Appl., 9(3):361-369.

[6]Fraga, S.L., Pereira, F.L., 2012. Hamilton-Jacobi-Bellman equation and feedback synthesis for impulsive control. IEEE Trans. Automat. Control, 57(1):244-249.

[7]Jiang, Y., Jiang, Z.P., 2012. Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica, 48(10):2699-2704.

[8]Jiang, Z.P., Jiang, Y., 2013. Robust adaptive dynamic programming for linear and nonlinear systems: an overview. Eur. J. Control, 19(5):417-425.

[9]Kurzhanski, A.B., Daryin, A.N., 2008. Dynamic programming for impulse controls. Ann. Rev. Control, 32(2):213-227.

[10]Lakshmikantham, V., Bainov, D.D., Simeonov, P.S., 1989. Theory of Impulsive Differential Equations. World Scientific, Singapore.

[11]Lewis, F.L., Vrabie, D., 2009. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circ. Syst. Mag., 9(3):32-50.

[12]Liu, B., Teo, K.L., Liu, X.Z., 2008. Optimal control and robust stability of uncertain impulsive dynamical systems. Asian J. Control, 10(3):314-326.

[13]Liu, D.R., Wei, Q.L., 2013. Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems. IEEE Trans. Cybern., 43(2):779-789.

[14]Liu, X., 1995. Impulsive control and optimization. Appl. Math. Comput., 73(1):77-98.

[15]Silva, G.N., Vinter, R.B., 1997. Necessary conditions for optimal impulsive control problems. SIAM J. Control Optim., 35(6):1829-1846.

[16]Vamvoudakis, K.G., Lewis, F.L., 2010. Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica, 46(5):878-888.

[17]Wang, F.Y., Zhang, H.G., Liu, D.R., 2009. Adaptive dynamic programming: an introduction. IEEE Comput. Intell. Mag., 4(2):39-47.

[18]Wang, J.R., Yang, Y.L., 2010. Optimal control of linear impulsive antiperiodic boundary value problem on infinite dimensional spaces. Discr. Dynam. Nat. Soc., Article ID 673013.

[19]Wang, X.H., 2008. Optimal Control of Impulsive Systems Using Adaptive Critic Neural Network. PhD Thesis, Missouri University of Science and Technology, Rolla, Missouri, USA.

[20]Wang, X.H., Balakrishnan, S.N., 2010. Optimal neurocontroller synthesis for impulse-driven systems. Neur. Networks, 23(1):125-134.

[21]Wang, X.H., Luo, W.Z., Balakrishnan, S.N., 2012.

[22]Linear impulsive system optimization using adaptive dynamic programming. 12th Int. Conf. on Control Automation Robotics and Vision, p.725-730.

[23]Werbos, P.J., 1974. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD Thesis, Harvard University, USA.

[24]Werbos, P.J., 2008. Foreword-ADP: the key direction for future research in intelligent control and understanding brain intelligence. IEEE Trans. Syst. Man Cybern. B, 38(4):898-900.

[25]Werbos, P.J., McAvoy, T., Su, T., 1992. Handbook of Intelligent Control. Van Nostrand Reinhold, New York.

[26]Wu, Z., Zhang, F., 2011. Stochastic maximum principle for optimal control problems of forward-backward systems involving impulse controls. IEEE Trans. Automat. Control, 56(6):1401-1406.

[27]Yang, T., 1999. Impulsive control. IEEE Trans. Automat. Control, 44(5):1081-1083.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - Journal of Zhejiang University-SCIENCE