Full Text:   <1549>

Summary:  <467>

CLC number: TP273.1

On-line Access: 2014-01-07

Received: 2013-05-27

Revision Accepted: 2013-11-26

Crosschecked: 2013-12-19

Cited: 1

Clicked: 3357

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
1. Reference List
Open peer comments

Journal of Zhejiang University SCIENCE C 2014 Vol.15 No.1 P.43-50

http://doi.org/10.1631/jzus.C1300145


Adaptive dynamic programming for linear impulse systems


Author(s):  Xiao-hua Wang, Juan-juan Yu, Yao Huang, Hua Wang, Zhong-hua Miao

Affiliation(s):  School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200072, China; more

Corresponding email(s):   x.wang@shu.edu.cn, zhhmiao@shu.edu.cn

Key Words:  Adaptive dynamic programming (ADP), Impulse system, Optimal control, Neural network


Xiao-hua Wang, Juan-juan Yu, Yao Huang, Hua Wang, Zhong-hua Miao. Adaptive dynamic programming for linear impulse systems[J]. Journal of Zhejiang University Science C, 2014, 15(1): 43-50.

@article{title="Adaptive dynamic programming for linear impulse systems",
author="Xiao-hua Wang, Juan-juan Yu, Yao Huang, Hua Wang, Zhong-hua Miao",
journal="Journal of Zhejiang University Science C",
volume="15",
number="1",
pages="43-50",
year="2014",
publisher="Zhejiang University Press & Springer",
doi="10.1631/jzus.C1300145"
}

%0 Journal Article
%T Adaptive dynamic programming for linear impulse systems
%A Xiao-hua Wang
%A Juan-juan Yu
%A Yao Huang
%A Hua Wang
%A Zhong-hua Miao
%J Journal of Zhejiang University SCIENCE C
%V 15
%N 1
%P 43-50
%@ 1869-1951
%D 2014
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.C1300145

TY - JOUR
T1 - Adaptive dynamic programming for linear impulse systems
A1 - Xiao-hua Wang
A1 - Juan-juan Yu
A1 - Yao Huang
A1 - Hua Wang
A1 - Zhong-hua Miao
J0 - Journal of Zhejiang University Science C
VL - 15
IS - 1
SP - 43
EP - 50
%@ 1869-1951
Y1 - 2014
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.C1300145


Abstract: 
We investigate the optimization of linear impulse systems with the reinforcement learning based adaptive dynamic programming (ADP) method. For linear impulse systems, the optimal objective function is shown to be a quadric form of the pre-impulse states. The ADP method provides solutions that iteratively converge to the optimal objective function. If an initial guess of the pre-impulse objective function is selected as a quadratic form of the pre-impulse states, the objective function iteratively converges to the optimal one through ADP. Though direct use of the quadratic objective function of the states within the ADP method is theoretically possible, the numerical singularity problem may occur due to the matrix inversion therein when the system dimensionality increases. A neural network based ADP method can circumvent this problem. A neural network with polynomial activation functions is selected to approximate the pre-impulse objective function and trained iteratively using the ADP method to achieve optimal control. After a successful training, optimal impulse control can be derived. Simulations are presented for illustrative purposes.

线性脉冲系统的自适应动态规划方法

研究目的:针对线性脉冲系统最优控制,研究了基于自适应动态规划的递归方法。通过神经网络逼近最优目标函数,得出最优控制率。求解过程适用于一般脉冲系统,无需初始稳定控制器,为此类系统的最优控制提供理论依据。
创新要点:目前自适应动态规划方法研究局限于连续和离散系统,对脉冲系统研究较少。 本文研究了线性脉冲系统的最优控制问题,运用自适应动态规划思路,完成了脉冲系统相关理论证明,证实了方法的收敛性。通过神经网络逼近最优目标函数,当迭代稳定后,神经网络获得稳定参数,同时获得最优脉冲控制率。
方法提亮:线性脉冲系统的最优目标函数是一个状态二次型,但其中的P矩阵表现为跳跃的脉冲形式。基于此发现,以迭代学习为基础的自适应动态规划方法适用于最优脉冲求解。本文提出的方法避免了直接迭代的矩阵求逆,大大降低了运算量。
重要结论:线性脉冲系统的最优目标函数表现为状态二次型,可通过自适应动态规划方法迭代求解,求解过程稳定。通过神经网络逼近最优目标函数,可避免矩阵求逆,大大降低计算量。

关键词:脉冲系统;自适应动态规划;最优控制;神经网络

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Ahmed, N.U., 2003. Existence of optimal controls for a general class of impulsive systems on Banach spaces. SIAM J. Control Optim., 42(2):669-685.

[2]Bainov, D.D., Simeonov, P.S., 1995. Impulsive Differential Equations: Asymptotic Properties of the Solutions. World Scientific, Singapore.

[3]Balakrishnan, S.N., Ding, J., Lewis, F.L., 2008. Issues on stability of ADP feedback controllers for dynamical systems. IEEE Tran. Syst. Man Cybern. B, 38(4):913-917.

[4]Bertsekas, D.P., 2011. Approximate policy iteration: a survey and some new methods. J. Control Theory Appl., 9(3):310-335.

[5]Dierks, T., Jagannathan, S., 2011. Online optimal control of nonlinear discrete-time systems using approximate dynamic programming. J. Control Theory Appl., 9(3):361-369.

[6]Fraga, S.L., Pereira, F.L., 2012. Hamilton-Jacobi-Bellman equation and feedback synthesis for impulsive control. IEEE Trans. Automat. Control, 57(1):244-249.

[7]Jiang, Y., Jiang, Z.P., 2012. Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica, 48(10):2699-2704.

[8]Jiang, Z.P., Jiang, Y., 2013. Robust adaptive dynamic programming for linear and nonlinear systems: an overview. Eur. J. Control, 19(5):417-425.

[9]Kurzhanski, A.B., Daryin, A.N., 2008. Dynamic programming for impulse controls. Ann. Rev. Control, 32(2):213-227.

[10]Lakshmikantham, V., Bainov, D.D., Simeonov, P.S., 1989. Theory of Impulsive Differential Equations. World Scientific, Singapore.

[11]Lewis, F.L., Vrabie, D., 2009. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circ. Syst. Mag., 9(3):32-50.

[12]Liu, B., Teo, K.L., Liu, X.Z., 2008. Optimal control and robust stability of uncertain impulsive dynamical systems. Asian J. Control, 10(3):314-326.

[13]Liu, D.R., Wei, Q.L., 2013. Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems. IEEE Trans. Cybern., 43(2):779-789.

[14]Liu, X., 1995. Impulsive control and optimization. Appl. Math. Comput., 73(1):77-98.

[15]Silva, G.N., Vinter, R.B., 1997. Necessary conditions for optimal impulsive control problems. SIAM J. Control Optim., 35(6):1829-1846.

[16]Vamvoudakis, K.G., Lewis, F.L., 2010. Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica, 46(5):878-888.

[17]Wang, F.Y., Zhang, H.G., Liu, D.R., 2009. Adaptive dynamic programming: an introduction. IEEE Comput. Intell. Mag., 4(2):39-47.

[18]Wang, J.R., Yang, Y.L., 2010. Optimal control of linear impulsive antiperiodic boundary value problem on infinite dimensional spaces. Discr. Dynam. Nat. Soc., Article ID 673013.

[19]Wang, X.H., 2008. Optimal Control of Impulsive Systems Using Adaptive Critic Neural Network. PhD Thesis, Missouri University of Science and Technology, Rolla, Missouri, USA.

[20]Wang, X.H., Balakrishnan, S.N., 2010. Optimal neurocontroller synthesis for impulse-driven systems. Neur. Networks, 23(1):125-134.

[21]Wang, X.H., Luo, W.Z., Balakrishnan, S.N., 2012.

[22]Linear impulsive system optimization using adaptive dynamic programming. 12th Int. Conf. on Control Automation Robotics and Vision, p.725-730.

[23]Werbos, P.J., 1974. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD Thesis, Harvard University, USA.

[24]Werbos, P.J., 2008. Foreword-ADP: the key direction for future research in intelligent control and understanding brain intelligence. IEEE Trans. Syst. Man Cybern. B, 38(4):898-900.

[25]Werbos, P.J., McAvoy, T., Su, T., 1992. Handbook of Intelligent Control. Van Nostrand Reinhold, New York.

[26]Wu, Z., Zhang, F., 2011. Stochastic maximum principle for optimal control problems of forward-backward systems involving impulse controls. IEEE Trans. Automat. Control, 56(6):1401-1406.

[27]Yang, T., 1999. Impulsive control. IEEE Trans. Automat. Control, 44(5):1081-1083.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - Journal of Zhejiang University-SCIENCE