CLC number: O224
On-line Access: 2023-06-21
Received: 2022-09-07
Revision Accepted: 2023-09-21
Crosschecked: 2022-10-28
Cited: 0
Clicked: 935
Citations: Bibtex RefMan EndNote GB/T7714
https://orcid.org/0000-0001-9943-5087
https://orcid.org/0000-0003-3761-0104
Zicong XIA, Yang LIU, Wenlian LU, Weihua GUI. Matrix-valued distributed stochastic optimization with constraints[J]. Frontiers of Information Technology & Electronic Engineering,in press.https://doi.org/10.1631/FITEE.2200381 @article{title="Matrix-valued distributed stochastic optimization with constraints", %0 Journal Article TY - JOUR
带约束的矩阵值分布式随机优化1浙江师范大学浙江省智能教育技术与应用重点实验室,中国金华市,321004 2浙江师范大学数学科学学院,中国金华市,321004 3复旦大学数学科学学院,中国上海市,200433 4中南大学自动化学院,中国长沙市,410083 摘要:本文研究带有不等式约束和等式约束的矩阵值分布随机优化问题。其中,问题的目标函数是具有随机变量的多个矩阵值函数的和,并以分布式方式解决了该问题。本文推导了处理约束的惩罚方法,并提出选择可行惩罚函数和惩罚增益的原则。针对随机优化问题,提出一种基于gossip模型的分布式优化算法,并对其收敛性进行证明和分析。最后,为验证所提算法的可行性,本文提供了两个数值示例。 关键词组: Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article
Reference[1]Bin SQ, Xia YS, 2014. Fast multi-channel image reconstruction using a novel two-dimensional algorithm. Multimed Tools Appl, 71(3):2015-2028. [2]Bouhamidi A, Jbilou K, 2012. A kronecker approximation with a convex constrained optimization method for blind image restoration. Optim Lett, 6(7):1251-1264. [3]Boyd S, Vandenberghe L, 2004. Convex Optimization. Cambridge University Press, Cambridge, UK. [4]Boyd S, Ghosh A, Prabhakar B, et al., 2006. Randomized gossip algorithms. IEEE Trans Inform Theory, 52(6):2508-2530. [5]Deng ZH, Liang S, Hong YG, 2018. Distributed continuous-time algorithms for resource allocation problems over weight-balanced digraphs. IEEE Trans Cybern, 48(11):3116-3125. [6]Huang LM, Xia YS, Huang LQ, et al., 2021. Two matrix-type projection neural networks for matrix-valued optimization with application to image restoration. Neur Process Lett, 53(3):1685-1707. [7]Jakovetic D, Xavier J, Moura JMF, 2011. Cooperative convex optimization in networked systems: augmented Lagrangian algorithms with directed gossip communication. IEEE Trans Signal Process, 59(8):3889-3902. [8]Jiang XR, Qin ST, Xue XP, 2021. Continuous-time algorithm for approximate distributed optimization with affine equality and convex inequality constraints. IEEE Trans Syst Man Cybern Syst, 51(9):5809-5818. [9]Koloskova A, Stich SU, Jaggi M, 2019. Decentralized stochastic optimization and gossip algorithms with compressed communication. Proc 36th Int Conf on Machine Learning, p.3478-3487. [10]Li H, Fang C, Lin ZC, 2020. Accelerated first-order optimization algorithms for machine learning. Proc IEEE, 108(11):2067-2082. [11]Li JF, Li W, Huang R, 2016. An efficient method for solving a matrix least squares problem over a matrix inequality constraint. Comput Optim Appl, 63(2):393-423. [12]Li XX, Xie LH, Hong YG, 2020. Distributed continuous-time nonsmooth convex optimization with coupled inequality constraints. IEEE Trans Contr Netw Syst, 7(1):74-84. [13]Liang S, Zeng XL, Hong YG, 2018a. Distributed nonsmooth optimization with coupled inequality constraints via modified Lagrangian function. IEEE Trans Autom Contr, 63(6):1753-1759. [14]Liang S, Zeng XL, Hong YG, 2018b. Distributed sub-optimal resource allocation over weight-balanced graph via singular perturbation. Automatica, 95:222-228. [15]Liu QS, Wang J, 2013. A one-layer projection neural network for nonsmooth optimization subject to linear equalities and bound constraints. IEEE Trans Neur Netw Learn Syst, 24(5):812-824. [16]Liu QS, Wang J, 2015. A second-order multi-agent network for bound-constrained distributed optimization. IEEE Trans Autom Contr, 60(12):3310-3315. [17]Liu QS, Yang SF, Wang J, 2017. A collective neurodynamic approach to distributed constrained optimization. IEEE Trans Neur Netw Learn Syst, 28(8):1747-1758. [18]Lu J, Tang CY, Regier PR, et al., 2011. Gossip algorithms for convex consensus optimization over networks. IEEE Trans Autom Contr, 56(12):2917-2923. [19]Lv YW, Yang GH, Shi CX, 2020. Differentially private distributed optimization for multi-agent systems via the augmented Lagrangian algorithm. Inform Sci, 538:39-53. [20]Nedic A, Ozdaglar A, 2009. Distributed subgradient methods for multi-agent optimization. IEEE Trans Autom Contr, 54(1):48-61. [21]Rakhlin A, Shamir O, Sridharan K, 2012. Making gradient descent optimal for strongly convex stochastic optimization. Proc 29th Int Conf on Machine Learning, p.1571-1578. [22]Ruszczyński A, 2006. Nonlinear Optimization. Princeton University Press, Princeton, USA. [23]Shamir O, Srebro N, 2014. Distributed stochastic optimization and learning. Proc 52nd Annual Allerton Conf on Communication, Control, and Computing, p.850-857. [24]Shi XL, Cao JD, Wen GH, et al., 2019. Finite-time consensus of opinion dynamics and its applications to distributed optimization over digraph. IEEE Trans Cybern, 49(10):3767-3779. [25]Wan P, Lemmon MD, 2009. Event-triggered distributed optimization in sensor networks. Proc Int Conf on Information Processing in Sensor Networks, p.49-60. [26]Wang D, Wang Z, Wen CY, 2021. Distributed optimal consensus control for a class of uncertain nonlinear multiagent networks with disturbance rejection using adaptive technique. IEEE Trans Syst Man Cybern Syst, 51(7):4389-4399. [27]Wang XY, Wang GD, Li SH, 2021. Distributed finite-time optimization for disturbed second-order multiagent systems. IEEE Trans Cybern, 51(9):4634-4647. [28]Xia YS, Chen TP, Shan JJ, 2014. A novel iterative method for computing generalized inverse. Neur Comput, 26(2):449-465. [29]Xia ZC, Liu Y, Lu JQ, et al., 2021. Penalty method for constrained distributed quaternion-variable optimization. IEEE Trans Cybern, 51(11):5631-5636. [30]Xia ZC, Liu Y, Kou KI, et al., 2022. Clifford-valued distributed optimization based on recurrent neural networks. IEEE Trans Neur Netw Learn Syst, early access. [31]Xia ZC, Liu Y, Qiu JL, et al., 2023. An RNN-based algorithm for decentralized-partial-consensus constrained optimization. IEEE Trans Neur Netw Learn Syst, 34(1):534-542. [32]Xiao L, Boyd S, 2004. Fast linear iterations for distributed averaging. Syst Contr Lett, 53(1):65-78. [33]Xing CW, Wang S, Chen S, et al., 2021. Matrix-monotonic optimization – part I: single-variable optimization. IEEE Trans Signal Process, 69:738-754. [34]Yang SF, Liu QS, Wang J, 2017. Distributed optimization based on a multiagent system in the presence of communication delays. IEEE Trans Syst Man Cybern Syst, 47(5):717-728. [35]Yang T, Yi XL, Wu JF, et al., 2019. A survey of distributed optimization. Annu Rev Contr, 47:278-305. [36]Yuan DM, 2014. Gossip-based gradient-free method for multi-agent optimization: constant step size analysis. Proc 33rd Chinese Control Conf, p.1349-1353. [37]Yue DD, Baldi S, Cao JD, et al., 2022. Distributed adaptive optimization with weight-balancing. IEEE Trans Autom Contr, 67(4):2068-2075. [38]Zeng XL, Yi P, Hong YG, 2017. Distributed continuous-time algorithm for constrained convex optimizations via nonsmooth analysis approach. IEEE Trans Autom Contr, 62(10):5227-5233. [39]Zeng XL, Chen J, Hong YG, 2022. Distributed optimization design of iterative refinement technique for algebraic Riccati equations. IEEE Trans Syst Man Cybern Syst, 52(5):2833-2847. [40]Zhang SC, Xia YH, Xia YS, et al., 2022. Matrix-form neural networks for complex-variable basis pursuit problem with application to sparse signal reconstruction. IEEE Trans Cybern, 52(7):7049-7059. [41]Zhou HB, Zeng XL, Hong YG, 2017. An exact penalty method for constrained distributed optimization. Proc 36th Chinese Control Conf, p.8827-8832. [42]Zhou HB, Zeng XL, Hong YG, 2019. Adaptive exact penalty design for constrained distributed optimization. IEEE Trans Autom Contr, 64(11):4661-4667. [43]Zhu MH, Martínez S, 2012. On distributed convex optimization under inequality and equality constraints. IEEE Trans Autom Contr, 57(1):151-164. [44]Zhu ZH, Li QW, Tang GG, et al., 2021. The global optimization geometry of low-rank matrix optimization. IEEE Trans Inform Theory, 67(2):1308-1331. Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou
310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE |
Open peer comments: Debate/Discuss/Question/Opinion
<1>