Full Text:   <693>

Summary:  <93>

CLC number: O224

On-line Access: 2023-06-21

Received: 2022-09-07

Revision Accepted: 2023-09-21

Crosschecked: 2022-10-28

Cited: 0

Clicked: 821

Citations:  Bibtex RefMan EndNote GB/T7714


Zicong XIA


Yang LIU


Wenlian LU


Weihua GUI


-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2023 Vol.24 No.9 P.1239-1252


Matrix-valued distributed stochastic optimization with constraints

Author(s):  Zicong XIA, Yang LIU, Wenlian LU, Weihua GUI

Affiliation(s):  Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province, Zhejiang Normal University, Jinhua 321004, China; more

Corresponding email(s):   201531700128@zjnu.edu.cn, liuyang@zjnu.edu.cn, wenlian@fudan.edu.cn, gwh@csu.edu.cn

Key Words:  Distributed optimization, Matrix-valued optimization, Stochastic optimization, Penalty method, Gossip model

Share this article to: More |Next Article >>>

Zicong XIA, Yang LIU, Wenlian LU, Weihua GUI. Matrix-valued distributed stochastic optimization with constraints[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(9): 1239-1252.

@article{title="Matrix-valued distributed stochastic optimization with constraints",
author="Zicong XIA, Yang LIU, Wenlian LU, Weihua GUI",
journal="Frontiers of Information Technology & Electronic Engineering",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Matrix-valued distributed stochastic optimization with constraints
%A Zicong XIA
%A Yang LIU
%A Wenlian LU
%A Weihua GUI
%J Frontiers of Information Technology & Electronic Engineering
%V 24
%N 9
%P 1239-1252
%@ 2095-9184
%D 2023
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.2200381

T1 - Matrix-valued distributed stochastic optimization with constraints
A1 - Zicong XIA
A1 - Yang LIU
A1 - Wenlian LU
A1 - Weihua GUI
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 24
IS - 9
SP - 1239
EP - 1252
%@ 2095-9184
Y1 - 2023
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.2200381

In this paper, we address matrix-valued distributed stochastic optimization with inequality and equality constraints, where the objective function is a sum of multiple matrix-valued functions with stochastic variables and the considered problems are solved in a distributed manner. A penalty method is derived to deal with the constraints, and a selection principle is proposed for choosing feasible penalty functions and penalty gains. A distributed optimization algorithm based on the gossip model is developed for solving the stochastic optimization problem, and its convergence to the optimal solution is analyzed rigorously. Two numerical examples are given to demonstrate the viability of the main results.




Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1]Bin SQ, Xia YS, 2014. Fast multi-channel image reconstruction using a novel two-dimensional algorithm. Multimed Tools Appl, 71(3):2015-2028.

[2]Bouhamidi A, Jbilou K, 2012. A kronecker approximation with a convex constrained optimization method for blind image restoration. Optim Lett, 6(7):1251-1264.

[3]Boyd S, Vandenberghe L, 2004. Convex Optimization. Cambridge University Press, Cambridge, UK.

[4]Boyd S, Ghosh A, Prabhakar B, et al., 2006. Randomized gossip algorithms. IEEE Trans Inform Theory, 52(6):2508-2530.

[5]Deng ZH, Liang S, Hong YG, 2018. Distributed continuous-time algorithms for resource allocation problems over weight-balanced digraphs. IEEE Trans Cybern, 48(11):3116-3125.

[6]Huang LM, Xia YS, Huang LQ, et al., 2021. Two matrix-type projection neural networks for matrix-valued optimization with application to image restoration. Neur Process Lett, 53(3):1685-1707.

[7]Jakovetic D, Xavier J, Moura JMF, 2011. Cooperative convex optimization in networked systems: augmented Lagrangian algorithms with directed gossip communication. IEEE Trans Signal Process, 59(8):3889-3902.

[8]Jiang XR, Qin ST, Xue XP, 2021. Continuous-time algorithm for approximate distributed optimization with affine equality and convex inequality constraints. IEEE Trans Syst Man Cybern Syst, 51(9):5809-5818.

[9]Koloskova A, Stich SU, Jaggi M, 2019. Decentralized stochastic optimization and gossip algorithms with compressed communication. Proc 36th Int Conf on Machine Learning, p.3478-3487.

[10]Li H, Fang C, Lin ZC, 2020. Accelerated first-order optimization algorithms for machine learning. Proc IEEE, 108(11):2067-2082.

[11]Li JF, Li W, Huang R, 2016. An efficient method for solving a matrix least squares problem over a matrix inequality constraint. Comput Optim Appl, 63(2):393-423.

[12]Li XX, Xie LH, Hong YG, 2020. Distributed continuous-time nonsmooth convex optimization with coupled inequality constraints. IEEE Trans Contr Netw Syst, 7(1):74-84.

[13]Liang S, Zeng XL, Hong YG, 2018a. Distributed nonsmooth optimization with coupled inequality constraints via modified Lagrangian function. IEEE Trans Autom Contr, 63(6):1753-1759.

[14]Liang S, Zeng XL, Hong YG, 2018b. Distributed sub-optimal resource allocation over weight-balanced graph via singular perturbation. Automatica, 95:222-228.

[15]Liu QS, Wang J, 2013. A one-layer projection neural network for nonsmooth optimization subject to linear equalities and bound constraints. IEEE Trans Neur Netw Learn Syst, 24(5):812-824.

[16]Liu QS, Wang J, 2015. A second-order multi-agent network for bound-constrained distributed optimization. IEEE Trans Autom Contr, 60(12):3310-3315.

[17]Liu QS, Yang SF, Wang J, 2017. A collective neurodynamic approach to distributed constrained optimization. IEEE Trans Neur Netw Learn Syst, 28(8):1747-1758.

[18]Lu J, Tang CY, Regier PR, et al., 2011. Gossip algorithms for convex consensus optimization over networks. IEEE Trans Autom Contr, 56(12):2917-2923.

[19]Lv YW, Yang GH, Shi CX, 2020. Differentially private distributed optimization for multi-agent systems via the augmented Lagrangian algorithm. Inform Sci, 538:39-53.

[20]Nedic A, Ozdaglar A, 2009. Distributed subgradient methods for multi-agent optimization. IEEE Trans Autom Contr, 54(1):48-61.

[21]Rakhlin A, Shamir O, Sridharan K, 2012. Making gradient descent optimal for strongly convex stochastic optimization. Proc 29th Int Conf on Machine Learning, p.1571-1578.

[22]Ruszczyński A, 2006. Nonlinear Optimization. Princeton University Press, Princeton, USA.

[23]Shamir O, Srebro N, 2014. Distributed stochastic optimization and learning. Proc 52nd Annual Allerton Conf on Communication, Control, and Computing, p.850-857.

[24]Shi XL, Cao JD, Wen GH, et al., 2019. Finite-time consensus of opinion dynamics and its applications to distributed optimization over digraph. IEEE Trans Cybern, 49(10):3767-3779.

[25]Wan P, Lemmon MD, 2009. Event-triggered distributed optimization in sensor networks. Proc Int Conf on Information Processing in Sensor Networks, p.49-60.

[26]Wang D, Wang Z, Wen CY, 2021. Distributed optimal consensus control for a class of uncertain nonlinear multiagent networks with disturbance rejection using adaptive technique. IEEE Trans Syst Man Cybern Syst, 51(7):4389-4399.

[27]Wang XY, Wang GD, Li SH, 2021. Distributed finite-time optimization for disturbed second-order multiagent systems. IEEE Trans Cybern, 51(9):4634-4647.

[28]Xia YS, Chen TP, Shan JJ, 2014. A novel iterative method for computing generalized inverse. Neur Comput, 26(2):449-465.

[29]Xia ZC, Liu Y, Lu JQ, et al., 2021. Penalty method for constrained distributed quaternion-variable optimization. IEEE Trans Cybern, 51(11):5631-5636.

[30]Xia ZC, Liu Y, Kou KI, et al., 2022. Clifford-valued distributed optimization based on recurrent neural networks. IEEE Trans Neur Netw Learn Syst, early access.

[31]Xia ZC, Liu Y, Qiu JL, et al., 2023. An RNN-based algorithm for decentralized-partial-consensus constrained optimization. IEEE Trans Neur Netw Learn Syst, 34(1):534-542.

[32]Xiao L, Boyd S, 2004. Fast linear iterations for distributed averaging. Syst Contr Lett, 53(1):65-78.

[33]Xing CW, Wang S, Chen S, et al., 2021. Matrix-monotonic optimization – part I: single-variable optimization. IEEE Trans Signal Process, 69:738-754.

[34]Yang SF, Liu QS, Wang J, 2017. Distributed optimization based on a multiagent system in the presence of communication delays. IEEE Trans Syst Man Cybern Syst, 47(5):717-728.

[35]Yang T, Yi XL, Wu JF, et al., 2019. A survey of distributed optimization. Annu Rev Contr, 47:278-305.

[36]Yuan DM, 2014. Gossip-based gradient-free method for multi-agent optimization: constant step size analysis. Proc 33rd Chinese Control Conf, p.1349-1353.

[37]Yue DD, Baldi S, Cao JD, et al., 2022. Distributed adaptive optimization with weight-balancing. IEEE Trans Autom Contr, 67(4):2068-2075.

[38]Zeng XL, Yi P, Hong YG, 2017. Distributed continuous-time algorithm for constrained convex optimizations via nonsmooth analysis approach. IEEE Trans Autom Contr, 62(10):5227-5233.

[39]Zeng XL, Chen J, Hong YG, 2022. Distributed optimization design of iterative refinement technique for algebraic Riccati equations. IEEE Trans Syst Man Cybern Syst, 52(5):2833-2847.

[40]Zhang SC, Xia YH, Xia YS, et al., 2022. Matrix-form neural networks for complex-variable basis pursuit problem with application to sparse signal reconstruction. IEEE Trans Cybern, 52(7):7049-7059.

[41]Zhou HB, Zeng XL, Hong YG, 2017. An exact penalty method for constrained distributed optimization. Proc 36th Chinese Control Conf, p.8827-8832.

[42]Zhou HB, Zeng XL, Hong YG, 2019. Adaptive exact penalty design for constrained distributed optimization. IEEE Trans Autom Contr, 64(11):4661-4667.

[43]Zhu MH, Martínez S, 2012. On distributed convex optimization under inequality and equality constraints. IEEE Trans Autom Contr, 57(1):151-164.

[44]Zhu ZH, Li QW, Tang GG, et al., 2021. The global optimization geometry of low-rank matrix optimization. IEEE Trans Inform Theory, 67(2):1308-1331.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE