Full Text:   <585>

Summary:  <180>

CLC number: TP311

On-line Access: 2018-08-06

Received: 2016-12-03

Revision Accepted: 2017-04-17

Crosschecked: 2018-06-07

Cited: 0

Clicked: 1742

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Lin-bo Qiao

http://orcid.org/0000-0002-8285-2738

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2018 Vol.19 No.6 P.755-762

10.1631/FITEE.1601771


Stochastic extra-gradient based alternating direction methods for graph-guided regularized minimization


Author(s):  Qiang Lan, Lin-bo Qiao, Yi-jie Wang

Affiliation(s):  College of Computer, National University of Defense Technology, Changsha 410073, China; more

Corresponding email(s):   lanqiang_nudt@163.com, qiao.linbo@nudt.edu.cn, wwyyjj1971@vip.sina.com

Key Words:  Stochastic optimization, Graph-guided minimization, Extra-gradient method, Fused logistic regression, Graph-guided regularized logistic regression


Qiang Lan, Lin-bo Qiao, Yi-jie Wang. Stochastic extra-gradient based alternating direction methods for graph-guided regularized minimization[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(6): 755-762.

@article{title="Stochastic extra-gradient based alternating direction methods for graph-guided regularized minimization",
author="Qiang Lan, Lin-bo Qiao, Yi-jie Wang",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="19",
number="6",
pages="755-762",
year="2018",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.1601771"
}

%0 Journal Article
%T Stochastic extra-gradient based alternating direction methods for graph-guided regularized minimization
%A Qiang Lan
%A Lin-bo Qiao
%A Yi-jie Wang
%J Frontiers of Information Technology & Electronic Engineering
%V 19
%N 6
%P 755-762
%@ 2095-9184
%D 2018
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.1601771

TY - JOUR
T1 - Stochastic extra-gradient based alternating direction methods for graph-guided regularized minimization
A1 - Qiang Lan
A1 - Lin-bo Qiao
A1 - Yi-jie Wang
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 19
IS - 6
SP - 755
EP - 762
%@ 2095-9184
Y1 - 2018
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.1601771


Abstract: 
In this study, we propose and compare stochastic variants of the extra-gradient alternating direction method, named the stochastic extra-gradient alternating direction method with Lagrangian function (SEGL) and the stochastic extra-gradient alternating direction method with augmented Lagrangian function (SEGAL), to minimize the graph-guided optimization problems, which are composited with two convex objective functions in large scale. A number of important applications in machine learning follow the graph-guided optimization formulation, such as linear regression, logistic regression, Lasso, structured extensions of Lasso, and structured regularized logistic regression. We conduct experiments on fused logistic regression and graph-guided regularized regression. Experimental results on several genres of datasets demonstrate that the proposed algorithm outperforms other competing algorithms, and SEGAL has better performance than SEGL in practical use.

图引导正则最小化的随机超梯度的交替方向方法

概要:提出并比较额外梯度交替方向的几种随机变体方法,称为带拉格朗日函数(SEGL)的随机超梯度交替方向法和带增广拉格朗日函数(SEGAL)的随机超梯度交替方向法。这些方法由两个大规模凸目标函数组成,可最小化图形引导的优化问题。机器学习中一些重要应用遵循图导引优化公式等作为线性回归、逻辑回归、Lasso结构化扩展以及结构化正则化逻辑回归的原则。通过融合逻辑回归和图形引导正则化回归,在几类数据集上进行了试验。试验结果表明所提算法优于其他竞争算法,且在实际应用中,SEGAL比SEGL性能更好。

关键词:随机优化;图形引导最小化;超梯度法;融合逻辑回归;图导向正则化逻辑回归

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Azadi S, Sra S, 2014. Towards an optimal stochastic alternating direction method of multipliers. Int Conf on Machine Learning, p.620-628.

[2]Boyd S, Parikh N, Chu E, et al., 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn, 3(1):1-122.

[3]Cortes C, Vapnik V, 1995. Support-vector networks. Mach Learn, 20(3):273-297.

[4]Gao X, Jiang B, Zhang S, 2017. On the information-adaptive variants of the ADMM: an iteration complexity perspective. J Sci Comput, 76(1):327-363.

[5]Hastie T, Tibshirani R, Friedman J, 2001. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag, New York, USA.

[6]Hsieh CJ, Sustik MA, Dhillon IS, et al., 2013. BIG & QUIC: sparse inverse covariance estimation for a million variables. Advances in Neural Information Processing Systems, p.3165-3173.

[7]Johnson R, Zhang T, 2013. Accelerating stochastic gradient descent using predictive variance reduction. Advances in Neural Information Processing Systems, p.315-323.

[8]Lin T, Ma S, Zhang S, 2015. An extra-gradient-based alternating direction method for convex minimization. Found Comput Math, 17(1):35-59.

[9]Lin T, Qiao L, Zhang T, et al., 2018. Stochastic primal-dual proximal extra-gradient descent for compositely regularized optimization. Neurocomputing, 273:516-525.

[10]Ouyang H, He N, Tran L, et al., 2013. Stochastic alternating direction method of multipliers. Int Conf on Machine Learning, p.80-88.

[11]Qiao LB, Zhang BF, Su JS, et al., 2017. A systematic review of structured sparse learning. Front Inform Technol Electron Eng, 18(4):445-463.

[12]Suzuki T, 2013. Dual averaging and proximal gradient descent for online alternating direction multiplier method. Int Conf on Machine Learning, p.392-400.

[13]Tibshirani R, Saunders M, Rosset S, et al., 2005. Sparsity and smoothness via the fused lasso. J R Stat Soc Ser B, 67:91-108.

[14]Tibshirani R, 1996. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B, 1:267-288.

[15]Wang H, Banerjee A, 2013. Online alternating direction method (longer version). arXiv preprint 1306.3721, 1(1):1-8.

[26]Yang JF, Yuan XM, 2013. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Math Comput, 82(281):301-329.

[17]Zhao P, Yang J, Zhang T, et al., 2015. Adaptive stochastic alternating direction method of multipliers. Int Conf on Machine Learning, p.69-77.

[18]Zhong W, Kwok JT, 2013. Fast stochastic alternating direction method of multipliers. Int Conf on Machine Learning, p.46-54.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - Journal of Zhejiang University-SCIENCE