File Download
There are no files associated with this item.
Links for fulltext
(May Require Subscription)
- Publisher Website: 10.1007/s43069-021-00059-y
- Find via
Supplementary
-
Citations:
- Appears in Collections:
Article: Randomized Smoothing Variance Reduction Method for Large-Scale Non-smooth Convex Optimization
Title | Randomized Smoothing Variance Reduction Method for Large-Scale Non-smooth Convex Optimization |
---|---|
Authors | |
Keywords | Non-smooth optimization Randomized smoothing Variance reduction Stochastic gradient descent Linear convergence |
Issue Date | 2021 |
Publisher | Springer. The Journal's web site is located at https://www.springer.com/journal/43069 |
Citation | Operations Research Forum, 2021, v. 2 n. 2, article no. 26 How to Cite? |
Abstract | We consider a new method for minimizing the average of a large number of non-smooth and convex functions. Such a problem often arises in typical machine learning problems, but is computationally challenging. We apply an implementable randomized smoothing method and propose a multistage scheme to progressively reduce the variance of the gradient estimator of the smoothed functions. Our algorithm achieves a linear convergence rate. Both its time complexity and gradient complexity are superior to the current standard algorithms for non-smooth minimization as well as subgradient-based algorithms. Besides, our algorithm works well without the error-bound condition on the minimizing sequence as well as the commonly imposed (but strong) smoothness and strongly convexity condition. We show that our algorithm has wide applications in optimization and machine learning problems. As an illustrative example, we demonstrate experimentally that our algorithm performs well on large-scale ranking problems and risk-aware portfolio optimization problems. |
Persistent Identifier | http://hdl.handle.net/10722/305822 |
ISSN | 2023 SCImago Journal Rankings: 0.388 |
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Huang, W | - |
dc.contributor.author | Zhang, X | - |
dc.date.accessioned | 2021-10-20T10:14:49Z | - |
dc.date.available | 2021-10-20T10:14:49Z | - |
dc.date.issued | 2021 | - |
dc.identifier.citation | Operations Research Forum, 2021, v. 2 n. 2, article no. 26 | - |
dc.identifier.issn | 2662-2556 | - |
dc.identifier.uri | http://hdl.handle.net/10722/305822 | - |
dc.description.abstract | We consider a new method for minimizing the average of a large number of non-smooth and convex functions. Such a problem often arises in typical machine learning problems, but is computationally challenging. We apply an implementable randomized smoothing method and propose a multistage scheme to progressively reduce the variance of the gradient estimator of the smoothed functions. Our algorithm achieves a linear convergence rate. Both its time complexity and gradient complexity are superior to the current standard algorithms for non-smooth minimization as well as subgradient-based algorithms. Besides, our algorithm works well without the error-bound condition on the minimizing sequence as well as the commonly imposed (but strong) smoothness and strongly convexity condition. We show that our algorithm has wide applications in optimization and machine learning problems. As an illustrative example, we demonstrate experimentally that our algorithm performs well on large-scale ranking problems and risk-aware portfolio optimization problems. | - |
dc.language | eng | - |
dc.publisher | Springer. The Journal's web site is located at https://www.springer.com/journal/43069 | - |
dc.relation.ispartof | Operations Research Forum | - |
dc.subject | Non-smooth optimization | - |
dc.subject | Randomized smoothing | - |
dc.subject | Variance reduction | - |
dc.subject | Stochastic gradient descent | - |
dc.subject | Linear convergence | - |
dc.title | Randomized Smoothing Variance Reduction Method for Large-Scale Non-smooth Convex Optimization | - |
dc.type | Article | - |
dc.identifier.email | Huang, W: huangwj@hku.hk | - |
dc.identifier.authority | Huang, W=rp02898 | - |
dc.description.nature | link_to_subscribed_fulltext | - |
dc.identifier.doi | 10.1007/s43069-021-00059-y | - |
dc.identifier.hkuros | 327219 | - |
dc.identifier.volume | 2 | - |
dc.identifier.issue | 2 | - |
dc.identifier.spage | article no. 26 | - |
dc.identifier.epage | article no. 26 | - |
dc.publisher.place | Switzerland | - |