Gradient methods | Machine learning algorithms | Convex optimization | Stochastic optimization

Stochastic variance reduction

(Stochastic) variance reduction is an algorithmic approach to minimizing functions that can be decomposed into finite sums. By exploiting the finite sum structure, variance reduction techniques are able to achieve convergence rates that are impossible to achieve with methods that treat the objective as an infinite sum, as in the classical Stochastic approximation setting. Variance reduction approaches are widely used for training machine learning models such as logistic regression and support vector machines as these problems have finite-sum structure and uniform conditioning that make them ideal candidates for variance reduction. (Wikipedia).

Video thumbnail

Stochastic Normalizing Flows

Introduction to the paper https://arxiv.org/abs/2002.06707

From playlist Research

Video thumbnail

Frederic Legoll: Variance reduction approaches for stochastic homogenization

Find this video and other talks given by worldwide mathematicians on CIRM's Audiovisual Mathematics Library: http://library.cirm-math.fr. And discover all its functionalities: - Chapter markers and keywords to watch the parts of your choice in the video - Videos enriched with abstracts, b

From playlist Probability and Statistics

Video thumbnail

Covariance (1 of 17) What is Covariance? in Relation to Variance and Correlation

Visit http://ilectureonline.com for more math and science lectures! To donate:a http://www.ilectureonline.com/donate https://www.patreon.com/user?u=3236071 We will learn the difference between the variance and the covariance. A variance (s^2) is a measure of how spread out the numbers of

From playlist COVARIANCE AND VARIANCE

Video thumbnail

Hybrid sparse stochastic processes and the resolution of (...) - Unser - Workshop 2 - CEB T1 2019

Michael Unser (EPFL) / 12.03.2019 Hybrid sparse stochastic processes and the resolution of linear inverse problems. Sparse stochastic processes are continuous-domain processes that are specified as solutions of linear stochastic differential equations driven by white Lévy noise. These p

From playlist 2019 - T1 - The Mathematics of Imaging

Video thumbnail

Basic stochastic simulation b: Stochastic simulation algorithm

(C) 2012-2013 David Liao (lookatphysics.com) CC-BY-SA Specify system Determine duration until next event Exponentially distributed waiting times Determine what kind of reaction next event will be For more information, please search the internet for "stochastic simulation algorithm" or "kin

From playlist Probability, statistics, and stochastic processes

Video thumbnail

Giovanni Peccati: Some applications of variational techniques in stochastic geometry I

Some variance estimates on the Poisson space, Part I I will introduce some basic tools of stochastic analysis on the Poisson space, and describe how they can be used to develop variational inequalities for assessing the magnitude of variances of geometric quantities. Particular attention

From playlist Winter School on the Interplay between High-Dimensional Geometry and Probability

Video thumbnail

How to find the number of standard deviations that it takes to represent all the data

👉 Learn how to find the variance and standard deviation of a set of data. The variance of a set of data is a measure of spread/variation which measures how far a set of numbers is spread out from their average value. The standard deviation of a set of data is a measure of spread/variation

From playlist Variance and Standard Deviation

Video thumbnail

Mini Batch Gradient Descent | Deep Learning | with Stochastic Gradient Descent

Mini Batch Gradient Descent is an algorithm that helps to speed up learning while dealing with a large dataset. Instead of updating the weight parameters after assessing the entire dataset, Mini Batch Gradient Descent updates weight parameters after assessing the small batch of the datase

From playlist Optimizers in Machine Learning

Video thumbnail

Stochastic Gradient Descent and Machine Learning (Lecture 3) by Praneeth Netrapalli

PROGRAM: BANGALORE SCHOOL ON STATISTICAL PHYSICS - XIII (HYBRID) ORGANIZERS: Abhishek Dhar (ICTS-TIFR, India) and Sanjib Sabhapandit (RRI, India) DATE & TIME: 11 July 2022 to 22 July 2022 VENUE: Madhava Lecture Hall and Online This school is the thirteenth in the series. The schoo

From playlist Bangalore School on Statistical Physics - XIII - 2022 (Live Streamed)

Video thumbnail

Plamen Turkedjiev: Least squares regression Monte Carlo for approximating BSDES and semilinear PDES

Abstract: In this lecture, we shall discuss the key steps involved in the use of least squares regression for approximating the solution to BSDEs. This includes how to obtain explicit error estimates, and how these error estimates can be used to tune the parameters of the numerical scheme

From playlist Probability and Statistics

Video thumbnail

Talk Andrea Tosin: Kinetic modelling of traffic flow control

The lecture was held within the of the Hausdorff Trimester Program: Kinetic Theory Abstract: In this talk, we present a hierarchical description of control problems for vehicular traffic, which aim to mitigate speed-dependent risk factors and to dampen structural uncertainties responsible

From playlist Summer School: Trails in kinetic theory: foundational aspects and numerical methods

Video thumbnail

Fifth Imaging & Inverse Problems (IMAGINE) OneWorld SIAM-IS Virtual Seminar Series Talk

Date: Wednesday, November 1, 10:00am EDT Speaker: Xiaoqun Zhang, Shanghai Jiao Tong University Title: Stochastic primal dual splitting algorithms for convex and nonconvex composite optimization in imaging Abstract: Primal dual splitting algorithms are largely adopted for composited optim

From playlist Imaging & Inverse Problems (IMAGINE) OneWorld SIAM-IS Virtual Seminar Series

Video thumbnail

Grigorios A Pavliotis: Accelerating convergence and reducing variance for Langevin samplers

Grigorios A. Pavliotis: Accelerating convergence and reducing variance for Langevin samplers Markov Chain Monte Carlo (MCMC) is a standard methodology for sampling from probability distributions (known up to the normalization constant) in high dimensions. There are (infinitely) many diff

From playlist HIM Lectures 2015

Video thumbnail

DDPS | Data-driven information geometry approach to stochastic model reduction

Description: Reduced-order models are often obtained by projection onto a subspace; standard least squares in linear spaces is a familiar technique that can also be applied to stochastic phenomena as exemplified by polynomial chaos expansions. Optimal approximants are obtained by minimizin

From playlist Data-driven Physical Simulations (DDPS) Seminar Series

Video thumbnail

Introductory lectures on first-order convex optimization (Lecture 2) by Praneeth Netrapalli

DISCUSSION MEETING : STATISTICAL PHYSICS OF MACHINE LEARNING ORGANIZERS : Chandan Dasgupta, Abhishek Dhar and Satya Majumdar DATE : 06 January 2020 to 10 January 2020 VENUE : Madhava Lecture Hall, ICTS Bangalore Machine learning techniques, especially “deep learning” using multilayer n

From playlist Statistical Physics of Machine Learning 2020

Video thumbnail

Maxime Laborde: "A Lyapunov analysis for accelerated gradient methods: From deterministic to sto..."

High Dimensional Hamilton-Jacobi PDEs 2020 Workshop II: PDE and Inverse Problem Methods in Machine Learning "A Lyapunov analysis for accelerated gradient methods: From deterministic to stochastic case" Maxime Laborde - McGill University Abstract: Su, Boyd and Candés showed that Nesterov'

From playlist High Dimensional Hamilton-Jacobi PDEs 2020

Video thumbnail

Stochastic Analysis and Applications in Gene Networks by Chunhe Li

PROGRAM TIPPING POINTS IN COMPLEX SYSTEMS (HYBRID) ORGANIZERS: Partha Sharathi Dutta (IIT Ropar, India), Vishwesha Guttal (IISc, India), Mohit Kumar Jolly (IISc, India) and Sudipta Kumar Sinha (IIT Ropar, India) DATE: 19 September 2022 to 30 September 2022 VENUE: Ramanujan Lecture Hall an

From playlist TIPPING POINTS IN COMPLEX SYSTEMS (HYBRID, 2022)

Video thumbnail

STRATIFIED, SYSTEMATIC, and CLUSTER Random Sampling (12-4)

To create a Stratified Random Sample, divide the population into smaller subgroups called strata, then use random sampling within each stratum. Strata are formed based on members’ shared (qualitative) characteristics or attributes. Stratification can be proportionate to the population size

From playlist Sampling Distributions in Statistics (WK 12 - QBA 237)

Related pages

Logistic regression | Random variable | Duality (optimization) | Stochastic optimization | Condition number | Lipschitz continuity | Coordinate descent | Online machine learning | Stochastic approximation | Stochastic gradient descent | Convex conjugate | Proximal operator