Root-finding algorithms

Broyden's method

In numerical analysis, Broyden's method is a quasi-Newton method for finding roots in k variables. It was originally described by C. G. Broyden in 1965. Newton's method for solving f(x) = 0 uses the Jacobian matrix, J, at every iteration. However, computing this Jacobian is a difficult and expensive operation. The idea behind Broyden's method is to compute the whole Jacobian only at the first iteration and to do rank-one updates at other iterations. In 1979 Gay proved that when Broyden's method is applied to a linear system of size n × n, itterminates in 2 n steps, although like all quasi-Newton methods, it may not converge for nonlinear systems. (Wikipedia).

Video thumbnail

Broyden's Method

Broyden's Method for solving systems of nonlinear equations. Lesson covers motivation, history, examples, discussion, and order of this Quasi-Newton Method. It also explains the "Good" and "Bad", as well as the third version of the method. Example code hosted on GitHub https://github.com/o

From playlist Solving Systems of Nonlinear Equations

Video thumbnail

C46 Solving the previous problem by another method

There are more ways than one to solve Cauchy-Euler equations. In this video I revert to the substitution method.

From playlist Differential Equations

Video thumbnail

A09 The Hamiltonian

Moving on from Lagrange's equation, I show you how to derive Hamilton's equation.

From playlist Physics ONE

Video thumbnail

Euler’s method - How to use it?

► My Differential Equations course: https://www.kristakingmath.com/differential-equations-course Euler’s method is a numerical method that you can use to approximate the solution to an initial value problem with a differential equation that can’t be solved using a more traditional method,

From playlist Differential Equations

Video thumbnail

Approximating the Jacobian: Finite Difference Method for Systems of Nonlinear Equations

Generalized Finite Difference Method for Simultaneous Nonlinear Systems by approximating the Jacobian using the limit of partial derivatives with the forward finite difference. Example code on GitHub https://www.github.com/osveliz/numerical-veliz Chapters 0:00 Intro 0:13 Prerequisites 0:3

From playlist Solving Systems of Nonlinear Equations

Video thumbnail

Steffensen's Method for Systems of Nonlinear Equations

Generalized Steffensen's Method for Simultaneous Nonlinear Systems originally credited to J. F. Traub. Video shows how to solve nonlinear systems by approximating the Jacobian. Example code on GitHub https://www.github.com/osveliz/numerical-veliz Chapters 0:00 Prerequisites 0:20 Intro 0:

From playlist Solving Systems of Nonlinear Equations

Video thumbnail

Halley's Method for Systems of Nonlinear Equations

Halley's Method for Solving Systems of Nonlinear Equations. Submission for The Summer of Math Exposition. Lesson includes motivation & explanation of notation, description of the method, numerical example, discussion of order, and comparison with the Method of Tangent Hyperbolas. Example c

From playlist Solving Systems of Nonlinear Equations

Video thumbnail

Generalized False Position & Alternative Secant Methods

False Position Method for Nonlinear Systems (aka Generalized Regula Falsi) along with two Alternative Secant Methods. Includes discussion of history and primary sources along with numeric examples and visualizations. Example code hosted on GitHub https://github.com/osveliz/numerical-veliz

From playlist Solving Systems of Nonlinear Equations

Video thumbnail

Secant Method for Systems of Nonlinear Equations

Generalized Secant Method for Simultaneous Nonlinear Systems originally credited to Wolfe and Bittner. Lesson shows how to solve nonlinear systems without the Jacobian, nor the need to approximate it, in a straightforward and visual manner. Example code on GitHub https://www.github.com/osv

From playlist Solving Systems of Nonlinear Equations

Video thumbnail

Neural Networks Demystified [Part 6: Training]

After all that work it's finally time to train our Neural Network. We'll use the BFGS numerical optimization algorithm and have a look at the results. Supporting Code: https://github.com/stephencwelch/Neural-Networks-Demystified Yann Lecun's Efficient BackProp Paper: http://yann.lecun

From playlist Neural Networks Demystified

Video thumbnail

C45 Example problem solving a Cauchy Euler equation

Solving problems is the ONLY way get to learn these techniques. Another Cauchy-Euler equation solved.

From playlist Differential Equations

Video thumbnail

Nonlinear Finite Element Solvers

To learn more about Wolfram Technology Conference, please visit: https://www.wolfram.com/events/technology-conference/ Speaker: Oliver Rubenkonig Wolfram developers and colleagues discussed the latest in innovative technologies for cloud computing, interactive deployment, mobile devices,

From playlist Wolfram Technology Conference 2018

Video thumbnail

Gaussian Integral 8 Original Way

Welcome to the awesome 12-part series on the Gaussian integral. In this series of videos, I calculate the Gaussian integral in 12 different ways. Which method is the best? Watch and find out! In this video, I present the classical way using polar coordinates, the one that Laplace original

From playlist Gaussian Integral

Video thumbnail

Gaussian Integral 4 Feynman way

Welcome to the awesome 12-part series on the Gaussian integral. In this series of videos, I calculate the Gaussian integral in 12 different ways. Which method is the best? Watch and find out! In this video, I use a technique similar to Feynman's technique by differentiating a more complic

From playlist Gaussian Integral

Video thumbnail

Labeling a System by Solving Using Elimination Method

👉Learn how to solve a system (of equations) by elimination. A system of equations is a set of equations which are collectively satisfied by one solution of the variables. The elimination method of solving a system of equations involves making the coefficient of one of the variables to be e

From playlist Solve a System of Equations Using Elimination | Medium

Video thumbnail

Newton's Method for Systems of Nonlinear Equations

Generalized Newton's method for systems of nonlinear equations. Lesson goes over numerically solving multivariable nonlinear equations step-by-step with visual examples and explanation of the Jacobian, the backslash operator, and the inverse Jacobian. Example code in MATLAB / GNU Octave on

From playlist Newton's Method

Video thumbnail

C34 Expanding this method to higher order linear differential equations

I this video I expand the method of the variation of parameters to higher-order (higher than two), linear ODE's.

From playlist Differential Equations

Video thumbnail

Gaussian Integral 7 Wallis Way

Welcome to the awesome 12-part series on the Gaussian integral. In this series of videos, I calculate the Gaussian integral in 12 different ways. Which method is the best? Watch and find out! In this video, I calculate the Gaussian integral by using a technique that is very similar to the

From playlist Gaussian Integral

Video thumbnail

Harvard AM205 video 4.9 - Quasi-Newton methods

Harvard Applied Math 205 is a graduate-level course on scientific computing and numerical methods. The previous video in this series discussed using the Newton method to find local minima of a function; while this method can be highly efficient, it requires the exact Hessian of the functio

From playlist Optimizers in Machine Learning

Related pages

Matrix norm | Jacobian matrix and determinant | Davidon–Fletcher–Powell formula | Quasi-Newton method | Broyden–Fletcher–Goldfarb–Shanno algorithm | Gradient | Newton's method in optimization | Secant method | Newton's method | Sherman–Morrison formula | Hessian matrix | Underdetermined system