Numerical Methods

1. Iterative Methods for Linear Algebra Problems
   1. Basic Concepts
      1. System of Linear Equations
         1. Definition and Properties
         2. Importance in Computational Problems
         3. Scalability Issues with Direct Methods
      2. Iterative Methods Overview
         1. Comparison with Direct Methods
         2. Applicability to Sparse and Large Systems
         3. Flexibility for Parallel Computation
      3. Convergence Criteria
         1. Residual Norm
         2. Convergence Tolerance
         3. Maximum Iterations
   2. Methods
      1. Jacobi Method
         1. Algorithm Description
            1. Diagonal Dominance Requirement
            2. Implementation Steps
            3. Example Problems
         2. Advantages and Limitations
            1. Simplicity
            2. Slow Convergence on Non-Trivial Systems
      2. Gauss-Seidel Method
         1. Algorithm Description
            1. Successive Substitution
            2. Implementation Steps
            3. Example Problems
         2. Comparisons with Jacobi Method
            1. Faster Convergence under Certain Conditions
            2. Dependency on Initial Guess
         3. Relaxation Techniques
      3. Successive Over-Relaxation (SOR)
         1. Algorithm Description
            1. Relaxation Parameter Selection
            2. Iterative Update Formula
         2. Convergence Improvement
            1. Application to Systems with Faster Convergence Needs
            2. Sensitivity to Parameter Choice
      4. Conjugate Gradient Method
         1. Application to Symmetric Positive Definite Matrices
         2. Algorithm Description
            1. Mathematical Foundation
            2. Iterative Steps
         3. Performance and Convergence
            1. Memory Efficiency for Large Systems
            2. Effect of Preconditioning
      5. Generalized Minimal Residual (GMRES) Method
         1. Application to Nonsymmetric Linear Systems
         2. Algorithm Description
            1. Krylov Subspace Methods
            2. Restart Strategies for Memory Control
         3. Convergence Characteristics
            1. Dependence on Spectrum of Matrix
            2. Behavior with Preconditioners
      6. Bi-Conjugate Gradient Stabilized (BiCGSTAB) Method
         1. Enhancement over Bi-Conjugate Gradient
         2. Algorithm Description
            1. Avoidance of Breakdown
            2. Implementation Details
         3. Contexts of Use
            1. Ill-Conditioned and Nonsymmetric Systems
         4. Comparison with GMRES and Conjugate Gradient
   3. Convergence Analysis
      1. Spectral Radius and Convergence
         1. Importance in Evaluating Iterative Methods
         2. Spectral Properties of Matrices
      2. Rate of Convergence
         1. Impact of Matrix Condition Number
         2. Linear vs. Superlinear Convergence
      3. Influence of Initial Guess
         1. Sensitivity Analysis
         2. Strategies for Choice of Initial Guess
   4. Preconditioning Techniques
      1. Importance and Purpose
         1. Reduction of Condition Number
         2. Acceleration of Convergence
      2. Types of Preconditioners
         1. Jacobi Preconditioning
         2. Incomplete LU Decomposition (ILU)
         3. Multigrid Methods
            1. Algorithm Layers: Smoothing, Interpolation, Restriction
            2. V-cycle vs. W-cycle Approaches
      3. Impact on Iterative Methods
         1. Improvement in Convergence Rate
         2. Examples and Case Studies
      4. Challenges
         1. Computational Cost vs. Benefit Trade-off
         2. Applicability to Different Iterative Methods