• Instructor: Parimala Kancharla: (office: A17.03.06, email: parimala@iitmandi.ac.in)
  • Office Hours: (or by appointment)
  • Discussion Forum: Piazza code-ds411 ,To Join
  • Credit Structure: 3-1-0-4
  • Class Venue: A17-1B
  • Class Timings: F-slot
  • Syllabus
  • TAs
    1. Pallav Dwivedi (s23104@students.iitmandi.ac.in)
    2. Siddharath Shakya (s23048@students.iitmandi.ac.in)
    3. Kriti Khare
    4. Mridul Sharma
    5. Anika Srivasthava
    6. Soujatya Sarkar
    7. Abhishek Dileep

    Evaluation

  • Tutorials/Quizzes - 10%
  • Midsem - 25%
  • Endsem - 40%
  • Lab
    1. Assignments - 25%

    Resources

    S.N Topic Slides Notes External References
    1. Introduction Slides Notes
    1. Cost function of neural network is non-convex?
    2. Basic Calculus Slides Notes
    1. How does autograd work?
    2. Difference between True and Numerical Gradient
    3. Universal Approximation Theorem-Back Propagation Notes
    1. Theoritical proof for UAT
    2. Visual Proof for UAT
    4. Auto Differentiation - Why Backpropagation? Slides Notes
    1. Auto Differentiation
    2. A Step-by-step Introduction to the Implementation of Automatic Differentiation
    5. Jacobian and Hessian Matrix Slides Notes
    1. Jacobian and Hessian
    2. How are Jacobian and Hessian Matrices used in ML
    6. Convexset and Convex Function Slides Notes
    1. Understanding Convexity
    7. Taylor's Approximation Slides Notes
    1. Taylor's Theorem Visualization (Change (a,N) and Observe)
    8. Quadratic Approximation Notes
    1. Quadratic Approximation using Hessian
    9. Equivalent Definitions for Convex Function Notes
    1. Practice Problems for Convex Function
    10. Convex Function - Continued Slides Notes
    1. Self Practice Problems-1
    2. Self Practice Problems-2
    3. Self Practice Problems-3
    11. GD with Linesearch Slides Notes
    1. Exact line search and backtracking
    2. Slides from Prof.Boyd-Backtracking Line Search and Exact Line Search
    12. GD with Backtracking Slides Notes Notes-II
    1. Backtracking Line Search
    13. Stochastic Gradient Descent Slides Notes
    1. GD Vs SGD Vs BatchGD
    2. Why Mini Batch works
    14. Gradient Descent - Convergence Analyis Notes
    1. GD Rate Analysis
    15. Gradient Descent with Momentum-NAG Slides Notes
    1. Why Momentum Really Works
    2. Visual Comparison of Optimizers
    16. RMS Prop and Adagrad Notes Examples
    1. Hinton's slides on RMSprop
    17. ADAM Slides Notes
      Summary of Optimizers
    1. Overview of Optimizers
    2. ADAM-Research Paper
    Self Practice Tutorial-IIIlink
    18. ADAMW Notes
    1. ADAMW Vs ADAM
    19. Second Order Gradient Descent Method Slides Notes
    1. Newton-Method
    20. Second Order Descent Method-II Notes
    1. Sophia-Paper
    21. Batch Normalization - Intialization
    1. Batch Normalization
    22. Linear Programming - Duality Slides from External Source Notes
    1. Duality - Linear Programming
    2. Duality - Linear Programming -II
    23. Linear Programming - Duality - II Slides from External Source Notes
    1. Practice Problems
    24. Lagrange Multipiers - Duality Slides Notes
    1. Practice Problems-Lagrnage Multipliers
    2. Practice Problems-Lagrnage Multipliers
    25. Dual SVM 21st lect -slides from ISL Notes
    1. KKT Conditions
    25. Sub Gradient Slides Notes
    1. Problems-Sub gradients
    2. Sub gradients
    26. Sub Gradient - Practice Problems Problems Solutions
    26-1 Adversarial Robustness - Applications Slides
    27. Slater's Condition Notes
    1. External References
    28. Distrubuted ML Notes
    1. Gpipe Paper
    2. Parallel DL Paper-1
    3. Parallel DL Paper-2
    4. Parallel DL Paper-3
    29. ADMM and Dual Ascent

    Last year's course page

  • Link

    • Feedback form

  • Link