Evaluation

Tutorials/Quizzes - 10%

Midsem - 25%

Endsem - 40%

Lab

1. Assignments - 25%

Resources

S.N	Topic	Slides	Notes	External References
1.	Introduction	Slides	Notes	Cost function of neural network is non-convex?
2.	Basic Calculus	Slides	Notes	How does autograd work? Difference between True and Numerical Gradient
3.	Universal Approximation Theorem-Back Propagation		Notes	Theoritical proof for UAT Visual Proof for UAT
4.	Auto Differentiation - Why Backpropagation?	Slides	Notes	Auto Differentiation A Step-by-step Introduction to the Implementation of Automatic Differentiation
5.	Jacobian and Hessian Matrix	Slides	Notes	Jacobian and Hessian How are Jacobian and Hessian Matrices used in ML
6.	Convexset and Convex Function	Slides	Notes	Understanding Convexity
7.	Taylor's Approximation	Slides	Notes	Taylor's Theorem Visualization (Change (a,N) and Observe)
8.	Quadratic Approximation		Notes	Quadratic Approximation using Hessian
9.	Equivalent Definitions for Convex Function		Notes	Practice Problems for Convex Function
10.	Convex Function - Continued	Slides	Notes	Self Practice Problems-1 Self Practice Problems-2 Self Practice Problems-3
11.	GD with Linesearch	Slides	Notes	Exact line search and backtracking Slides from Prof.Boyd-Backtracking Line Search and Exact Line Search
12.	GD with Backtracking	Slides	Notes Notes-II	Backtracking Line Search
13.	Stochastic Gradient Descent	Slides	Notes	GD Vs SGD Vs BatchGD Why Mini Batch works
14.	Gradient Descent - Convergence Analyis		Notes	GD Rate Analysis
15.	Gradient Descent with Momentum-NAG	Slides	Notes	Why Momentum Really Works Visual Comparison of Optimizers
16.	RMS Prop and Adagrad		Notes Examples	Hinton's slides on RMSprop
17.	ADAM	Slides	Notes	Summary of Optimizers Overview of Optimizers ADAM-Research Paper
Self Practice Tutorial-IIIlink
18.	ADAMW		Notes	ADAMW Vs ADAM
19.	Second Order Gradient Descent Method	Slides	Notes	Newton-Method
20.	Second Order Descent Method-II		Notes	Sophia-Paper
21.	Batch Normalization - Intialization			Batch Normalization
22.	Linear Programming - Duality	Slides from External Source	Notes	Duality - Linear Programming Duality - Linear Programming -II
23.	Linear Programming - Duality - II	Slides from External Source	Notes	Practice Problems
24.	Lagrange Multipiers - Duality	Slides	Notes	Practice Problems-Lagrnage Multipliers Practice Problems-Lagrnage Multipliers
25.	Dual SVM	21st lect -slides from ISL	Notes	KKT Conditions
25.	Sub Gradient	Slides	Notes	Problems-Sub gradients Sub gradients
26.	Sub Gradient - Practice Problems		Problems Solutions

Assignments&Tutorials

S.N	Material	Due Date
1.	Assignment-0	08/02/2025
2.	Tutorial-1	-
3.	Tutorial-2	-
4.	Assignment-1	28/02/2025
5.	quiz-1 solutions	-
6.	Tutorial-3	-
5.	Assignment-2	07/04/2025

Last year's course page

Link

Feedback form

Link