Evaluation

Tutorials/Quizzes - 10%

Midsem - 25%

Endsem - 40%

Lab

1. Assignments - 25%

Resources

S.N	Topic	Slides	Notes	External References
1.	Introduction	Slides	Notes	Cost function of neural network is non-convex?
2.	Basic Calculus	Slides	Notes	How does autograd work? Difference between True and Numerical Gradient
3.	Universal Approximation Theorem-Back Propagation		Notes	Theoritical proof for UAT Visual Proof for UAT
4.	Auto Differentiation - Why Backpropagation?	Slides	Notes	Auto Differentiation A Step-by-step Introduction to the Implementation of Automatic Differentiation
5.	Jacobian and Hessian Matrix	Slides	Notes	Jacobian and Hessian How are Jacobian and Hessian Matrices used in ML
6.	Convexset and Convex Function	Slides	Notes	Understanding Convexity
7.	Taylor's Approximation	Slides	Notes	Taylor's Theorem Visualization (Change (a,N) and Observe)
8.	Quadratic Approximation		Notes	Quadratic Approximation using Hessian
9.	Equivalent Definitions for Convex Function		Notes	Practice Problems for Convex Function
10.	Convex Function - Continued	Slides	Notes	Self Practice Problems-1 Self Practice Problems-2 Self Practice Problems-3
11.	GD with Linesearch	Slides	Notes	Exact line search and backtracking Slides from Prof.Boyd-Backtracking Line Search and Exact Line Search
12.	GD with Backtracking	Slides	Notes Notes-II	Backtracking Line Search
13.	Stochastic Gradient Descent	Slides	Notes	GD Vs SGD Vs BatchGD Why Mini Batch works
14.	Gradient Descent - Convergence Analyis		Notes	GD Rate Analysis
15.	Gradient Descent with Momentum-NAG	Slides	Notes	Why Momentum Really Works Visual Comparison of Optimizers
16.	RMS Prop and Adagrad		Notes Examples	Hinton's slides on RMSprop
17.	ADAM	Slides	Notes	Summary of Optimizers Overview of Optimizers ADAM-Research Paper
Self Practice Tutorial-IIIlink
18.	ADAMW		Notes	ADAMW Vs ADAM

Assignments&Tutorials

S.N	Material	Due Date
1.	Assignment-0	08/02/2025
2.	Tutorial-1	-
3.	Tutorial-2	-
4.	Assignment-1	28/02/2025
5.	quiz-1 solutions	-

Last year's course page

Link

Feedback form

Link