Skip to main content
New

Backpropagation by Hand

Most people can describe backpropagation. This course makes you compute it — every derivative, every delta, every weight update — by hand, on real numbers, until the algorithm stops being magic.

Updated Jul 23, 2026

About this course

Most explanations of backpropagation skip the part that actually builds understanding. They give you the high-level story (chain rule, gradients, update rule) and then hand you a library that does the work. You leave feeling like you understand it, but you couldn't reproduce a single gradient if asked. This course takes the opposite approach. You start with a tiny network, specific numbers for weights and inputs, and you compute everything yourself: the forward pass, the loss, the output-layer delta, the hidden-layer gradients, and the weight updates. The goal is not to avoid using libraries. The goal is to get to a point where you know exactly what those libraries are doing. When a network fails to train, the people who debug it fastest are the ones who can reason about what the gradients must look like at each layer. That skill comes from having done the calculation yourself, slowly, with a pencil, at least once. This course gives you that experience across four progressively harder lessons, ending with a two-example mini-batch and a review of the mistakes that trip people up most often. You do not need a GPU, a Python environment, or a deep math background. You need the ability to multiply and add, some patience with notation, and a willingness to work through the same small network from multiple angles. By the end, you will have run a complete training step by hand and watched the loss go down. That moment changes how you see the algorithm.

Details

Last updated Jul 23, 2026
3 Units, 6 lessons
3 Assessments

Skills you'll gain with this course

Forward Pass Computation

Given weights, biases, and inputs, the student can compute every activation in a small feedforward network by hand using consistent layer notation.

Gradient Derivation

The student can apply the chain rule to derive the gradient of a loss with respect to any weight or bias in a two-layer network without software assistance.

Backpropagation Execution

The student can run a full backward pass on a small network, computing output-layer and hidden-layer deltas and producing explicit gradients for every parameter.

Weight Update Application

The student can apply the gradient descent update rule to every parameter in a network and verify that the update reduced the loss on the training example.

Mini-Batch Gradient Averaging

The student can compute per-example gradients for multiple inputs, average them correctly, and apply a single update that accounts for all examples.

Syllabus

3 Units • 6 Lessons • 3 Assessments

Ways To Learn Included

Every lesson enables you to learn in a variety of ways.

3 min read
587 words

These gases, such as carbon dioxide and methane, play a crucial role in regulating Earth's temperature. But what exactly are they, and how do they work? Let's find out.

Read
Carbon Dioxide
Flashcards
Quiz
What is the primary greenhouse gas responsible for trapping heat?
Carbon Dioxide
Locked In
Great job! That's the correct answer.
Quiz
The earth's atmosphere is composed
Lecture
Listen: Greenhouse gases explained
Podcast
Chat
0:05
Jam
Arcade
Explainer
Comic

FAQ

Course thumbnail