Listen "Neural Net Dropout"
Episode Synopsis
Neural networks are complex models with many parameters and can be prone to overfitting. There's a surprisingly simple way to guard against this: randomly destroy connections between hidden units, also known as dropout. It seems counterintuitive that undermining the structural integrity of the neural net makes it robust against overfitting, but in the world of neural nets, weirdness is just how things go sometimes.
Relevant links: https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf
Relevant links: https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf
More episodes of the podcast Linear Digressions
Distillation, or, How to Steal a Model
27/07/2026
Still summer break: back next week
13/07/2026
Summer break: back soon
06/07/2026
