Toronto, Canada
University of Toronto & Vector Institute
Toronto kept neural networks alive through the winter, then detonated the current era with one ImageNet result.
Key findings
- 2012
AlexNet
Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton
A deep CNN trained on two consumer GPUs cut ImageNet top-5 error from 26% to 15.3%.
Why it matters here · The result that made GPUs the substrate of AI and started the compute build-out we now track as its own segment.
26% → 15.3% top-5 error - 1986
Learning representations by back-propagating errors
Rumelhart, Hinton, Williams
Popularised gradient descent through multi-layer networks via the chain rule.
Why it matters here · Still the training algorithm for every model in this catalog, at every scale.
- 2012
Dropout
Srivastava, Hinton et al.
Randomly zeroing units during training as an implicit ensemble, sharply reducing overfitting.
Why it matters here · Made deep nets trainable on modest data — standard in almost every architecture that followed.
On the history timeline
Milestones on the 1943 → today timeline credited to this institution.
- 2012AlexNet — the deep learning big bangAlex Krizhevsky, Ilya Sutskever & Geoffrey HintonDeep learning boom