Introduction

0:00 Their efforts led to back propagation with gradient descent, the workhorse algorithm that powers virtually the entire field of machine learning today. Due to its remarkable success, researchers began to speculate that perhaps brains do something similar. However, there is a fundamental problem. The back propagation algorithm contradicts essential biological principles of brain function, making its exact implementation in neural tissue virtually impossible. In recent years, however, an alternative algorithm called predictive coding has emerged that is not only more aligned with the brain's biological hardware, but sometimes can work even better than the back propagation itself.

0:46 In this video, we will build predictive coding from first principles. Explore what issues of biological plausibility it addresses and how it might inspire the next revolution in machine learning. The fundamental challenge that computational systems must solve is called credit assignment. When you have a system with numerous parameters like connection weights between neurons that can be adjusted to achieve a desired output such as recognizing objects in an image or executing appropriate actions.