Title: Scaling predictive coding with μP
Abstract: Predictive coding is a promising local alternative to backpropagation, but its scalability to modern deep networks has remained unclear. This talk presents a unified perspective on scaling predictive coding networks through μP-inspired parameterisations: how the right scalings make very deep predictive coding networks trainable in practice, and why these scalings arise theoretically in the infinite-width and infinite-depth regimes. For linear networks that are much wider than they are deep, the theory shows that predictive coding recovers backpropagation-like gradients under the appropriate limits, while experiments suggest that these insights extend to nonlinear networks. Together, these results show that predictive coding can be made stable at depth, transfer hyperparameters across scales, and offer a principled path toward scalable local learning.
This talk is based on the two recent papers “MuPC: Scaling Predictive Coding to 100+ layers” https://arxiv.org/pdf/2505.13124 and “On the Infinite Width and Depth Limits of Predictive Coding Networks” https://arxiv.org/pdf/2602.07697.