Daniela was working on "The Lagrangian of Learning: Lagrangian Mechanics in Neural Network Training" supervised by Samuel Tovey. We congratulate her on this wonderful achievement and invite you to attend the defense.
Abstract: Gradient-based optimizers used to train neural networks such as stochastic gradient descent (SGD), heavy-ball momentum, and Adam, are typically introduced as separate methods for stochastic optimization, rather than derived from a common physical principle. In this thesis, I show that these optimizers arise as trajectories of a single Lagrangian, in which the network parameters are treated as the position of a particle moving within the potential of the loss. This identification makes testable predictions: training should conserve energy when there is no friction, and adding friction should match the behavior of a damped oscillator. I confirm both predictions experimentally on different models, and use the same framework to design two adaptive friction schemes that damp training only when needed which improves training measurably.
Best of luck Daniela on her future endeavors!
