Benjamin Eisley L&S Math & Physical Sciences
Free Probability in Infinite Depth Neural Networks
In the past few years, neural networks have gone from obscure to ubiquitous. This technology is shockingly versatile, but conceptually ill-understood: there is a large gap between practice and theory, and much has yet to even be conjectured. For example, scientists are baffled by the overfitting paradox. Overfitting is usually a problem when programmers model a complex system such as the brain. Programmers must base their model on finitely many examples of that system’s behavior. Traditionally, programs that perfectly replicate these examples forget the underlying system. Surprisingly, large neural networks do not in general suffer from this deficiency.
Recent developments suggest that free probability, traditionally used to understand large random matrices, can be used to explain the ways in which large neural networks typically behave. Our project would use free probability to explain the overfitting paradox by describing the average behavior of highly trained neural networks.
Message To Sponsor
I am looking forward to exploring free probability this summer! This project will let me use the abstract mathematics I have learned in my classes to contribute to our understanding of real world neural networks. I am incredibly grateful for this opportunity to do research - this is a major step towards my dream of becoming a mathematician who helps society!