Benjamin Eisley L&S Math & Physical Sciences
Free Probability in Infinite Depth Neural Networks
In the past few years, neural networks have gone from obscure to ubiquitous. This technology is shockingly versatile, but conceptually ill-understood: there is a large gap between practice and theory, and much has yet to even be conjectured. For example, scientists are baffled by the overfitting paradox. Overfitting is usually a problem when programmers model a complex system such as the brain. Programmers must base their model on finitely many examples of that system’s behavior. Traditionally, programs that perfectly replicate these examples forget the underlying system. Surprisingly, large neural networks do not in general suffer from this deficiency.
Recent developments suggest that free probability, traditionally used to understand large random matrices, can be used to explain the ways in which large neural networks typically behave. Our project would use free probability to explain the overfitting paradox by describing the average behavior of highly trained neural networks.
Message To Sponsor
Dear Anselm MPS, Thank you so much for sponsoring my research project! This summer convinced me that, contrary to my self doubt, I am ready for graduate school. I learned that I love research: that I am happiest when I am doing research alongside others in pursuit of knowledge that will push our world forward. I intend to continue the project into the school year, and to use part of your sponsorship to travel to the Joint Mathematics Conference in January in order to present my findings to the rest of the research community. I owe you a deep debt of gratitude. Best Regards, Benjamin Eisley