As will be clearly illustrated in this track, machine learning is a great tool we use for physics research. In this talk. I will view machine learning and the theoretical questions that surround it as an object to be studied using physics tools. This direction, in fact, has decades long tradition of building and solving simple models of neural networks. I will give examples of recent works that build on powerful methods of physics of disordered systems to analyze the gradient descent algorithm in a high dimensional non-convex setting, and design generative models for data for which optimal generalization curves are obtained in closed form.