Learners' Languages

In "Backprop as functor", the authors show that the fundamental elements of deep learning-gradient descent and backpropagation-can be conceptualized as a strong monoidal functor Para(Euc)& RARR; Learn from the category of parameterized Euclidean spaces to that of learners, a category developed explicitly to capture parameter update and backpropagation. It was soon realized that there is an isomorphism Learn & SIM;= Para(SLens), where SLens is the symmetric monoidal category of simple lenses as used in functional programming.In this note, we observe that SLens is a full subcategory of Poly, the category of polynomial functors in one variable, via the functor A 7 & RARR; AyA. Using the fact that (Poly, & OTIMES;) is monoidal closed, we show that a map A & RARR; B in Para(SLens) has a natural interpretation in terms of dynamical systems (more precisely, generalized Moore machines) whose interface is the internal-hom type [AyA,ByB].Finally, we review the fact that the category p-Coalg of dynamical systems on any p & ISIN; Poly forms a topos, and consider the logical propositions that can be stated in its internal language. We give gradient descent as an example, and we conclude by discussing some directions for future work.