Categorical Foundations of Gradient-Based Learning

Abstract

We propose a categorical foundation of gradientbased machine learning algorithms in terms of lenses, parametrised maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it encompasses a variety of gradient descent algorithms such as ADAM, AdaGrad, and Nesterov momentum, as well as a variety of loss functions such as as MSE and Softmax cross-entropy, shedding new light on their similarities and differences. Our approach also generalises beyond neural networks (modelled in categories of smooth maps), accounting for other structures relevant to gradient-based learning such as boolean circuits. Finally, we also develop a novel implementation of gradient-based learning in Python, informed by the principles introduced by our framework.Note the slightly counter-intuitive contravariance in the 2-cells which arises from the Grothendieck construction. This permits reindexing: given a morphism r : P ′ − → P in C we can reparameterize a given P -parameterized morphism to a P ′ -parameterized morphism. We often work with strict monoidal categories whose objects are natural numbers and whose monoidal product is addition. In such settings, Corollary 2.2. If C is a strict symmetric monoidal category, then Para(C) is a 2-category.We have shown how Para acts on some base category C. However, Para is also natural with respect to base change, i.e. given a functor F : C − → D, there is an induced functor Para(F ) : Para(C) − → Para(D): Proposition 2.3. Let C and D be strict symmetric monoidal categories. Let F : C − → D be a lax symmetric monoidal functor. Then there is an induced 2-functor Para(F ) : Para(C) − → Para(D)which agrees with F on objects.This 2-functor is straightforward: for a 1-cell (P, f ) : A − → B, it applies F to P and f and uses the (lax) comparison to get a map of the correct type. Lastly, we mention that Para(C) inherits the symmetric monoidal structure from C and that the induced 2-functor Para(F ) respects that structure. This will allow us to compose learners not only in series, but also in parallel. Cartesian reverse differential categoriesFundamental to all gradient-based learning is, of course, the gradient operation. In most cases this gradient operation is performed in the category of smooth maps between Euclidean spaces. However, recent work [7] has shown that gradient-based learning can also work well in other categories; for example, in a category of boolean circuits. Thus, to encompass these examples in a single framework, it is helpful to work in a category with an abstract gradient operation. Specifically, we will work in a Cartesian reverse differential category (first defined in [14]), a category in which every map has an associated reverse derivative. The reverse derivative is a generalization of the gradient operation: for example, in the category of smooth maps, the reverse derivative of a map of type f : R n − → R is essentially its derivative. Definition 2.4.

Diagrammatic Differentiation for Quantum Machine Learning

“…Differential categories [25] have been introduced to axiomatise the notion of derivative. More recently reverse derivative categories [26] generalised the notion of back-propagation, they have been proposed as a categorical foundation for gradient-based learning [27]. These frameworks all define the derivative of a morphism with respect to its domain.…”

Categorical composable cryptography

“…A special case also appears in the resource theory of contextuality as defined in [1], where one first defines deterministic free processes, and probabilistic (but classical) transformations d → e are then defined as transformations d ⊗ c → e where c is a non-contextual (and thus free) resource. This construction is discussed more generally in [27,38], but we modify it slightly by allowing one to choose a class of objects as "parameters" instead of taking that class to consist of all objects: this modification is important for resource theories as it lets one can control which resources are made freely available.…”

“…In Appendix C.3 we discuss a variant of a construction on monoidal categories, used in special cases in [35] and discussed in more detail in [27,38], that allows one to declare some resources free and thus enlarge the set of possible resource conversions.…”

Generalized Optimization: A First Step Towards Category Theoretic Learning Theory

“…Cockett et al (2019) introduce Cartesian reverse derivative categories in which we can define an operator that shares certain properties with reverse-mode automatic differentiation (RD.1 to RD.7 in Definition 13 of Cockett et al (2019)). Reverse derivative categories are remarkably general: the category of Euclidean spaces and differentiable functions between them is of course a reverse derivative category, as are the categories of polynomials over semirings and the category of Boolean circuits (Wilson and Zanasi, 2021).…”

“…Means and standard errors from 100 experiments of 10 polynomials each are shown. The code to run these experiments is on Github at tinyurl.com/ku3pjz56 Cruttwell et al (2021). explore categorical formulations of automatic differentiation.…”