In pattern recognition, statistical modeling, or regression, the amount of data is a critical factor affecting the performance. If the amount of data and computational resources are unlimited, even trivial algorithms will converge to the optimal solution. However, in the practical case, given limited data and other resources, satisfactory performance requires sophisticated methods to regularize the problem by introducing a priori knowledge. Invariance of the output with respect to certain transformations of the input is a typical example of such a priori knowledge. We introduce the concept of tangent vectors, which compactly represent the essence of these transformation invariances, and two classes of algorithms, tangent distance and tangent propagation, which make use of these invariances to improve performance.
In pattern recognition, statistical modeling, or regression, the amount of data is the most critical factor a ecting the performance. If the amount of data and computational resources are near in nite, many algorithms will provably converge to the optimal solution. When this is not the case, one has to introduce regularizers and a-priori knowledge to supplement t h e a vailable data in order to boost the performance. Invariance (or known dependence) with respect to transformation of the input is a frequent occurrence of such a-priori knowledge. In this chapter, we i n troduce the concept of tangent v ectors, which compactly represent the essence of these transformation invariances, and two classes of algorithms, \Tangent distance" and \Tangent propagation", which make u s e of these invariances to improve performance.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.