Spurious Vanishing Problem in Approximate Vanishing Ideal

Abstract

Approximate vanishing ideal is a concept from computer algebra that studies the algebraic varieties behind perturbed data points. To capture the nonlinear structure of perturbed points, the introduction of approximation to exact vanishing ideals plays a critical role. However, such an approximation also gives rise to a theoretical problem-the spurious vanishing problem-in the basis construction of approximate vanishing ideals; namely, obtained basis polynomials can be approximately vanishing simply because of the small coefficients. In this paper, we propose a first general method that enables various basis construction algorithms to overcome the spurious vanishing problem. In particular, we integrate coefficient normalization with polynomial-based basis constructions, which do not need the proper ordering of monomials to process for basis constructions. We further propose a method that takes advantage of the iterative nature of basis construction so that computationally costly operations for coefficient normalization can be circumvented. Moreover, a coefficient truncation method is proposed for further accelerations. From the experiments, it can be shown that the proposed method overcomes the spurious vanishing problem, resulting in shorter feature vectors while sustaining comparable or even lower classification error.INDEX TERMS Approximate commutative algebra, approximate vanishing ideal, machine learning.

Gradient Boosts the Approximate Vanishing Ideal

“…Our idea of using the gradient is general enough to be integrated with existing monomial-order-free algorithms. However, to avoid a unnecessarily abstract discussion, we focus on the Simple Basis Construction (SBC) algorithm (Kera and Hasegawa 2019), which was proposed by (Kera and Hasegawa 2019) based on Vanishing Component Analysis (VCA; Livni et al 2013). Most monomial-order-free algorithms can be discussed using SBC; thus, the following discussion is sufficiently general.…”

“…Step 2, we have a normalization matrix N(C t ) ∈ R |Ct|×|Ct| to resolve the spurious vanishing problem (Kera and Hasegawa 2019). For the coefficient normalization, the coefficient vector 5 of c i is denoted by n c (c i ), and the (i, j)th entry of N(C t ) is n c (c i ) n c (c j ).…”

“…(2) with this normalization matrix leads to polynomials C t v 1 , ..., C t v |Ct| at Step 3, which are normalized with respect to their coefficient vectors, i.e., ∀i, n c (C t v i ) = 1. Instead of n c , we can also define the normalization matrix from another mapping as long as it satisfies some requirements (Kera and Hasegawa 2019). Later, we propose a novel mapping n g , which is based on the gradient of a given polynomial.…”

“…(2) in Step 2, the basis polynomials of vanishing polynomials and nonvanishing polynomials (say, h) are normalized such that n g (h; X) = 1. Conceptually, this rescales h with respect to the gradient norm as h/ n g (h; X) , but in an optimal way (Kera and Hasegawa 2019). The gradient normalization is superior to the coefficient normalization in terms of computational cost; the former works in polynomial time complexity (cf., Proposition 2) and the latter requires exponential time complexity.…”

“…To sidestep this, g needs to be normalized by some scale. However, intuitive coefficient normalization is exponentially costly for monomial-order-free algorithms (Kera and Hasegawa 2019). Inconsistency of output with respect to the translation and scaling of input-Given translated or scaled data, the output of the basis construction can drastically change in terms of the number of polynomials and their nonlinearity, regardless of how well the parameter is chosen.…”