Approximation with polynomial kernels and SVM classifiers

Zhou, Ding‐Xuan; Jetter, Kurt

doi:10.1007/s10444-004-7206-2

Cited by 124 publications

(79 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such trees yield faster rates but they are computationally prohibitive. Recent risk bounds for polynomial-kernel support vector machines may offer a computationally tractable alternative to this approach [19]. One way or another, we feel that dyadic decision trees, or possibly new variants thereof, hold promise to address these issues.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Minimax-optimal classification with dyadic decision trees

Scott

Nowak

2006

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Decision trees are among the most popular types of classifiers, with interpretability and ease of implementation being among their chief attributes. Despite the widespread use of decision trees, theoretical analysis of their performance has only begun to emerge in recent years. In this paper it is shown that a new family of decision trees, dyadic decision trees (DDTs), attain nearly optimal (in a minimax sense) rates of convergence for a broad range of classification problems. Furthermore, DDTs are surprisingly adaptive in three important respects: They automatically (1) adapt to favorable conditions near the Bayes decision boundary; (2) focus on data distributed on lower dimensional manifolds; and (3) reject irrelevant features. DDTs are constructed by penalized empirical risk minimization using a new data-dependent penalty and may be computed exactly with computational complexity that is nearly linear in the training sample size. DDTs are the first classifier known to achieve nearly optimal rates for the diverse class of distributions studied here while also being practical and implementable. This is also the first study (of which we are aware) to consider rates for adaptation to intrinsic data dimension and relevant features.C. Scott is with the

show abstract

Section: Discussionmentioning

confidence: 99%

“…Other authors derive rates for existing practical discrimination rules, but these works are not comparable to ours, since different distributional assumptions or loss functions are considered. [15], [16], [17], [18], [19].…”

Section: B Rates Of Convergence In Classificationmentioning

confidence: 99%

Minimax-optimal classification with dyadic decision trees

Scott

Nowak

2006

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…The main advantage of this multi-kernel algorithm is to improve the regularization error by using varying hypothesis spaces (see [31,16,32]). The error analysis for this multi-kernel setting can be done in the same way if the covering number of σ∈Σ f ∈ H K σ : f K σ ≤ 1 satisfies (1.8).…”

Section: Recall the Set W(r) Defined By (42) Proposition 42 Immedimentioning

confidence: 99%

Learning Rates of Least-Square Regularized Regression

2005

Self Cite

View full text Add to dashboard Cite

This paper considers the regularized learning algorithm associated with the leastsquare loss and reproducing kernel Hilbert spaces. The target is the error analysis for the regression problem in learning theory. A novel regularization approach is presented, which yields satisfactory learning rates. The rates depend on the approximation property and the capacity of the reproducing kernel Hilbert space measured by covering numbers. When the kernel is C ∞ and the regression function lies in the corresponding reproducing kernel Hilbert space, the rate is m −ζ with ζ arbitrarily close to 1, regardless of the variance of the bounded probability distribution.Short Title: Least-square Regularized Regression

show abstract

“…(see, e.g. [2], [3], [5], [27] ) Let V be the p-norm hinge loss function in (6) and f z,λ be defined by (16). Then, for any f ∈ (H¯t m K m ) k , E(f z,λ ) − E(f M ) can be bounded by…”

Section: The Approximation Errormentioning

confidence: 99%