Multiple comparison procedures applied to model selection

Pizarro, J.; Guerrero, Elisa; Galindo, Pedro L.

doi:10.1016/s0925-2312(01)00653-1

Cited by 70 publications

(41 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A nonparametric Friedman's test was performed on the accuracies for total larvae across folds due to unequal variances and confirmed results were not significant between age groups (c 2,5 = 4.77, P = 0.092). It should be noted that this statistical test does not exactly conform to those presented for model selection in the literature (Vazquez et al 2001;Pizarro et al 2002). Those examples performed analyses on test data from the same source, and our test data were composed of different images across folds for each training set.…”

Section: Thompson Et Almentioning

confidence: 99%

Semi‐automated image analysis for the identification of bivalve larvae from a Cape Cod estuary

Thompson

Hare

Gallager

2012

Limnology & Ocean Methods

View full text Add to dashboard Cite

Machine-learning methods for identifying planktonic organisms are becoming well-established. Although similar morphologies among species make traditional image identification methods difficult for larval bivalves, species-specific shell birefringence patterns under polarized light permit identification by color and texturebased features. This approach uses cross-polarized images of bivalve larvae, extracts Gabor and color angle features from each image, and classifies images using a Support Vector Machine. We adapted this method, which was established on hatchery-reared larvae, to identify bivalve larvae from a series of field samples from a Cape Cod estuary in 2009. This method had 98% identification accuracy for four hatchery-reared species. We used a multiplex polymerase chain reaction (PCR) method to confirm field identifications and to compare accuracies to the software classifications. Image classification of larvae collected in the field had lower accuracies than both the classification of hatchery species and PCR-based identification due to error in visually classifying unknown larvae and variability in larval images from the field. A six-species field training set had the best correspondence to our visual classifications with 75% overall agreement and individual species agreements from 63% to 88%. Larval abundance estimates for a time-series of field samples showed good correspondence with visual methods after correction. Overall, this approach represents a cost-and time-saving alternative to molecular-based identifications and can produce sufficient results to address long-term abundance and transport-based questions on a species-specific level, a rarity in studies of bivalve larvae.

show abstract

Section: Thompson Et Almentioning

confidence: 99%

Semi‐automated image analysis for the identification of bivalve larvae from a Cape Cod estuary

Thompson

Hare

Gallager

2012

Limnology & Ocean Methods

View full text Add to dashboard Cite

show abstract

“…Some procedures for solving question 8 such as the 5 × 2 cv test are given by Dietterich (1998), further investigated by Alpaydin (1999) and applied in a benchmark study on ensemble methods (Dietterich 2000). Pizarro, Guerrero, and Galindo (2002) suggest to use some classical multiple test procedures for solving this problem. Mixed models are applied for the comparison of algorithms across benchmark problems (for example Lim, Loh, and Shih 2000;Kim and Loh 2003).…”

Section: Introductionmentioning

confidence: 99%

The Design and Analysis of Benchmark Experiments

Hothorn

Leisch

Zeileis

et al. 2005

Journal of Computational and Graphical Statistics

167

139

View full text Add to dashboard Cite

The assessment of the performance of learners by means of benchmark experiments is an established exercise. In practice, benchmark studies are a tool to compare the performance of several competing algorithms for a certain learning problem. Cross-validation or resampling techniques are commonly used to derive point estimates of the performances which are compared to identify algorithms with good properties. For several benchmarking problems, test procedures taking the variability of those point estimates into account have been suggested. Most of the recently proposed inference procedures are based on special variance estimators for the cross-validated performance.We introduce a theoretical framework for inference problems in benchmark experiments and show that standard statistical test procedures can be used to test for differences in the performances. The theory is based on well defined distributions of performance measures which can be compared with established tests. To demonstrate the usefulness in practice, the theoretical results are applied to regression and classification benchmark studies based on artificial and real world data.

show abstract

“…En cuanto a la elección del diccionario dentro del MOF, se observa que las diferencias de los resultados no son estadísticamente significativas cuando se realiza una prueba de selección del modelo de Kruskal-Wallis (Pizarro, Guerrero, & Galindo, 2002). Esto es positivo porque la metodología no queda sujeta a la elección del diccionario.…”

Section: Discusión General De Los Resultadosunclassified

Representación óptima de señales MER aplicada a la identificación de estructuras cerebrales durante la estimulación cerebral profunda

Cardona

Álvarez

Gutiérrez

2015

View full text Add to dashboard Cite

InvestIgacIón ResumenLa identificación de señales cerebrales provenientes de microelectrodos de registro (MER) es un procedimiento clave en la estimulación cerebral profunda (DBS en inglés) aplicada en pacientes con enfermedad de Parkinson (EP). El propósito de esta investigación es identificar con alta precisión una estructura cerebral llamada núcleo subtalámico (STN), ya que es la estructura objetivo donde se logran los mejores resultados terapéuticos de la DBS. Para ello, se presenta un enfoque de representación óptima de señales MER mediante el método de Frames (MOF por sus siglas en inglés), con el cual se obtienen coeficientes que minimizan la norma Euclidiana de orden 2. A partir de los coeficientes óptimos se realiza una extracción de características de las señales combinando diccionarios wavelet packet y coseno. Para tener un marco de comparación con el estado del arte, también se caracterizan las señales utilizando la transformada wavelet discreta (DWT) con diferentes funciones madre. La metodología propuesta se valida en una base de datos real, y se emplean máquinas de aprendizaje supervisadas simples, como el clasificador K-Nearest Neighbors (K-NN), el clasificador lineal bayesiano (LDC) y el cuadrático (QDC). Los resultados de clasificación que se obtienen con el método propuesto mejoran significativamente el rendimiento alcanzado con la DWT, de manera que se logra una identificación positiva del STN superior al 97,6%. Los índices de identificación logrados por el MOF son muy precisos, ya que potencialmente se puede obtener una tasa de falsos positivos menores al 2% durante la DBS. Palabras clave: aprendizaje de máquina, enfermedad de Parkinson, estimulación cerebral profunda, procesamiento digital de señales, señales MER. AbstractIdentification of brain signals from microelectrode recordings (MER) is a key procedure during deep brain stimulation (DBS) applied in Parkinson's disease patients. The main purpose of this research work is to identify with high accuracy a brain structure called subthalamic nucleus (STN), since it is the target structure where the DBS achieves the best therapeutic results. To do this, we present an approach for optimal representation of MER signals Representación óptima de señales MER aplicada a la identificación de estructuras cerebrales durante la estimulación cerebral profunda

show abstract

Multiple comparison procedures applied to model selection

Cited by 70 publications

References 14 publications

Semi‐automated image analysis for the identification of bivalve larvae from a Cape Cod estuary

Semi‐automated image analysis for the identification of bivalve larvae from a Cape Cod estuary

The Design and Analysis of Benchmark Experiments

Representación óptima de señales MER aplicada a la identificación de estructuras cerebrales durante la estimulación cerebral profunda

Contact Info

Product

Resources

About