Data mining in space physics: MineTool algorithm

Karimabadi, H.; Sipes, T.; White, Halbert; Marinucci, Massimiliano; Дмитриев, А. В.; Chao, J. K.; Driscoll, Jonathan; Balac, Natasha

doi:10.1029/2006ja012136

Cited by 10 publications

(13 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Automated techniques have also been used to identify magnetopause flux transfer events (FTEs) via bipolar field deflections [e.g., Kawano and Russell, 1996], albeit still requiring manual confirmation. Karimabadi et al [2007] introduced a data mining technique (MineTool) and later applied it to locate FTEs [Karimabadi et al, 2009], using a combination of magnetic field and plasma data. Recently, Malaspina and Gosling [2012] refined a technique to identify rotational discontinuities in the STEREO solar wind data using the gradient of the magnetic field (the method was initially developed by Vasquez et al [2007]).…”

Section: Introductionmentioning

confidence: 99%

Automated force‐free flux rope identification

et al. 2017

View full text Add to dashboard Cite

We describe a method developed to automatically identify quasi force‐free magnetotail flux ropes from in situ spacecraft magnetometer data. The method locates significant (greater than 1σ) deflections of the north‐south component of the magnetic field coincident with enhancements in other field components. The magnetic field data around the deflections are then processed using Minimum Variance Analysis (MVA) to narrow the selection down to those that exhibit the characteristics of flux ropes. The subset of candidates that fulfills the requirements are then compared to a cylindrical, linear (constant‐α) force‐free model. Those that can be well approximated as force free are then accepted. The model fit also provides a measure of the physical parameters that describe the flux rope (i.e., core field and radius). This process allows for the creation of a repeatable, consistent catalog of flux ropes. Automation allows a greater volume of data to be covered, saving time and allowing the exploration of potential selection biases. The technique is applied to MESSENGER (MErcury Surface, Space ENvironment, GEochemistry, and Ranging) magnetometer data in the Hermean magnetotail and successfully locates flux ropes, some of which match previously known encounters. Assumptions of the method and potential future applications are discussed.

show abstract

Section: Introductionmentioning

confidence: 99%

Automated force‐free flux rope identification

et al. 2017

View full text Add to dashboard Cite

show abstract

“…(I ) Clustering the unlabeled treatment data (67% of all the data, 33% was set aside for testing) using a Gaussian mixture model (using Expectation Maximization algorithm), (II) Adding a new labeled variable cluster with a label equals the optimal number of clusters found in the previous step, (III) Feeding this newly labeled data into a classification method called MineTool [61][62][63] resulting in a predictive model of medical error (PMME) (IV) Dynamically updating the model by repeating the previous three steps (i)-(iii) on 100 percent of the data, whereby simulating new data instances being collected through the oncology information system.…”

Section: Smart-tool For Anomaly Detection In Radiotherapy Treatmementioning

confidence: 99%

Anomaly Detection in Time Series Radiotherapy Treatment Data

Sipes

Karimabadi²,

Jiang

et al. 2014

2014 IEEE International Conference on Semantic Computing

View full text Add to dashboard Cite

The work presented here resulted in a valuable innovative technology tool for automatic detection of catastrophic errors in cancer radiotherapy, adding an important safeguard for patient safety. We designed a tool for Dynamic Modeling and Prediction of Radiotherapy Treatment Deviations from Intended Plans (SmartTool) to automatically detect and highlight potential errors in a radiotherapy treatment plan, based on the data from several thousand prostate cancer treatments at Moore Cancer Research Center at University of California San Diego. SmartTool determines if the treatment parameters are valid, against a previously built Predictive Model of a Medical Error (PMME). SmartTool has the following main features: 1) It communicates with a radiotherapy treatment management system, checking all the treatment parameters in the background prior to execution, and after the human expert QA is completed, 2) The anomalous treatment parameters, if any, are detected using an innovative intelligent algorithm in a completely automatic and unsupervised manner, 3) It is a self-learning and constantly evolving system; the model is dynamically updated with the new treatment data, 4) It incorporates expert knowledge through the feedback loop of the dynamic process which updates the model with any new false positives (FP) and false negatives (FN) ; 4) When an outlier treatment parameter is detected, SmartTool works by preventing the plan execution and highlighting the parameter for human intervention; 5) It is aimed at catastrophic errors, not small errors.

show abstract

“…Kadous achieved fairly to very accurate classification results in the domains of ECG (electrocardiograph) diagnosis (72% accuracy) and the sign language recognition (98% accuracy). We build on TClass for the physics domain application by (1) improving the feature collection and segmentation paradigms, (2) adding more features, and (3) utilizing MineTool [ Karimabadi et al , 2007] as the classification method of choice, which has the capacity to outperform standard data mining tools such as decision trees, artificial neural networks [ Ripley , 1996] and support vector machines [ Cortes and Vapnik , 1995].…”

Section: Minetool‐ts Algorithmmentioning

confidence: 99%

“…One is that potentially useful candidates get overlooked simply because there are too many variables to evaluate. Another is that if a systematic routine for evaluating and including variables is used, it can lead to overfit [ Karimabadi et al , 2007]. Finally, many candidate variables are likely to be redundant, which can cause difficulties for the estimation routines.…”

Section: Minetool‐ts Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

A new multivariate time series data analysis technique: Automated detection of flux transfer events using Cluster data

et al. 2009

Self Cite

View full text Add to dashboard Cite

[1] A new data mining technique called MineTool-TS is introduced which captures the time-lapse information in multivariate time series data through extraction of global features and metafeatures. This technique is developed into a JAVA-based data mining software which automates all the steps in the model building to make it more accessible to nonexperts. As its first application in space sciences, MineTool-TS is used to develop a model for automated detection of flux transfer events (FTEs) at Earth's magnetopause in the Cluster spacecraft time series data. The model classifies a given time series into one of three categories of non-FTE, magnetosheath FTE, or magnetospheric FTE. One important feature of MineTool-TS is the ability to explore the importance of each variable or combination of variables as indicators of FTEs. FTEs have traditionally been identified on the basis of their magnetic field signatures, but here we find that some plasma variables can also be effective indicators of FTEs. For example, the perpendicular ion temperature yields a model accuracy of $93%, while a model based solely on the normal magnetic field B N yields an accuracy of $95%. This opens up the possibility of searching for more unusual FTEs that may, for example, have no clear B N signature and create a more comprehensive and less biased list of FTEs for statistical studies. We also find that models using GSM coordinates yield comparable accuracy to those using boundary normal coordinates. This is useful since there are regions where magnetopause models are not accurate. Another surprising result is the finding that the algorithm can largely detect FTEs, and even distinguish between magnetosheath and magnetospheric FTEs, solely on the basis of models built from single parameters, something that experts may not do so straightforwardly on the basis of short time series intervals. The most accurate models use a combination of plasma and magnetic field variables and achieve a very high accuracy of prediction of $99%. We explain the high detection accuracies both in terms of the existence of clear physical signatures of FTEs (for the majority of cases) and in terms of the capability of the data mining technique to explore the data set in a much more thorough fashion than expert human eyes.

show abstract

Data mining in space physics: MineTool algorithm

Cited by 10 publications

References 7 publications

Automated force‐free flux rope identification

Automated force‐free flux rope identification

Anomaly Detection in Time Series Radiotherapy Treatment Data

A new multivariate time series data analysis technique: Automated detection of flux transfer events using Cluster data

Contact Info

Product

Resources

About