Large-scale mode identification and data-driven sciences

Mukhopadhyay, Subhadeep

doi:10.1214/17-ejs1229

Cited by 23 publications

(30 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This article provides a pragmatic and comprehensive framework for nonlinear time series modeling that is easier to use, more versatile and has a strong theoretical foundation based on recently developed theory on unified algorithms of data science via LP modeling (Parzen and Mukhopadhyay, 2012, 2013a,b, Mukhopadhyay and Parzen, 2014, Mukhopadhyay, 2016, 2017. Summary and broader implications of the proposed research:…”

Section: Resultsmentioning

confidence: 98%

See 1 more Smart Citation

Nonlinear Time Series Modeling: A Unified Perspective, Algorithm and Application

Mukhopadhyay

Parzen

2018

JRFM

Self Cite

View full text Add to dashboard Cite

A new comprehensive approach to nonlinear time series analysis and modeling is developed in the present paper. We introduce novel data-specific mid-distribution based Legendre Polynomial (LP) like nonlinear transformations of the original time series {Y (t)} that enables us to adapt all the existing stationary linear Gaussian time series modeling strategy and made it applicable for non-Gaussian and nonlinear processes in a robust fashion. The emphasis of the present paper is on empirical time series modeling via the algorithm LPTime. We demonstrate the effectiveness of our theoretical framework using daily S&P 500 return data between Jan/2/1963 -Dec/31/2009.Our proposed LPTime algorithm systematically discovers all the 'stylized facts' of the financial time series automatically all at once, which were previously noted by many researchers one at a time.

show abstract

Section: Resultsmentioning

confidence: 98%

“…Does the Normal probability distribution provide a good fit to the S&P 500 return data? insight into this question can be gained by looking at the distribution of the random variable U = G(Y ), called comparison density (Parzen, 1997, Mukhopadhyay, 2017 given by:…”

Section: Non-normality Diagnosismentioning

confidence: 99%

Nonlinear Time Series Modeling: A Unified Perspective, Algorithm and Application

Mukhopadhyay

Parzen

2018

JRFM

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the context of signal identification for instance, this implies that the researcher can specify the density function of the events associated to the signal (e.g, a Gaussian bump). In situations where this cannot be done, one possibility is to refer to nonparametric inferential methods (e.g., Chen et al, 2016;Mukhopadhyay, 2017;Algeri, 2019). More work is needed to extend TOHM to discrete regions Θ and provide a formal justification of its validity in non-regular setting, such as the one in Example 3.…”

Section: Discussionmentioning

confidence: 99%

Testing One Hypothesis Multiple Times: The Multidimensional Case

Algeri

Dyk

2019

Journal of Computational and Graphical Statistics

View full text Add to dashboard Cite

The identification of new rare signals in data, the detection of a sudden change in a trend, and the selection of competing models, are among the most challenging problems in statistical practice. These challenges can be tackled using a test of hypothesis where a nuisance parameter is present only under the alternative, and a computationally efficient solution can be obtained by the "Testing One Hypothesis Multiple times" (TOHM) method. In the one-dimensional setting, a fine discretization of the space of the non-identifiable parameter is specified, and a global p-value is obtained by approximating the distribution of the supremum of the resulting stochastic process. In this paper, we propose a computationally efficient inferential tool to perform TOHM in the multidimensional setting. Here, the approximations of interest typically involve the expected Euler Characteristics (EC) of the excursion set of the underlying random field. We introduce a simple algorithm to compute the EC in multiple dimensions and for arbitrarily large significance levels. This leads to an highly generalizable computational tool to perform hypothesis testing under non-standard regularity conditions.

show abstract

“…In order to understand if G is a good candidate for F, it is convenient to express the relationship among the two in a concise manner. The skew-G density model [9] is a universal representation scheme defined by…”

Section: Lp Approach To Statistical Modellingmentioning

confidence: 99%

“…with mean E[ LP j ] = LP j , variance V ( LP j ) = σ 2 j /n, where σ 2 j = 1 0 (Leg j (u) − LP j ) 2 d(u; G, F)∂ u (see [17]) and covariance Cov( LP j , LP k ) = σ jk /n, with σ 2 jk = 1 0 (Leg j (u) − LP j )(Leg k (u) − LP k )d(u; G, F)∂ u. Further, it can be shown [9] that, when f ≡ g,…”

Section: B Lp Density Estimatementioning

confidence: 99%

Detecting new signals under background mismodeling

Algeri

2020

Phys. Rev. D

View full text Add to dashboard Cite

Searches for new astrophysical phenomena often involve several sources of non-random uncertainties which can lead to highly misleading results. Among these, model-uncertainty arising from background mismodelling can dramatically compromise the sensitivity of the experiment under study. Specifically, overestimating the background distribution in the signal region increases the chances of missing new physics. Conversely, underestimating the background outside the signal region leads to an artificially enhanced sensitivity and a higher likelihood of claiming false discoveries. The aim of this work is to provide a unified statistical strategy to perform modelling, estimation, inference, and signal characterization under background mismodelling. The method proposed allows to incorporate the (partial) scientific knowledge available on the background distribution and provides a data-updated version of it in a purely nonparametric fashion without requiring the specification of prior distributions. Applications in the context of dark matter searches and radio surveys show how the tools presented in this article can be used to incorporate non-stochastic uncertainty due to instrumental noise and to overcome violations of classical distributional assumptions in stacking experiments.

show abstract

Large-scale mode identification and data-driven sciences

Cited by 23 publications

References 43 publications

Nonlinear Time Series Modeling: A Unified Perspective, Algorithm and Application

Nonlinear Time Series Modeling: A Unified Perspective, Algorithm and Application

Testing One Hypothesis Multiple Times: The Multidimensional Case

Detecting new signals under background mismodeling

Contact Info

Product

Resources

About