On the Reliability of Microvariability Tests in Quasars

Abstract

Microvariations probe the physics and internal structure of quasars. Unpredictability and small flux variations make this phenomenon elusive and difficult to detect. Variance based probes such as the C and F tests, or a combination of both, are popular methods to compare the light-curves of the quasar and a comparison star. Recently, detection claims in some studies depend on the agreement of the results of the C and F tests, or of two instances of the F -test, in rejecting the non-variation null hypothesis. However, the C-test is a non-reliable statistical procedure, the F -test is not robust, and the combination of tests with concurrent results is anything but a straightforward methodology. A priori Power Analysis calculations and post hoc analysis of Monte-Carlo simulations show excellent agreement for the Analysis of Variance test to detect microvariations, as well as the limitations of the F -test. Additionally, combined tests yield correlated probabilities that make the assessment of statistical significance unworkable. However, it is possible to include data from several field stars to enhance the power in a single Ftest, increasing the reliability of the statistical analysis. This would be the preferred methodology when several comparison stars are available. An example using two stars and the enhanced Ftest is presented. These results show the importance of using adequate methodologies, and avoid inappropriate procedures that can jeopardize microvariability detections. Power analysis and Monte-Carlo simulations are useful tools for research planning, as they can reveal the robustness and reliability of different research approaches.

Testing Microvariability in Quasar Differential Light Curves Using Several Field Stars

“…These limitations place a strong emphasis on careful handling of the data and the use of robust and powerful statistical techniques for the analysis. Analyzing aperiodic variations has been the subject of recent work on AGNs (Emmanoulopoulos et al 2010;de Diego 2010de Diego , 2014, the latter is hereafter referred to as Paper I), as well as young stars and massive stars (Findeisen et al 2015). …”

Microvariability in AGNs: study of different statistical methods I. Observational Analysis

“…de Diego (2010) studied the χ 2 test, the F test for variances, the ANOVA test, and the C criterion for a set of simulated light curves, concluding that the most robust methodologies are the ANOVA and χ 2 tests, while the F statistic is less powerful but still a reliable tool, and, finally, the C criterion should be avoided because it is not a proper statistical test. Further analysis about these tests is presented in de Diego (2014), where a study of the Bartels and Runs non-parametric test was added. In that work, the author proposed that the best choices to detect microvariability in AGN light curves are the use of an ANOVA or an enhanced−F test (in the latter, several comparison stars are used to define a combined variance, instead of using a single star).…”

“…The need of a large number of points in light curves strongly limits the use of the χ 2 test. The same applies to the ANOVA test: despite its claimed power to detect microvariability (de Diego 2010(de Diego , 2014, this test is seldom used, because it requires a large number of data points too (Joshi et al 2011); moreover, data grouping might be impractical for faint objects requiring relatively long integration times, and could lead to false results if data within a time span larger than the (unknown) variability time-scale are grouped. In fact, some doubtful results from the use of the ANOVA test in AGN microvariability studies (de Diego et al 1998) have already been discussed in Romero et al (1999).…”

Comparative intranight optical variability of X-ray and γ-ray-detected narrow-line Seyfert 1 galaxies

“…The second version of the F−test employed here is the, so-called, F enh −test (de Diego 2014). Its chief merit is that it transforms the DLCs of the comparison stars to the same photometric noise level as if the magnitudes of the comparison stars are exactly matched to the mean magnitude of the AGN monitored (thereby making the analysis free from the effect of the magnitude difference between the target AGN and the comparison star(s), which can significantly impact some versions of the F-test wherein the factor η is ignored, e.g., see Joshi et al 2011).…”