Statistical performance of radio interferometric calibration

Abstract

Calibration is an essential step in radio interferometric data processing that corrects the data for systematic errors and in addition, subtracts bright foreground interference to reveal weak signals hidden in the residual. These weak and unknown signals are much sought after to reach many science goals but the effect of calibration on such signals is an ever present concern. The main reason for this is the incompleteness of the model used in calibration. Distributed calibration based on consensus optimization has been shown to mitigate the effect due to model incompleteness by calibrating data covering a wide bandwidth in a computationally efficient manner. In this paper, we study the statistical performance of direction dependent distributed calibration, i.e., the distortion caused by calibration on the residual statistics. In order to study this, we consider the mapping between the input uncalibrated data and the output residual data. We derive an analytical relationship for the influence of the input on the residual and use this to find the relationship between the input and output probability density functions. The eigenspectrum of the Jacobian of this mapping is a direct indicator of the statistical performance of calibration. The analysis developed in this paper can also be applied to other data processing steps in radio interferometry such as imaging and foreground subtraction as well as to many other machine learning problems.

A numerical study of 21-cm signal suppression and noise increase in direction-dependent calibration of LOFAR data

“…Since in our experiments full convergence is not always met, the number of degrees of freedom will only be partly be reduced by regularisation. A more formal method to derive the remaining number of degrees of freedom is given in Yatawatta (2019).…”

Spatially constrained direction dependent calibration

“…We reuse our previous work (Yatawatta 2018a;Yatawatta 2019b) and derive the influence function (Hampel et al 1986) as a perfor-mance measure. From (9), we select a single frequency, i.e., 𝑓 𝑖 = 𝑓 , and a single direction 𝑘.…”

“…where D J grad(𝑔 𝑓 (J 𝑘 𝑓 )) is the Hessian of 𝑔 𝑓 (J 𝑘 𝑓 ) whose exact expression can be found in Yatawatta (2019b). Substituting ( 27) into (25), we get…”

“…where J is the Jacobian of the mapping between the input and the output, which is dependent on the model s( 𝜽) in (33) which in turn is dependent on (31). The exact expressions can be found in Yatawatta (2018a); Yatawatta (2019b). The determinant of J is given as…”

“…In section 2, we define the radio interferometric data model and describe our spatially constrained distributed calibration algorithm. In section 3, we derive performance criteria based on the influence function (Hampel et al 1986;Yatawatta 2019a). We perform simulations and present the results in 4 to illustrate the improvement due to the spatial constraint.…”