A multi-center study on factors influencing the reproducibility ofin vitrodrug-response studies

Abstract

Evidence that some influential biomedical results cannot be recapitulated has increased calls for data that is findable, accessible, interoperable, and reproducible (FAIR). Here, we study factors influencing the reproducibility of a prototypical cell-based assay: responsiveness of cultured cell lines to anti-cancer drugs. Such assays are important for drug development, mechanism of action studies, and patient stratification. This study involved seven research centers comprising the NIH LINCS Program Consortium, which aims to systematically characterize the responses of human cells to perturbation by gene disruption, small molecule drugs, and components of the microenvironment. We found that factors influencing the measurement of drug response vary substantially with the compound being analyzed and thus, the underlying biology. For example, substitution of a surrogate assay such as CellTiter-Glo® for direct microscopy-based cell counting is acceptable in the case of neratinib or alpelisib, but not palbociclib or etoposide. Uncovering and controlling for such context sensitivity requires systematic measurement of assay robustness in the face of biological variation, which is distinct from assay precision and sensitivity. Conversely, validating assays only over a narrow range of conditions has the potential to introduce serious systematic error in a large dataset spanning many compounds and cell lines.

Efficient Parameter Estimation Enables the Prediction of Drug Response Using a Mechanistic Pan-Cancer Pathway Model

“…The computed correlation (0.49 ± 0.01) and RMSE (0.26 ± 0.01) are marginally weaker than what was achieved with the mechanistic model but significant with p corr = 1.8 3 10 À2 and p RMSE = 3.8 3 10 À3 . This mediocre agreement of data from different databases was already characterized in previous studies (Haibe-Kains et al, 2013) and has been attributed to differences in assay protocols (Haverty et al, 2016), evaluation methods (Fallahi-Sichani et al, 2013;Pozdeyev et al, 2016), as well as shortcomings in the employed assay protocols (Hafner et al, 2017;Niepel et al, 2017). Accordingly, the slightly better performance of the mechanistic model compared to sigmoid interpolation is encouraging, but the low magnitude of differences is understandable as assay protocols are currently not considered in the mechanistic model.…”

“…The CellTiterGlo assay quantifies ATP levels as proxy for the number of healthy cells. Still, several concerns regarding reproducibility of results have been raised (Hafner et al, 2017;Niepel et al, 2017). The (relative) cell viability is the net sum of cell division and cell death, which are governed by a complex interplay of cellular signaling processes regulating, e.g., the balance between pro-growth and (anti-)apoptotic signals in response to extracellular stimuli or the presence of activating mutations within respective signal transduction cascades.…”

Functional drug susceptibility testing using single-cell mass predicts treatment outcome in patient-derived cancer neurosphere models

“…However, this finding should not be viewed as an indictment of the CTG assay as a whole and should be viewed in light of the large body of existing work in which the assay has successfully been used for drug susceptibility testing. As others have described previously (Niepel et al, 2019), even simple experiments, such as measuring the response of cells to in vitro drug exposure, can be affected by factors such as cell-counting protocols, micro-titer plate selection, and cell-seeding density; all of which result in significant inter-center variability. We ran the CTG assay using one particular assay format (i.e., plate type, cell-seeding density, feeding schedule, drug dose, and time-point selection), and although our selection was a reasonable choice, it is entirely possible that, under different conditions, the assay could be more predictive.…”

Stress, Health and Well-being of Clinical Embryologists

“…Clinical embryology involves a lot of repetitive tasks, be it preparing dishes, changing the embryos into different wells during culture or vitrificationwarming or ICSI. Manual repetitive tasks are known to be a source of human error in the laboratory (11) .…”