A systematic evaluation of data processing and problem formulation of CRISPR off-target site prediction

Abstract

CRISPR/Cas9 system is widely used in a broad range of gene-editing applications. While this editing technique is quite accurate in the target region, there may be many unplanned off-target sites (OTSs). Consequently, a plethora of computational methods have been developed to predict off-target cleavage sites given a guide RNA and a reference genome. However, these methods are based on small-scale datasets (only tens to hundreds of OTSs) produced by experimental techniques to detect OTSs with a low signal-to-noise ratio. Recently, CHANGE-seq, a new in vitro experimental technique to detect OTSs, was used to produce a dataset of unprecedented scale and quality (>200 000 OTS over 110 guide RNAs). In addition, the same study included in cellula GUIDE-seq experiments for 58 of the guide RNAs. Here, we fill the gap in previous computational methods by utilizing these data to systematically evaluate data processing and formulation of the CRISPR OTSs prediction problem. Our evaluations show that data transformation as a pre-processing phase is critical prior to model training. Moreover, we demonstrate the improvement gained by adding potential inactive OTSs to the training datasets. Furthermore, our results point to the importance of adding the number of mismatches between guide RNAs and their OTSs as a feature. Finally, we present predictive off-target in cellula models based on both in vitro and in cellula data and compare them to state-of-the-art methods in predicting true OTSs. Our conclusions will be instrumental in any future development of an off-target predictor based on high-throughput datasets.

Generating, modeling, and evaluating a large-scale set of CRISPR/Cas9 off-target sites with bulges

“…We observed opposite effects of class weighting on the XGBoost and neural network models. Consistent with our previous study (35), the classification and regression XGBoost models benefit from class weighting in all model variations. However, when examining the neural network models, we found no such improvement and even a decrease in the average AUPR for most models.…”

“…In our deep neural networks, the prediction of off-target activity is based on aligned sgRNA and off-target sequences, which include the PAM. To accommodate aligned sequences with bulges, we extend the one-hot-encoding approach utilized in our previous study (35), which could only handle mismatches, to handle bulges. Given an aligned sgRNA sequence S t and an aligned off-target sequence S o , we encode the pair of aligned nucleotides or gaps (represented by -) at position i in the aligned sequences into a 5 × 5 matrix C i ( j t , j o ) using the following scheme: where 1 ≤ j t , j o ≤ 5 represent the four possible nucleotides and a gap.…”

“…For a baseline comparison, we implemented a modified version of the XGBoost model used in previous studies (30; 35) to enable the prediction of OTS with bulges. The sequence encoding is as the encoding for our deep neural network models, but with an additional flattening due to the limitation of XG-Boost in processing two-dimensional data.…”

“…We defined experimentally obtained OTS as the active sets. In the CHANGE-seq study and our previous work (32), experimentally identified OTS cleaved in vitro with read counts below 100 were filtered out as noisy measurements. This filtering criterion resulted in a reduced size of the CHANGE-seq active set to 70, 425 OTS.…”

Off-target effects in CRISPR/Cas9 gene editing

“…This technique relies on the delivery of double-stranded oligonucleotides (dsODNs) with known sequences, which can integrate into DSBs during NHEJ (non-homologous end joining). The integrated dsODNs provide templates for targeted PCR amplification and sequencing of the tagged DNA fragments ( Tsai et al, 2015 ; Malinin et al, 2021 ; Yaish et al, 2022 ) ( Figure 1B ). GUIDE-seq can detect off-target sites with indel frequencies as low as 0.03% ( Tsai et al, 2015 ).…”