Numerous chemical data sets have
become available for quantitative
structure–activity relationship (QSAR) modeling studies. However,
the quality of different data sources may be different based on the
nature of experimental protocols. Therefore, potential experimental
errors in the modeling sets may lead to the development of poor QSAR
models and further affect the predictions of new compounds. In this
study, we explored the relationship between the ratio of questionable
data in the modeling sets, which was obtained by simulating experimental
errors, and the QSAR modeling performance. To this end, we used eight
data sets (four continuous endpoints and four categorical endpoints)
that have been extensively curated both in-house and by our collaborators
to create over 1800 various QSAR models. Each data set was duplicated
to create several new modeling sets with different ratios of simulated
experimental errors (i.e., randomizing the activities of part of the
compounds) in the modeling process. A fivefold cross-validation process
was used to evaluate the modeling performance, which deteriorates
when the ratio of experimental errors increases. All of the resulting
models were also used to predict external sets of new compounds, which
were excluded at the beginning of the modeling process. The modeling
results showed that the compounds with relatively large prediction
errors in cross-validation processes are likely to be those with simulated
experimental errors. However, after removing a certain number of compounds
with large prediction errors in the cross-validation process, the
external predictions of new compounds did not show improvement. Our
conclusion is that the QSAR predictions, especially consensus predictions,
can identify compounds with potential experimental errors. But removing
those compounds by the cross-validation procedure is not a reasonable
means to improve model predictivity due to overfitting.