SCAFFOLD-AWARE PREDICTION OF ESTROGEN RECEPTOR ALPHA INHIBITION WITH UNCERTAINTY-GUIDED CANDIDATE PRIORITIZATION

Abstract

An approach that relies on computational prioritization might be useful for identifying estrogen receptor alpha (ERα) antagonists, but false positive gains can be experienced when the same molecular scaffolds are found in both training and test sets. This study evaluated the efficacy of scaffold-aware prediction and uncertainty-based candidate prioritization with an ERα antagonist activity set. Following molecular structure standardization and removal of invalid and duplicate molecules, 5,845 unique molecules remain with 255 labelled as positive. The sets of compounds were divided into training (n = 4,465), validation (n = 561), and independent test set (n = 819) where no overlap occurred among these groups by scaffold. Class weighted logistic regression, random forest and extra-trees models were compared with morgan fingerprints. Random forest had the highest validation precision–recall area under the curve (PR-AUC; 0.354) and was then refitted on the combined development set. It performed well on the scaffold-separated test set: PR-AUC = 0.323 (95% bootstrap confidence interval: 0.203–0.473), and an ROC-AUC = 0.840 (0.787–0.887). Precision was 0.545 and recall was 0.150 at the threshold which was selected for validation. Overall test prevalence was 40/819; 6/10 highest ranked test compounds were known positives. Prediction difference between 5 random seeds was averaged and reported as model disagreement but not as calibrated uncertainty. The results also indicate that ranked selection of the candidates for experimental follow-up is appropriate and that classification based on thresholds did not correctly classify many of the known positives. Independent chemical-series validation is still required.