Log in to save to my catalogue

UMAP-based clustering split for rigorous evaluation of AI models for virtual screening on cancer cel...

UMAP-based clustering split for rigorous evaluation of AI models for virtual screening on cancer cel...

https://devfeature-collection.sl.nsw.gov.au/record/TN_cdi_doaj_primary_oai_doaj_org_article_88dd833ed7d74fa7993bbc0371d8fe49

UMAP-based clustering split for rigorous evaluation of AI models for virtual screening on cancer cell lines

About this item

Full title

UMAP-based clustering split for rigorous evaluation of AI models for virtual screening on cancer cell lines

Publisher

Cham: Springer International Publishing

Journal title

Journal of cheminformatics, 2025-06, Vol.17 (1), p.94-18, Article 94

Language

English

Formats

Publication information

Publisher

Cham: Springer International Publishing

More information

Scope and Contents

Contents

Virtual Screening (VS) of large compound libraries using Artificial Intelligence (AI) models is a highly effective approach for early drug discovery. Data splitting is crucial for benchmarking the performance of such AI models. Traditional random data splits often result in structurally similar molecules in both training and test sets, which conflict with the reality of VS libraries that typically contain structurally diverse compounds. To tackle this challenge, scaffold split, which groups molecules by shared core structure, and Butina clustering, which clusters molecules by chemotypes, have long been used. However, we show that these methods still introduce high similarities between training and test sets, leading to overestimated model performance. Our study examined four representative AI models across 60 NCI-60 datasets, each comprising approximately 33,000–54,000 molecules tested on different cancer cell lines. Each dataset was split in four ways: random, scaffold, Butina clustering and the more realistic Uniform Manifold Approximation and Projection (UMAP) clustering. Using Linear Regression, Random Forest, Transformer-CNN, and GEM, we trained a total of 8400 models and evaluated under four splitting methods. These comprehensive results show that UMAP split provides more challenging and realistic benchmarks for model evaluation, followed by Butina splits, then scaffold splits and closely after random splits. Consequently, we recommend using UMAP splits instead of overly optimistic Butina splits and especially scaffold splits for molecular property prediction, including VS. Lastly, we illustrate how misaligned ROC AUC is with VS goals, despite its common use. The code and datasets for reproducibility are available at
https://github.com/Rong830/UMAP_split_for_VS
and archived in
https://zenodo.org/records/14736486
.
Scientific contribution
This work advances the field by introducing UMAP clustering as a robust splitting method for molecular datasets, improving over traditional methods like Butina clustering and especially...

Alternative Titles

Full title

UMAP-based clustering split for rigorous evaluation of AI models for virtual screening on cancer cell lines

Authors, Artists and Contributors

Identifiers

Primary Identifiers

Record Identifier

TN_cdi_doaj_primary_oai_doaj_org_article_88dd833ed7d74fa7993bbc0371d8fe49

Permalink

https://devfeature-collection.sl.nsw.gov.au/record/TN_cdi_doaj_primary_oai_doaj_org_article_88dd833ed7d74fa7993bbc0371d8fe49

Other Identifiers

ISSN

1758-2946

E-ISSN

1758-2946

DOI

10.1186/s13321-025-01039-8

How to access this item