Ulster University Logo

Ulster Institutional Repository

Avoiding model selection bias in small-sample genomic datasets

Biomedical Sciences Research Institute Computer Science Research Institute Environmental Sciences Research Institute Nanotechnology & Advanced Materials Research Institute

Berrar, D, Bradbury, I and Dubitzky, Werner (2006) Avoiding model selection bias in small-sample genomic datasets. BIOINFORMATICS, 22 (10). pp. 1245-1250. [Journal article]

Full text not available from this repository.

DOI: 10.1093/bioinformatics/btl066

Abstract

Motivation: Genomic datasets generated by high-throughput technologies are typically characterized by a moderate number of samples and a large number of measurements per sample. As a consequence, classification models are commonly compared based on resampling techniques. This investigation discusses the conceptual difficulties involved in comparative classification studies. Conclusions derived from such studies are often optimistically biased, because the apparent differences in performance are usually not controlled in a statistically stringent framework taking into account the adopted sampling strategy. We investigate this problem by means of a comparison of various classifiers in the context of multiclass microarray data. Results: Commonly used accuracy-based performance values, with or without confidence intervals, are inadequate for comparing classifiers for small-sample data. We present a statistical methodology that avoids bias in cross-validated model selection in the context of small-sample scenarios. This methodology is valid for both k-fold cross-validation and repeated random sampling.

Item Type:Journal article
Faculties and Schools:Faculty of Life and Health Sciences
Faculty of Life and Health Sciences > School of Biomedical Sciences
Research Institutes and Groups:Biomedical Sciences Research Institute
Biomedical Sciences Research Institute > Molecular Medicine
Biomedical Sciences Research Institute > Molecular Medicine > Nano Systems Biology
ID Code:4475
Deposited By:Professor Werner Dubitzky
Deposited On:05 Jan 2010 15:34
Last Modified:11 Aug 2010 16:55

Repository Staff Only: item control page