Systematic Evaluation of Gene Expression Data Analysis Methods Using Benchmark Data

Due to limited amount of experimental validation datasets, data analysis methods for identifying differential expression based on high-throughput expression profiling technologies such as microarray and RNA-seq cannot be statistically validated properly, and thus guidelines for selecting an appropriate method are lacking. We applied mRNA spike-in approaches to develop a comprehensive set of experimental benchmark data and used it to evaluate various methods for identification of differential expression. Our results show that using the median log ratio to identify differential expression is superior to more complex and popular methods such as modified t-statistics. The median log ratio method is robust that a reasonably high accuracy of identification of differentially expressed genes can be achieved even for data with a small number of replicates and strong experimental variation between replicates. Machine learning for classification of differential expression based on the benchmark dataset indicates the existence of even more accurate methods for identification of differential expression. With this dataset, it can be also demonstrated that the methods prediction of false discovery rate based on a small number of replicates could be very inaccurate.