Recently, Jean Xavier Mazoit from the Animal Ethics Committee CEEA26 at Paris-Saclay University conducted a highly anticipated study. The research results have been released in preprint format and have not yet been peer-reviewed.

This study focuses on the three authoritative multidisciplinary journals in the field of bioscience---Nature, Proceedings of the National Academy of Sciences (PNAS), and Science. The research team retrieved experimental studies that require ethical committee approval and have been published in these three journals before June 30, 2023, and randomly selected 50 articles from each journal. Given that PNAS announced the establishment of a Statistical Review Committee in September 2023 and has begun to perform its duties, the research team further searched for and analyzed 50 articles published in this journal since January 1, 2024.

The researchers conducted a detailed review of the key content in the article, comprising clear approval statements of the animal ethics committee, calculation methods of appropriate sample size, implementation of randomization and blinding methods, minimum sample size for reporting, the existence of repeated experiments, and methods to limit type 1 and type 2 errors.

Surprisingly, the research results revealed a problem: among all the analyzed articles, not a single one fully and clearly reported its experimental design. Nearly half of the articles have the problem of extremely small sample sizes, with each group having less than 4 samples. Many articles confuse the concepts of biological repetition and technological repetition. Meanwhile, many articles did not make appropriate corrections when multiple comparisons were made. These errors have resulted in a significant increase in both type 1 and type 2 errors. Ultimately, the study found that only 10% articles utilized the correct statistical methods.

This study emphasizes an important problem in bioscience field: Even articles from top tier journals may have deficiencies in statistical methods, which can affect the accuracy and reliability of research results. This study also reminds researchers and journal editors to pay more attention to the standardization of experimental design and statistical methods when conducting research and publishing papers, so as to ensure the quality of scientific research results.