Defects4J: a database of existing faults to enable controlled testing studies for Java programs. Empirical studies in software testing research may not be comparable, reproducible, or characteristic of practice. One reason is that real bugs are too infrequently used in software testing research. Extracting and reproducing real bugs is challenging and as a result hand-seeded faults or mutants are commonly used as a substitute. This paper presents Defects4J, a database and extensible framework providing real bugs to enable reproducible studies in software testing research. The initial version of Defects4J contains 357 real bugs from 5 real-world open source pro- grams. Each real bug is accompanied by a comprehensive test suite that can expose (demonstrate) that bug. Defects4J is extensible and builds on top of each program’s version con- trol system. Once a program is configured in Defects4J, new bugs can be added to the database with little or no effort. Defects4J features a framework to easily access faulty and fixed program versions and corresponding test suites. This framework also provides a high-level interface to common tasks in software testing research, making it easy to con- duct and reproduce empirical studies. Defects4J is publicly available at http://defects4j.org.
Keywords for this software
References in zbMATH (referenced in 5 articles )
Showing results 1 to 5 of 5.
- Xavier Devroey, Alessio Gambi, Juan Pablo Galeotti, René Just, Fitsum Kifetew, Annibale Panichella, Sebastiano Panichella: JUGE: An Infrastructure for Benchmarking Java Unit Test Generators (2021) arXiv
- Fernanda Madeiral, Simon Urli, Marcelo Maia, Martin Monperrus: Bears: An Extensible Java Bug Benchmark for Automatic Program Repair Studies (2019) arXiv
- Naji Dmeiri, David A. Tomassi, Yichen Wang, Antara Bhowmick, Yen-Chuan Liu, Premkumar Devanbu, Bogdan Vasilescu, Cindy Rubio-González: BugSwarm: Mining and Continuously Growing a Dataset of Reproducible Failures and Fixes (2019) arXiv
- Feyzi, Farid; Parsa, Saeed: A program slicing-based method for effective detection of coincidentally correct test cases (2018)
- Yuan Yuan; Wolfgang Banzhaf: ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programming (2017) arXiv