The SDEval benchmarking toolkit. In this paper we will present SDeval, a software project that contains tools for creating and running benchmarks with a focus on problems in computer algebra. It is built on top of the Symbolic Data project, able to translate problems in the database into executable code for various computer algebra systems. The included tools are designed to be very flexible to use and to extend, such that they can be easily deployed even in contexts of other communities. We also address particularities of benchmarking in the field of computer algebra. Furthermore, with SDEval, we provide a feasible and automatable way of reproducing benchmarks published in current research works, which appears to be a difficult task in general due to the customizability of the available programs.