Reliable benchmarking: requirements and solutions
Benchmarking is a widely used method in experimental computer science, in particular, for
the comparative evaluation of tools and algorithms. As a consequence, a number of …
the comparative evaluation of tools and algorithms. As a consequence, a number of …
TOOLympics 2019: An overview of competitions in formal methods
Abstract Evaluation of scientific contributions can be done in many different ways. For the
various research communities working on the verification of systems (software, hardware, or …
various research communities working on the verification of systems (software, hardware, or …
Advances in automatic software verification: SV-COMP 2020
D Beyer - International Conference on Tools and Algorithms for …, 2020 - Springer
This report describes the 2020 Competition on Software Verification (SV-COMP), the 9^ th
edition of a series of comparative evaluations of fully automatic software verifiers for C and …
edition of a series of comparative evaluations of fully automatic software verifiers for C and …
[HTML][HTML] Design and results of the second international competition on computational models of argumentation
Abstract Argumentation is a major topic in the study of Artificial Intelligence. Since the first
edition in 2015, advancements in solving (abstract) argumentation frameworks are assessed …
edition in 2015, advancements in solving (abstract) argumentation frameworks are assessed …
[PDF][PDF] Advances in Automatic Software Testing: Test-Comp 2022.
D Beyer - FASE, 2022 - library.oapen.org
Test-Comp 2022 is the 4th edition of the Competition on Software Testing. Research
competitions are a means to provide annual comparative evaluations. Test-Comp focusses …
competitions are a means to provide annual comparative evaluations. Test-Comp focusses …
The SMT competition 2015–2018
Abstract The International Satisfiability Modulo Theories Competition is an annual
competition between Satisfiability Modulo Theories (SMT) solvers. The 2018 edition of the …
competition between Satisfiability Modulo Theories (SMT) solvers. The 2018 edition of the …
[PDF][PDF] Software Testing: 5th Comparative Evaluation: Test-Comp 2023.
D Beyer - FASE, 2023 - library.oapen.org
The 5th edition of the Competition on Software Testing (Test-Comp 2023) provides again an
overview and comparative evaluation of automatic test-suite generators for C programs. The …
overview and comparative evaluation of automatic test-suite generators for C programs. The …
Software model checking: 20 years and beyond
D Beyer, A Podelski - Principles of Systems Design: Essays Dedicated to …, 2022 - Springer
We give an overview of the development of software model checking, a general approach to
algorithmic program verification that integrates static analysis, model checking, and …
algorithmic program verification that integrates static analysis, model checking, and …
[PDF][PDF] Status report on software testing: Test-Comp 2021
D Beyer - Fundamental Approaches to Software Engineering, 2021 - library.oapen.org
This report describes Test-Comp 2021, the 3rd edition of the Competition on Software
Testing. The competition is a series of annual comparative evaluations of fully automatic …
Testing. The competition is a series of annual comparative evaluations of fully automatic …
[HTML][HTML] The 2016 and 2017 QBF solvers evaluations (QBFEVAL'16 and QBFEVAL'17)
After a break of about five years, in 2016 the classical QBFEVAL has been revived.
QBFEVAL is a competitive evaluation of solvers for quantified Boolean formulas (QBF), the …
QBFEVAL is a competitive evaluation of solvers for quantified Boolean formulas (QBF), the …