Biography

István Forgács PhD is a test expert and an author. István is the co-author of the books Paradigm Shift in Software Testing, Practical Test Design, and Agile Testing Foundations. He is the creator and key contributor of the test-first, codeless test design automation tool Harmony. He is the co-author of the website https://test-design.org/ which is a unique place where testers can exercise test design by executing and improving their tests. He introduced three test design techniques: combinative testing, action-state testing, and general predicate testing.

Talk

Do You Know How Good You Are at Test Design? We Don’t Think So. Why?

The productivity of individual developers can be measured by many factors, especially on the bugs revealed by themselves, reported by the testers and the users. Unfortunately, the productivity of the testers is difficult as it needs ‘testing the tests’. The only method, fault-based testing, especially mutation testing is hardly used, mutation testing tools create mainly weak mutants.

To the best of our knowledge, there are no websites available where a SUT is running and the testers can learn which tests are missing (if any) and why. Thus, testers are unable to measure their technical abilities. The consequence is that testers have only a vague assumption about their test design efficiency.

Istavn and Prof. Attila Kovacs made a website https://test-design.org/practical-exercises/ where testers can exercise and improve their test design knowledge. The process of the development of the exercises was the following:
• First, we developed a mutation-testing framework.
• We created some (at present 9) original and interesting specifications
including 5-10 requirements.

For each specification:
• We made the test design.
• We implemented (and tested) the code.
• We produced some mutants based on the initial real bugs.
• We add more mutants according to typical real-life malfunctions.
• We automated the tests.
• We executed the tests, evaluated the test design, and finally, made some
improvements if it was needed.

After finishing an exercise, the system responds with the percentage of the killed mutants, i.e., the test efficiency, with the missing tests. There are three types of exercises: (1) specifications including compound predicates, (2) specifications describing stateful behaviour, and (3) mixing the previous two.

With the techniques presented in their new book Paradigm Shift in Software Testing, reliable test sets can be designed for all the exercises (100% of the mutants will be killed). The website shows the testers the missing test cases as well. In this way they can study why a given test is missing and how can they improve their test design.

In his presentation, Istvan will show (demo) how to use their website, their test design tools for testing features that are difficult to test.

Session Keywords