Metamorphic testing is an approach to both test case generation and test result verification.
A central element is a set of metamorphic relations, which are necessary properties of the …

Context: Advancements in machine learning (ML) lead to a shift from the traditional view of
software development, where algorithms are hard-coded by humans, to ML systems …

This paper provides a comprehensive survey of techniques for testing machine learning
systems; Machine Learning Testing (ML testing) research. It covers 144 papers on testing …

The past decade has seen the great potential of applying deep neural network (DNN) based
software to safety-critical scenarios, such as autonomous driving. Similar to traditional …

Recent advances in Deep Neural Networks (DNNs) have led to the development of DNN-
driven autonomous cars that, using sensors like camera, LiDAR, etc., can drive without any …

While Deep Neural Networks (DNNs) have established the fundamentals of image-based
autonomous driving systems, they may exhibit erroneous behaviors and cause fatal …

Neural networks are difficult to interpret and debug. We introduce testing techniques for
neural networks that can discover errors occurring only for rare inputs. Specifically, we …

A test oracle determines whether a test execution reveals a fault, often by comparing the
observed program output to the expected output. This is not always practical, for example …

Virtually any software running on a computer has been processed by a compiler or a
compiler-like tool. Because compilers are such a crucial piece of infrastructure for building …

We have recently witnessed tremendous success of Machine Learning (ML) in practical
applications. Computer vision, speech recognition and language translation have all seen a …