Languages
DE
Traditional software operates according to hard-coded rules. If the result of a function deviates from the expected behavior, the error can often be pinpointed in the program code and specifically corrected. But how does quality assurance work when a system’s behavior isn’t determined solely by predefined rules, but is learned from training data?
This is where artificial intelligence (AI) presents new challenges for software quality assurance. An AI-based image recognition system, for example, can correctly identify thousands of objects yet still misclassify individual images. Such a misclassification often cannot be traced back to a specific line of code and fixed in a targeted manner.
For testing, this means a fundamental shift in perspective: Instead of focusing solely on detecting individual errors, the quality of an AI system must be statistically evaluated across a large number of test cases. What matters is not only the number of tests but also how well the test data used reflects the future operational environment.
But under these conditions, how can we reliably determine whether an AI system meets the required quality standards?
AI systems based on machine learning differ fundamentally from traditionally programmed systems. They learn from a large number of training examples by adjusting a model’s parameters during training. For example, an image recognition system learns to distinguish between dogs and cats based on numerous images. The underlying features are not entirely dictated by fixed rules but are derived from the training data.
This has direct implications for quality assurance: Even a more powerful AI model will not necessarily process every input correctly. It is therefore crucial to determine how frequently errors occur and under what conditions they arise.
The quality of an AI system must therefore also be evaluated statistically. Rather than focusing exclusively on individual test results, the question becomes how reliably the system performs across a wide variety of different inputs.
A simple example illustrates how much the selection of test data influences the result: An AI-based image recognition system distinguishes between dogs and cats. It recognizes cats with 95 percent accuracy, but dogs with only 85 percent.
Two testers are tasked with independently evaluating the system’s quality. Both use 1,000 images each, but select different test datasets. While the first tester uses mostly images of cats, the second uses primarily images of dogs.
The results differ significantly:
Although both testers evaluate the same system using the same number of images, they obtain different results. However, neither of the two test datasets reflects the actual distribution of dogs and cats assumed in the example. When this distribution is taken into account, the accuracy is approximately 90.7 percent.
This example illustrates a fundamental problem with AI testing: a large number of test cases alone does not guarantee meaningful results. It is also crucial that the test data realistically reflects the conditions of the system’s future application.
Otherwise, even extensive testing can provide a distorted picture of an AI system’s actual performance. Only a representative selection of test data enables reliable statistical assessments of quality.
In traditional software development, systematic testing procedures ensure that as many relevant functions and input scenarios as possible are verified. Methods such as equivalence class partitioning or combinatorial testing help cover extensive test scenarios with a sufficient number of tests.
Test coverage is also important when testing AI systems. However, it is not sufficient on its own.
To obtain reliable test results, two requirements must therefore be met:
For example, an object detection system for automated vehicles should not be tested only in sunny, rainy, and foggy conditions. To meaningfully evaluate its performance, the varying probabilities of these weather conditions occurring in the intended operating area must also be taken into account.
This does not mean that rare or safety-critical edge cases should be tested less thoroughly. Such scenarios may require targeted additional testing. However, their statistical frequency must be taken into account when evaluating overall quality.
The intended operating environment of a system is referred to as the Operational Design Domain (ODD). It describes the framework within which a system must perform its intended functions.
For an automated train, for example, this includes the track infrastructure, signals, weather conditions, and potential obstacles in the track area. These factors influence the situations that the AI system must recognize and handle during operation.
However, for a representative test strategy, it is not sufficient to simply address the relevant influencing factors with an arbitrary number of test cases. It is also necessary to consider how frequently certain situations occur and what dependencies exist between them. For example, weather conditions can influence the probability of certain visibility conditions or obstacle situations.
A systematic description of the operational environment thus forms the basis for selecting suitable test cases. If this description is further supplemented with statistical information, a model can be developed from which representative tests can be derived.
Due to their statistical properties, a crucial question arises with AI systems: How many tests are necessary to reliably demonstrate a certain level of quality?
Suppose an AI-based object recognition system is expected to correctly recognize at least 99 percent of all relevant objects. Even if it makes not a single error in 100 test runs, this does not sufficiently prove that it will meet this requirement during subsequent operation. The observed result is subject to statistical uncertainty.
Therefore, when testing AI, in addition to the required quality, the desired confidence level and the necessary number of test cases must also be taken into account. Statistical methods can be used to estimate the test scope necessary to verify a quality requirement with a specified level of statistical confidence.
An AI test should therefore not end simply when all planned test cases have been executed, but rather when the results provide a sufficiently reliable indication of the required quality. A prerequisite for this is that the test data representatively reflects the intended operational environment.
The unique characteristics of AI systems require a correspondingly adapted testing strategy. Traditional testing methods such as equivalence class formation, robustness testing, and simulations remain important components of quality assurance. However, they must be supplemented with statistical methods in order to make reliable statements about the performance of an AI system.
In practice, this means that the intended operational environment must be systematically described as early as the test planning phase. On this basis, relevant test scenarios can be identified and test data selected that ensure both sufficient coverage and a representative distribution.
Furthermore, measurable quality objectives and appropriate statistical acceptance criteria must be defined. This makes it possible not only to determine which errors occurred during testing but also to assess whether the system achieves the required quality with the desired statistical confidence.
Statistical testing methods therefore do not replace traditional methods but rather supplement them with a crucial dimension: the reliable evaluation of AI quality within the intended area of application.
Testing AI systems requires a broader perspective on software quality. Only the combination of systematic test coverage, representative test data, and statistical evaluation criteria enables reliable conclusions about how reliably a system will perform in its future operational environment.
ITPower Solutions helps companies systematically and transparently evaluate the quality of their AI-based systems. We develop appropriate testing strategies, assist in selecting representative test data, and establish statistically sound quality and test-completion criteria. In this way, we lay the foundation for reliable quality assessments, reduce uncertainties in the evaluation of AI systems, and support our clients in making informed decisions about their deployment.
I am your sales representative and will be happy to advise you on all questions relating to our services and products! Get in touch or simply make an appointment for a free consultation call.
Sebastian Stritz
E-Mail: sebastian.stritz@itpower.de
Phone: +49 (0)30 6098501-17