Probabilistic and non-deterministic AI-based systemsdo not always produce the same output for identical inputs. This makes traditional testing approaches ineffective. Instead, the best approach is torun tests multiple times and analyze results statistically.
* Statistical Validity:Running tests multiple times ensures that observed results are statistically significant. Instead of relying on a single test run,analyzing multiple iterations helps determine trends, probabilities, and outliers.
* Expected Result Tolerance:AI-based systems may produce different results within an acceptable range. Defining acceptable tolerances (e.g., "result must be within 2% of the optimal value") improves test effectiveness.
* A (Run Several Times for the Same Correct Result):AI systems are ofteninherently non- deterministicand may not return the exact same result every time. Expecting identical outputs contradicts the nature of these systems.
* B & C (Decomposing Tests into Data Ingestion Tests):While data ingestion quality is important, it does notdirectlysolve the issue of probabilistic test results. Statistical analysis is the key approach.
* ISTQB CT-AI Syllabus (Section 8.4: Challenges Testing Probabilistic and Non-Deterministic AI- Based Systems)
* "For probabilistic systems, running a test multiple times may be necessary to obtain a statistically valid test result.".
* "Where a single definitive output is not possible, results should be analyzed statistically rather than relying on individual test cases.".
Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:Sinceprobabilistic AI systems do not always return the same result, the best approach is torun multiple test iterations and validate results statistically. Hence, thecorrect answer is D.