For an AI-based fraud detection system, PMI-CPMAI-aligned guidance on benefits realization and performance management stresses that success metrics must be quantified against a clear baseline and monitored continuously over time. To properly define and measure KPIs such as detection accuracy, reduced investigation time, and cost savings, the project team should first establish a baseline using historical data comparisons (D). That means analyzing historical fraud cases, prior detection rates, average investigation duration, and historical financial losses to understand "pre-AI" performance. This provides a reference point against which improvements can be measured in a verifiable way.
In addition, PMI-CPMAI emphasizes continuous performance monitoring (B) as part of AI lifecycle governance. Fraud patterns, transaction volumes, and user behavior evolve, so model performance relative to KPIs must be tracked on an ongoing basis using dashboards and periodic evaluations. This supports early detection of performance degradation, allows recalibration of thresholds, and validates that business benefits (e.g., decreased losses, reduced workload) are being sustained.
Relying only on qualitative feedback, random benchmarks, or purely theoretical targets does not meet PMI-CPMAI expectations for evidence-based measurement and governance. Therefore, the two appropriate approaches are: implementing a continuous performance monitoring system (B) and establishing a baseline using historical data comparisons (D).