Whenlabeling guidelines change, there is a risk oflabel shift-inconsistency between older and newer labels. The BEST recommendation is torelabel stratified samplesacross the dataset to estimate the extent of the shift and adjust datasets accordingly (D). This allows the team to quantify inconsistencies, align labels to the updated guidelines, and ensure that retraining reflects a coherent labeling policy. AAIA emphasizes managingdata and label qualityas core aspects of AI operations.
Adding layers (A) or training a larger model (C) does not solve inconsistent labeling; it may even entrench confusion. Increasing class weights on recent data only (B) can bias the model and does not repair inconsistent historical labels. Therefore,relabeling structured, representative samples to assess and correct label shiftis the most appropriate and audit-aligned recommendation.
References:
ISACA,AAIA Exam Content Outline- Domain 2: Data Management Specific to AI (labeling consistency, data drift, label shift).
ISACA AI risk guidance on dataset versioning and labeling policy changes.