Change Classification and Substantial Modification Detection in CI/CD

The governance pipeline must classify every change to determine its regulatory significance under the EU AI Act. Article 3(23) defines substantial modification as a change to an AI system after market placement that is not foreseen in the initial conformity assessment and which affects compliance with Chapter 2 requirements or modifies the intended purpose. The pipeline implements a structured three-category classification.

Routine changes do not affect model behaviour, risk profile, fairness metrics, or intended purpose. Examples include infrastructure patches, logging improvements, and UI adjustments. These proceed through standard pipeline gates with automated evidence generation and require only Technical SME approval. Significant changes affect model behaviour, fairness metrics, feature set, training data, or performance profile but do not meet the substantial modification threshold. These require the full governance gate suite, enhanced evidence generation, AISDP module updates, and AI Governance Lead approval.

Substantial modifications meet the Article 3(23) criteria: they affect Chapter 2 compliance or modify the intended purpose. These trigger the full governance gate suite, a conformity re-assessment, and a new Declaration of Conformity. They require joint approval from the AI Governance Lead and Legal and Regulatory Advisor.

The following changes are presumptively substantial modifications unless the AI System Assessor documents a reasoned determination otherwise. A change to the model architecture, such as replacing one algorithm with another. A change to the intended purpose or target population. Retraining on a dataset whose distribution differs materially from the assessed dataset, measured by a Population Stability Index above 0.25. A fairness metric shift exceeding the declared tolerance for any protected characteristic subgroup. Removal or material modification of a human oversight mechanism. A change to the system's deployment context that was not foreseen in the initial conformity assessment.

The Technical SME implements the change classification as an automated pre-screening step in the governance pipeline. The classifier analyses the change's scope covering which files, configurations, and artefacts are modified, computes the metrics delta from the evaluation stage, and applies the classification rules defined by the AI Governance Lead. The automated classification is a recommendation; the AI System Assessor reviews and confirms the classification before the pipeline proceeds.

The classification logic should be codified as policy-as-code using Open Policy Agent and evaluated automatically at the start of each pipeline execution. The policy examines the change's characteristics and returns the classification, the indicators that triggered it, whether re-assessment is required, and the approval authority.

Without policy-as-code automation, the AI System Assessor classifies each change manually before deployment. The Assessor completes a Change Classification Form recording the change description, the artefacts modified, whether the change affects model behaviour, whether it affects the intended purpose, and the classification determination with supporting reasoning. The AI Governance Lead reviews and approves the classification.

Manual classification is adequate for systems with infrequent changes, quarterly or less. The risk is inconsistent classification across different assessors; the AI Governance Lead mitigates this through calibration exercises using historical change examples to align assessor judgement.