TRACE Phase 4

Control

Operationalise compliance through continuous monitoring, human oversight, and incident response. AI systems are dynamic — their behaviour evolves as data distributions shift, user patterns change, and operational contexts develop. Governance must therefore operate continuously, detecting deviations, enabling oversight, and responding to incidents in real time.

Core Activities

Governance in production

Five operational capabilities that ensure compliance requirements continue to be met during production operation — where the Architect phase designs for compliance, the Control phase sustains it.

Post-Market Monitoring System

EU AI Act Art. 72GDPRNIS2

Post-market monitoring is a legal requirement under EU AI Act Article 72 for providers of high-risk AI systems and a practical necessity for any system with ongoing governance obligations. The monitoring system continuously observes AI system behaviour against defined performance, fairness, and compliance thresholds.

Model performance: tracking accuracy, precision, recall, latency, and throughput against validation baselines — degradation may indicate data drift or system malfunction
Fairness metrics: continuous monitoring across protected characteristics using demographic parity, equalised odds, calibration, or domain-appropriate measures
Data drift: statistical tests (Kolmogorov-Smirnov, Population Stability Index), feature distribution monitoring, and embedding space analysis for unstructured data
Concept drift: monitoring changes in the relationship between input features and target outcomes via ground truth labels or proxy indicators
User feedback: structured collection from deployers, users, and affected persons as required by Art. 72(2)
Automated alerting: critical alerts trigger immediate human review, warnings trigger scheduled investigation, informational alerts logged for evidence

Human Oversight Operationalisation

EU AI Act Art. 14ISO 42001 Clause 9.1

The Architecture phase defines the oversight model; the Control phase makes it operational. EU AI Act Article 14(4) requires that oversight measures enable individuals to fully understand the system's capabilities, properly monitor operation, decide not to use or override outputs, and intervene or interrupt the system.

Role definitions: who performs oversight, qualifications required, authority to override or halt, operational support provided
Oversight dashboards: real-time system status, recent outputs with explanations, flagged decisions, aggregate metrics, alert summaries, trend visualisations
Escalation procedures: what findings trigger escalation, who the targets are at each level, required information, and response timelines
Automation bias countermeasures: rotating personnel, calibration exercises against ground truth, independent assessment documentation, override rate monitoring
Every oversight action recorded immutably: decisions reviewed, overrides applied, escalations raised, system halts, and interventions performed

Governance Committee and Accountability

ISO 42001 Clause 5.1ISO 27001 Clause 5.1

AI governance decisions should not rest with individual operators alone. Formal governance structures provide strategic direction, resolve cross-functional conflicts, and ensure accountability at appropriate seniority. The committee includes AI/ML engineering, data science, legal and compliance, DPO, CISO, risk management, and business units.

Committee mandate: policy approval, high-risk system classification review, significant change approval, monitoring and incident review, remediation oversight, regulatory interaction strategy
Operating cadence: monthly for active high-risk deployments, ad hoc for critical incidents or time-sensitive regulatory developments
Designated roles: AI Officer (overall accountability), DPO (GDPR Art. 37-39 advisory), CISO (NIS2 and ISO 27001 security), sector-specific roles as required
Decision rights matrix: which decisions require committee approval, which delegate to designated roles, and which delegate to operational teams
Board-level reporting: system counts, compliance posture, incidents, maturity trajectory, emerging regulatory risks — satisfying top management requirements across standards

Incident Detection and Response

EU AI Act Art. 73GDPR Art. 33-34NIS2

AI systems fail in ways that differ from conventional software failures. Model outputs can become biased, discriminatory, or harmful without any visible infrastructure error. The incident response framework must account for AI-specific failure modes: model failure, discriminatory output, data breaches, adversarial attacks, availability disruption, unintended autonomous action, and fundamental rights impact.

Detection: automated monitoring alerts, deployer reports, user complaints, affected person grievances, regulatory notifications, media, SIEM alerts, and internal audit — all feeding centralised intake
Response playbooks per category: triage and severity classification, containment (model quarantine, fallback, output suppression), root cause analysis, remediation, and staged recovery
EU AI Act Art. 73: serious incidents reported to market surveillance authority within 15 days — death, serious health damage, critical infrastructure disruption, fundamental rights breach
GDPR Art. 33-34: personal data breach notification to supervisory authority within 72 hours, data subject communication for high-risk breaches
NIS2: early warning within 24 hours, incident notification within 72 hours, final report within one month
Post-incident review for Medium+ severity: root cause, detection time, containment time, remediation, regulatory notifications, lessons learned

Continuous Compliance Monitoring

Regulatory changePolicy-as-code

Beyond system-level monitoring, the Control phase encompasses ongoing monitoring of the compliance programme itself and the regulatory environment. The AI regulatory landscape is evolving at pace — delegated acts, implementing acts, harmonised standards, AI Office guidance, EDPB opinions, and sector-specific updates all create new obligations.

Regulatory change monitoring: systematic scanning of new instruments, assessment of impact on System Regulatory Profiles and the Obligation Register, re-triage or re-rating where appropriate
Automated compliance checking: policy-as-code for infrastructure configurations, automated documentation completeness checks, data governance rules enforced through pipeline gates
Periodic reviews: quarterly for high-risk systems, semi-annually for moderate-risk, annually for low-risk — verifying controls, documentation currency, and monitoring effectiveness
Re-triage triggers: regulatory change, monitoring findings revealing unclassified risks, incident findings revealing architectural deficiencies, evidence gaps identified during audit preparation

Incident Taxonomy

AI-specific failure modes

AI systems fail in ways that differ substantially from conventional software failures. Model outputs can become biased or harmful without any visible infrastructure error.

Model Failure

Materially incorrect, inappropriate, or harmful outputs. Performance degradation below acceptable thresholds, hallucination, or failure to handle edge cases.

Discriminatory Output

Outputs that disproportionately disadvantage individuals or groups based on protected characteristics.

Data Breach

Unauthorised access to, loss of, or alteration of personal data processed by the AI system. Triggers GDPR Art. 33-34 notification.

Adversarial Attack

Deliberate manipulation through crafted inputs, model poisoning, prompt injection, or other adversarial techniques.

Availability Disruption

System unavailability or significant degradation affecting users, deployers, or affected persons who depend on it.

Fundamental Rights Impact

Adverse effects on individuals' fundamental rights as defined in the EU Charter of Fundamental Rights.

Artefacts

Control phase outputs

Six categories of operational deliverable that maintain governance effectiveness throughout the AI system lifecycle.

Post-Market Monitoring Plan & Reports

What is monitored, how, by whom, what thresholds trigger action, and periodic reports summarising findings, alerts, investigations, and actions taken.

Human Oversight Procedures & Logs

Oversight roles, dashboard specifications, escalation paths, override protocols, and immutable records of all oversight actions.

Governance Committee Charter & Records

Committee mandate, composition, operating cadence, decision rights, meeting minutes, and decision records.

Incident Response Playbooks & Reports

Category-specific response procedures, structured incident records, post-incident review documentation, and corrective action tracking.

Regulatory Notification Records

Evidence of all mandatory notifications under applicable frameworks, including content, timing, and regulatory responses.

Continuous Compliance Dashboard

Real-time and periodic views of compliance posture across all systems and frameworks, with regulatory change impact tracking.

Operationalise Your Compliance

Continuous monitoring, human oversight, and incident response — governance that operates at the same cadence as your AI systems.

Request Access Talk to Us