Oversight Fatigue and Long-Term Vigilance

High-risk AI systems may operate for years, and maintaining effective oversight over these timescales is a significant organisational challenge. Research in safety-critical industries including aviation, nuclear power, and healthcare consistently demonstrates that human vigilance degrades over time, particularly when the system operates reliably and incidents are rare.

Normalisation of deviance occurs when a system operates within expected parameters for months or years and the organisation's sensitivity to anomalies diminishes. Thresholds originally set to trigger urgent review become tolerated as normal. Minor non-conformities that would have prompted immediate action in the system's first quarter of operation are accepted as routine. This phenomenon is well-documented in safety-critical industries, and AI operational oversight is susceptible to the same dynamic.

Oversight fatigue manifests through gradual erosion of governance effectiveness. The risk register review becomes a formality. Monitoring dashboards are glanced at rather than analysed. The quarterly governance meeting shortens from two hours to thirty minutes. Each individual erosion is minor; cumulatively, they create gaps that are invisible from the inside because the people closest to the system have normalised them.

Four countermeasures address the degradation of oversight effectiveness over time. Personnel rotation on a 6 to 12 month cycle is the primary structural countermeasure. A new person brings a fresh perspective: they notice anomalies the previous person had normalised, ask questions about processes the previous person had stopped questioning, and identify documentation gaps the previous person had worked around. The AI Governance Lead plans the rotation schedule in advance with a handover period including knowledge transfer and a documented checklist.

Threshold drift checks address the gradual relaxation of operational thresholds. Over time, teams may informally adjust alert thresholds upward to reduce alert volume, and each adjustment is individually reasonable but the cumulative effect decreases sensitivity to genuine problems. Quarterly threshold drift checks compare current operational thresholds in monitoring configuration, CI pipeline gates, and alert rules against the values documented in the aisdp. Any discrepancy must be either reverted to the documented threshold or formally approved by updating the AISDP with the new threshold and the rationale.

Fresh eyes reviews bring personnel not involved in daily operations into periodic deep-dives. An internal auditor, a team member from a different system, or an external consultant reviews monitoring data, the evidence repository, the non-conformity register, and governance meeting minutes with no prior context. Their findings often reveal systemic issues that the operational team has normalised.

Automation of oversight reduces dependence on sustained human vigilance. Automated threshold monitoring, automated compliance checks verifying the deployed system matches the AISDP-documented configuration, and automated anomaly detection provide a baseline that does not degrade over time. Human oversight should focus on activities requiring judgement: interpreting anomalies, making governance decisions, and assessing whether real-world impact aligns with intended purpose.

Budget and staffing continuity is essential. Oversight resources are vulnerable to budget pressure, particularly when the system operates without incident. The annual oversight cost is treated by the AI Governance Lead as a committed operational expense, not a discretionary allocation. If the budget is reduced, the impact on oversight capability is assessed and any resulting compliance risk documented in the risk register.

An organisation's first high-risk AI system receives intensive attention. By the time the portfolio reaches ten or twenty systems, the oversight framework must scale or it will fail. The governance model designed for a single system will not survive a portfolio.

Shared infrastructure is the foundation. Monitoring infrastructure, evidence repositories, document management systems, and CI/CD pipelines are designed as shared services supporting multiple systems. Shared monitoring with multi-tenant configuration allows a single team to oversee all systems from a unified dashboard, with per-system metric labels enabling both aggregate views and per-system drill-down. Shared infrastructure also enables cross-system analysis detecting patterns such as a common vulnerability across multiple systems using the same GPAI model.

Tiered oversight recognises that not all high-risk systems require the same intensity. A credit scoring system affecting millions of consumers warrants more intensive oversight than an internal document classification system. Higher-tier systems receive more frequent reviews, dedicated personnel, and more granular monitoring. Lower-tier systems receive scheduled reviews, shared personnel, and standard monitoring configurations. The AI Governance Lead documents tier assignments and reviews them annually.

Centralised governance with distributed execution ensures consistent standards while keeping operational knowledge local. The AI Governance Lead provides central coordination: maintaining the portfolio-level risk register, ensuring consistent standards, and reporting to executive leadership. Day-to-day oversight execution is distributed to the teams closest to each system.