Traceability: Technical and Business Dimensions

For a given system output, technical traceability must identify the exact model version, code version, data versions, infrastructure state, and input data that produced it. Each element is referenced by a deterministic identifier: model artefact hash, Git commit SHA, DVC data version, container image digest, and logged input record. This traceability chain enables the engineering team to reproduce any historical inference, diagnose root causes of unexpected outputs, and demonstrate to a technical auditor that the system's behaviour is deterministic and traceable.

The model registry, code repository, data versioning system, container registry, and logging infrastructure must all be integrated so that a single query against the composite version identifier retrieves the complete provenance chain. The serving infrastructure tags every inference request with the composite system version at the point of execution, embedded in the log record and immutable after the fact.

Model artefacts are stored with a cryptographic hash of their contents, verified at model load time to ensure the served model is identical to the one tested and approved. Feature transformation code is shared between training and serving pipelines, not duplicated, eliminating training-serving skew as a source of traceability breaks. Deployment events are recorded in an immutable deployment ledger capturing the before-state, after-state, the authorising person, and the evidence reviewed.

Business traceability answers a different question from technical traceability: is the system achieving the outcomes it was designed to achieve, and are those outcomes aligned with the organisation's stated intent. This dimension is owned by product management and business stakeholders rather than the engineering team, requiring different metrics, cadences, and reporting formats.

The product manager tracks five dimensions. Outcome alignment assesses whether the system's actual deployment outcomes are consistent with the intended purpose documented in aisdp Module 1. Affected person experience assesses whether individuals are receiving the transparency, explanations, and redress pathways documented in Module 8. Deployer satisfaction assesses whether deployer organisations find the system useful, trustworthy, and aligned with their compliance obligations.

Override and intervention patterns track the proportion of recommendations modified by human operators and what the modification pattern reveals about alignment with human judgement. Complaint and escalation volumes track whether affected persons are raising concerns and whether those concerns are resolved.

The organisation needs a translation layer between technical metrics and business outcomes. A small drop in accuracy is a technical fact; its business significance depends on whether it causes more qualified candidates to be wrongly excluded, disproportionately affects certain groups, or triggers deployer complaints. The monitoring system should include composite indicators bridging both dimensions, enabling technical and business stakeholders to detect alignment drift from their own perspectives.

The model registry is the central index that makes the traceability chain navigable. Without it, the links between code, data, and model artefacts may exist in principle but are impractical to follow. With a properly configured registry, any authorised person can start from a model version and trace backwards to the exact experiment, training run, dataset version, code commit, and pipeline execution that produced it.

The registry structure follows a lifecycle model. MLflow uses four stages: None for initial registration, Staging for models under validation, Production for deployed models, and Archived for retired models. Stage transitions must be governed: promotion from Staging to Production requires approval from the AI Governance Lead or delegate, and the approval event is logged with the approver's identity and timestamp. This approval log is Module 10 evidence demonstrating that no model reaches production without governance sign-off.

When a system reaches end-of-life, the production model transitions to Archived. The AI System Assessor verifies the artefact's cryptographic signature one final time before archival. The model registry records the decommission date, the end-of-life trigger, and the archive storage location as structured metadata. The archived model must remain retrievable for the ten-year retention period under Article 18.