Explainability & Traceability

The EU AI Act requires high-risk AI systems to be transparent, interpretable, and traceable. Explain what your system does, trace how it reached each decision, and prove compliance through audit-ready documentation — these are not optional features, they are legal requirements.

Explainability

Methods for making AI interpretable

Six approaches to AI explainability — from post-hoc attribution to model documentation — each addressing different aspects of Article 13 transparency obligations.

Feature Importance

Post-hoc

Methods like SHAP and LIME that attribute model predictions to input features. These enable deployers to explain which factors drove a specific decision — essential for Article 13 transparency obligations and GDPR's right to explanation.

Attention Visualisation

Intrinsic

Visualising attention weights in transformer-based models to show which parts of the input the model focused on. Useful for natural language and image models, but attention does not always equal explanation — context matters.

Counterfactual Explanations

Contrastive

Explaining decisions by showing what would need to change for a different outcome. 'Your application was declined because X — if X were Y, the outcome would be different.' The most actionable form of explanation for affected individuals.

Model Cards

Documentation

Structured documentation of model purpose, performance metrics, limitations, ethical considerations, and intended use. Model cards are the bridge between technical performance and regulatory compliance — required for both GPAI and high-risk systems.

Concept-Based Explanations

Semantic

Explanations in terms of human-understandable concepts rather than raw features. These map model behaviour to domain concepts that regulators and affected parties can meaningfully evaluate.

Uncertainty Quantification

Calibration

Communicating the model's confidence level alongside its prediction. High-risk decisions made with low model confidence should trigger human review — connecting explainability directly to human oversight requirements.

Traceability

Proving what your AI system did

Traceability requirements under Articles 11, 12, and 17 — decision logs, version control, and audit trails that make compliance demonstrable.

Requirement	EU AI Act	What This Means
Decision Logs	Art. 12 — Record-keeping	High-risk AI systems must automatically log events during operation. Decision logs must capture inputs, outputs, timestamps, and the system state at the time of each decision — enabling post-hoc reconstruction.
Input/Output Recording	Art. 12(2)	Logging capabilities must enable tracing of input data back to the specific decision output. For high-risk systems, this creates a full chain of evidence from data ingestion to user-facing result.
Version Control	Art. 11 — Technical documentation	Every version of the AI system must be documented and identifiable. When model versions change, the technical documentation must be updated — and previous versions retained for audit trail continuity.
Audit Trails	Art. 17 — Quality management	Quality management systems must include audit trail capabilities. This means not just logging what the system did, but documenting the governance process — who approved deployments, reviewed performance, and signed off on changes.
Data Provenance	Art. 10 — Data governance	Traceability extends to training data. The full provenance chain — from data source to curation to training — must be documented and reconstructable for regulatory review.

Transparent AI

From feature importance to audit trails — structured guidance for building AI systems that explain their decisions and prove their compliance.

Request Access Talk to Us