- May 12, 2026
- Reading time: 8min
Article 10 of the EU AI Act sets data governance and quality requirements for training, validation and testing datasets used by high-risk AI systems. For Annex III high-risk AI systems, these requirements are scheduled to apply from 2 December 2027.
That means compliance does not start with the model. It starts much earlier, with the data used to build it. Annotation, labelling, cleaning, bias examination, data quality and documentation all form part of this process. This guide explains what Article 10 requires and what it means for the people responsible for preparing and managing training data.
This guide is written for the teams doing that work: data leads, MLOps engineers, heads of AI, and compliance officers at companies whose AI systems fall into the high-risk category. We will walk through what Article 10 requires, what compliant data preparation actually looks like in practice, and what your team needs to have in place before the deadline.
This article reflects our interpretation of the EU AI Act as a data annotation provider. It does not constitute legal advice. We recommend consulting qualified legal counsel for your specific compliance obligations.
Note: The AI Omnibus entered into force on 27 July 2026. As a result, the application date for high-risk AI systems covered by Annex III has been extended to 2 December 2027. High-risk AI systems embedded in products covered by Annex I have an extended application date of 2 August 2028.
What is high-risk AI under the EU AI Act?
The EU AI Act classifies AI systems by the risk they pose to individuals’ rights, safety and access to services. Most AI applications remain lightly regulated or unregulated. High-risk systems, however, face a comprehensive set of obligations, and Article 10’s training data requirements apply specifically to this category.
Under Annex III of the Act, a system is classified as high-risk if it operates in one of the following domains:
- Biometric identification and categorisation – facial recognition, emotion recognition, remote biometric ID systems
- Critical infrastructure – AI managing energy grids, water systems, transport networks
- Education and vocational training – systems that determine admission, assess students, or influence professional certification
- Employment and worker management – CV screening, candidate ranking, performance monitoring, hiring and promotion decisions
- Essential private and public services – credit scoring, insurance risk assessment, social benefits eligibility
- Law enforcement – crime analytics, predictive policing, risk assessment tools used by authorities
- Migration and border management – asylum and visa processing, risk assessment at borders
- Administration of justice – AI supporting legal or judicial decisions
One of the most consequential aspects of the regulation is its geographic scope. Any organisation, wherever it is based, must comply if its AI systems are used within the EU or if their outputs affect EU residents.
A quick self-check: if your AI system falls within one of the high-risk categories in Annex III, the Article 10 requirements for training, validation and testing data may apply from 2 December 2027.
What does Article 10 actually require?
Article 10 applies to training, validation and testing datasets, meaning the entire model development lifecycle, not just deployment. The following requirements apply.
Relevance and representativeness – Article 10 requires training, validation and testing datasets to be subject to appropriate data-governance and management practices. These include examining whether datasets are relevant, sufficiently representative and appropriate for the intended purpose and geographical, contextual, behavioural or functional setting in which the system will be used.
Quality – Article 10 requires appropriate data-governance and management practices, including assessment of the availability, quantity and suitability of the datasets needed for the system. Data must also have appropriate statistical properties, including where applicable characteristics related to the people or groups on which the system is intended to be used.
Bias examination and mitigation – Providers must actively examine datasets for biases that could affect health, safety or fundamental rights, or lead to prohibited discrimination, and must take appropriate measures to detect, prevent and mitigate identified biases.
Documentation of all data preparation processes – This is the requirement most teams underestimate. Article 10(2)(c) explicitly names the operations that must be documented, which brings us to the next section.
The six data preparation processes named in Article 10
Article 10(2)(c) explicitly identifies six data-preparation operations: annotation, labelling, cleaning, updating, enrichment and aggregation. These operations are part of the data governance and management practices required for the relevant datasets.
Annotation – attaching labels, attributes or relationships to raw data points. In medical AI, this might mean a clinician marking lesion boundaries in an MRI. In automotive AI, it means bounding boxes around pedestrians in video frames.
Annotation processes should be clearly defined and documented as part of the broader data-governance and management process. Depending on the use case, teams may also use quality control measures such as multiple-annotator review and inter-annotator agreement metrics to demonstrate consistency.
Labelling – assigning categorical classification labels at scale. Must be reproducible: different labellers given the same data point and guidelines should consistently arrive at the same label. Document your taxonomy, consistency evidence, and any corrections applied.
Cleaning – removal of errors, duplicates and corrupted entries. Requires a documented pipeline: what was removed, by what logic, in what sequence, authorised by whom. Version control is essential.
Updating – changes to datasets and retraining inputs should be tracked and documented as part of an appropriate data-management process, including what changed and why. This connects directly to preventing model collapse, dataset quality issues in retraining cycles are one of its primary causes.
Enrichment – adding supplementary data to improve coverage or representativeness. Consider provenance: where did the data come from, under what conditions was it collected, and could it introduce new biases while addressing existing ones?
Aggregation – combining multiple data sources into a single training corpus. Document the provenance, quality profile and relevant bias considerations of each source.
As our 2026 data annotation trends research, these requirements are already influencing annotation workflows across industries. Teams that start early have more time to identify and address data-governance gaps.
Why annotator diversity is a compliance question, not just an ethical one
Article 10 places emphasis on dataset relevance, representativeness and bias examination. While the regulation does not prescribe the demographic composition of an annotation workforce, teams can consider the perspectives and expertise involved in annotation when designing processes intended to identify potential gaps or sources of bias in their data.
Annotation teams with limited perspectives can introduce blind spots, particularly when the data represents populations, environments or contexts that are unfamiliar to the people doing the annotation.
For example, an annotation process that draws on a narrow range of perspectives may miss relevant patterns, edge cases or sources of bias in the data. A medical imaging model annotated by clinicians from one type of healthcare environment may underperform across others. These are documented failures in deployed systems, and they are precisely what Article 10’s bias and representativeness requirements are designed to prevent.
Humans in the Loop was built from the start around a different approach. As a social enterprise founded in 2017, our workforce draws from conflict-affected and displaced communities across Bulgaria, Turkey, Syria, Iraq, Afghanistan, Lebanon, Yemen, Portugal, Ukraine, Moldova, Kenya and the DRC, with over 50% women annotators. This is not a diversity initiative sitting alongside our work. It is structural to how we operate, grounded in our ethical AI approach and fair work policy.
The annotation quality case for this is substantive: annotators who bring different cultural contexts, different relationships to institutions, different lived experiences of the situations AI systems are being trained to navigate, they catch what homogeneous teams miss. This is one way an annotation provider can contribute to broader efforts to identify gaps, contextual differences and potential sources of bias in training data.
We explore this in more depth in our piece on how human annotation drives responsible AI, and it is the reason we believe ethical data annotation and regulatory compliance are not separate conversations.
Humans in the Loop provides human (not AI-generated) data annotation by an ethically sourced, geographically diverse workforce. If you’re thinking about how your training data holds up against Article 10’s representativeness and bias requirements, start a free pilot or book a call with our team.
What well-governed annotation looks like in practice
A compliant annotation workflow for a high-risk AI system has four non-negotiable elements:
Documented task specification. Before annotation begins, guidelines must define what is being labelled, how edge cases are handled, and what quality standards apply. This document is part of your compliance record.
Qualified annotators. Medical image annotation requires clinical knowledge, Automotive scene annotation requires understanding of road conditions across contexts. Documenting the expertise and training involved can also help teams demonstrate how their data-preparation process was managed.
Multiple annotators and IAA measurement. Multiple independent annotations and agreement measures such as Cohen’s Kappa or Fleiss’ Kappa can be useful quality control tools for assessing consistency, depending on the task. These are examples of good annotation practice, rather than metrics explicitly mandated by Article 10.
Audit logs. For projects where traceability is important, maintaining records of annotation decisions, corrections and quality checks can help teams demonstrate how their data-preparation process was managed and support broader documentation requirements.
None of this is bureaucratic overhead invented by regulation. It is what rigorous annotation practice looks like. The EU AI Act has formalised standards that responsible annotation teams have been working toward for years.
Common mistakes when preparing training data for the EU AI Act
Even when teams understand the requirements of Article 10, some common data-governance gaps can make it harder to demonstrate that training, validation and testing datasets have been appropriately managed
Treating bias as a model problem – Article 10 explicitly requires examination of training, validation and testing datasets for possible biases. Model-level testing remains important, but it does not replace the data-level examination required by Article 10.
No evidence of annotation quality control – Agreement measurement can be a useful way to assess consistency in suitable annotation tasks, but Article 10 does not prescribe a specific IAA metric. The broader requirement is appropriate data governance and management.
No external documentation trail – Internal annotation can be high quality, but without appropriate documentation of the data-preparation process, including relevant methodology and quality control measures, you may find yourself unable to demonstrate how your training data was prepared and managed.
Treating the 2027 deadline as a reason to wait – Building a complete Article 10 documentation trail retroactively can be a significant undertaking. The confirmed deadline of 2 December 2027 gives teams more time to prepare, but starting the work earlier gives teams more time to identify and address data-governance gaps before the requirements apply.
How to prepare before December 2027: a practical checklist
Update – August 2026: The AI Omnibus entered into force on 27 July 2026, extending the application date for high-risk AI systems covered by Annex III to 2 December 2027. High-risk AI systems covered by Article 6(1) and Annex I have an application date of 2 August 2028. Teams can use the additional time to strengthen their data-governance, documentation and quality processes ahead of the applicable deadline.
- Map your AI systems against Annex III and determine which high-risk use cases may apply.
- Review training, validation and testing datasets for relevance, suitability and representativeness.
- Consider whether your annotation process includes the perspectives and expertise needed to identify relevant gaps or potential sources of bias.
- Document relevant data preparation processes, including annotation, labelling, cleaning, updating, enrichment and aggregation.
- Use appropriate quality control measures, including agreement measurement where relevant to the annotation task, and retain the results.
- Conduct and document an appropriate bias examination and record any measures taken to detect, prevent or mitigate identified biases.
- Identify data gaps and shortcomings and document how they are addressed.
- Establish data provenance and version control so changes to datasets can be tracked.
- Review your documentation with qualified legal/compliance professionals where necessary.
The bottom line
The EU AI Act is not asking for the impossible. It is asking for rigour, documentation and accountability in processes that should already be part of responsible AI development. For teams building training datasets informally, the path to compliance requires real investment. For teams already operating to high standards, it is largely about formalising what they do.
At Humans in the Loop, we have been making the case since 2017 for ethical data annotation done by humans, with accountability built into the process and perspectives from diverse communities.
Book a call with our team to talk through your training data pipeline or start a free pilot to see our annotation workflow in action.
Frequently Asked Questions About EU AI Act Article 10
What is Article 10 of the EU AI Act?
Article 10 establishes data governance and management requirements for training, validation and testing datasets used by relevant high-risk AI systems.
What does Article 10 require for training data?
Article 10 requires appropriate data governance and management practices covering areas including data collection and origin, data preparation, data quality, representativeness, bias examination and the identification of relevant data gaps or shortcomings.
Does Article 10 apply to annotation?
Yes. Article 10 explicitly identifies annotation as one of the relevant data-preparation processing operations covered by its data-governance requirements.
Does the EU AI Act require inter-annotator agreement?
No specific IAA metric is prescribed by Article 10. Agreement measurement can be a useful quality-control practice for suitable annotation tasks, but it should not be presented as a specific legal requirement.
When do the Article 10 requirements for Annex III high-risk AI systems apply?
2 December 2027, under the current 2026 amendments. The exact application timeline can differ for other categories of high-risk systems, so teams should check the applicable provisions for their system.
