Data provenance is the documented history of data: where it came from, who created it, how it was collected, what rights apply and what has been done to it since. For AI teams, data provenance has moved from good practice to necessity. Regulators, customers and courts increasingly ask what a model was trained on. Without provenance, those questions can't be answered. This guide explains what data provenance means for AI, what to record and how to build it into sourcing.
Why data provenance matters for AI
Legal compliance
Copyright, privacy and contract law all depend on knowing a dataset's source. Data protection law requires a lawful basis for processing personal data. AI-specific regulation, such as the EU AI Act, includes obligations around training-data documentation for certain providers.
Licence management
Datasets often carry different terms. Provenance tells you which terms apply to which records.
Quality and debugging
When a model behaves unexpectedly, provenance helps trace the problem to particular sources.
Evaluation integrity
Knowing where test data came from helps prevent contamination, where test items leak into training data.
Trust
Customers buying AI products increasingly ask for assurances about data sources.
What to record
Source
The organisation or system the data came from, including the original systems for business records.
Creator and collection
Who created the data, when, how and for what original purpose.
Rights
The licence or legal basis, including the permitted uses, restrictions, term and exclusivity. See data licensing agreements.
Consent and notice
For personal data, what individuals were told and what lawful basis applies.
Authority
Who granted the licence and why they were entitled to.
Transformations
Cleaning, filtering, pseudonymisation, annotation, augmentation and merging, with versions and dates.
Lineage
Which datasets and models used the data.
Retention and deletion
Obligations and the dates by which they must be met.
Provenance for business records
Business records add particular provenance questions:
- Which company created the records, and does it still exist? For closed companies, authority may rest with an office-holder.
- Which systems and date ranges are included?
- Were client-confidential or third-party materials excluded?
- How were people pseudonymised, and is the mapping retained or destroyed?
- Which links between records are original and which were reconstructed?
That last point matters for RL environments and evaluations, where inferred links could mislead graders. See RL environments.
Standards and practices
- Datasheets for datasets and data cards: structured documentation of motivation, composition, collection, preprocessing, uses and maintenance.
- Content credentials: cryptographic provenance for media, such as C2PA.
- Data catalogues and lineage tools: to track datasets through pipelines.
- Manifests and hashes: fix exactly which files were delivered under which licence.
Building provenance into sourcing
- Write the permitted use into the sourcing brief from the start.
- Verify the licensor's authority before collection.
- Record source systems, date ranges and exclusions at delivery.
- Hash and version every delivery.
- Log every transformation in the pipeline.
- Link datasets to the models trained or evaluated on them.
- Review provenance before each new use.
Common gaps
- Datasets assembled from several sources with no record of which records came from where
- Licences stored separately from the data they govern
- Pseudonymisation applied without documenting the method
- Evaluation sets reused without checking for overlap with training data
- Undocumented scraping
How Maevar helps
Maevar introduces record owners to vetted buyers. Before any introduction, we look at fit, the authority to license, the systems and date ranges, exclusions and the permitted use. The buyer pays our introduction fee. The owner and the buyer agree terms, preparation and how personal details are handled, directly. Discuss your use case or read where to buy AI training data.
A minimal provenance record
For each delivery, keep at least:
| Field | Example content |
|---|---|
| Dataset ID and version | Unique identifier and version number |
| Source organisation | Licensor name and status |
| Authority | Who signed and in what capacity |
| Systems and dates | Source systems and date range |
| Exclusions | Record types removed |
| Personal data handling | Method and test results |
| Licence reference | Agreement and permitted uses |
| Delivery manifest | File list with hashes |
| Transformations | Pipeline steps and versions |
| Downstream uses | Models and evaluations using it |
Who owns provenance
Provenance fails when nobody is responsible for it. Assign an owner for each dataset who keeps the record current, approves new uses and answers questions from legal, security and customers. Review the record whenever a dataset is used for something new.
Provenance and model documentation
Model cards and system documentation increasingly summarise training data sources. Good dataset provenance makes these summaries accurate and defensible, rather than vague statements written after the fact.
FAQ
What is data provenance?
The documented origin and history of data, including its source, rights and transformations.
Why is data provenance important in AI?
It supports legal compliance, licence management, debugging, evaluation integrity and customer trust.
What is the difference between provenance and lineage?
Lineage usually describes data's path through systems and pipelines. Provenance is broader and includes origin, rights and context.
How do I document provenance?
Use datasheets or data cards, manifests with hashes, licence records and transformation logs.
Does synthetic data need provenance?
Yes. Record the generator, its inputs and the settings used. See synthetic vs real data.