Within core architecture and collection taxonomy, every persistence decision — how you design a museum object schema, where you attach Getty AAT and TGN identifiers, how you map LIDO into your internal database — assumes you have already chosen the shape of your canonical record. That choice is not obvious. The cultural-heritage sector maintains at least five overlapping schema standards, each authored for a different purpose, at a different level of granularity, with a different theory of what a “collection object” even is. Before you can write a single crosswalk, you have to know the landscape and decide which standard your internal model will treat as authoritative.

This guide is the decision layer that sits before mapping. It compares LIDO, CIDOC-CRM, Dublin Core, and the two dominant collection-management data models — The Museum System (TMS) and CollectiveAccess — across the criteria that actually determine fit: descriptive purpose, granularity, event modeling, controlled-vocabulary support, interoperability, and tooling. It then walks through scoring those standards against your institution’s real needs, selecting one canonical internal schema, and defining the crosswalks that translate every other standard into it. The goal is not to crown a universal winner — there is none — but to make the trade-off explicit and repeatable.

Workflow Context

A common failure mode is to adopt whichever schema your first data source happens to speak. A museum receives a LIDO harvest from an aggregator, models its database around LIDO’s event-centric structure, and then discovers that its own TMS export, its Dublin Core OAI feed, and its born-digital CollectiveAccess records all resist that shape. The database becomes an accident of import order rather than a deliberate design.

The alternative is to treat schema selection as its own architectural stage. Your internal model — the canonical schema that the rest of the system reads and writes — should be chosen for how well it serves your workflows, not for how closely it mirrors any one external standard. Every external standard then becomes an edge format: something you translate on the way in and on the way out through an explicit crosswalk. This is the same separation the ingestion layer already enforces, where an async ingestion pipeline normalizes heterogeneous sources before anything touches the write path. Here the concern is one level up: deciding what those sources normalize to.

The decision has downstream weight. A canonical schema that omits event modeling cannot losslessly round-trip CIDOC-CRM provenance chains. One that stores rights as free text cannot drive the automated access routing the delivery layer depends on. Choosing well here is what lets the later stages — resolution, rights automation, and IIIF delivery — stay simple.

Prerequisites

Before scoring standards or writing crosswalks, have the following in place:

  • Python 3.9+ for the PEP 604 X | None union syntax and match statements used throughout.
  • pydantic (v2) to model each standard’s scorecard and validate the canonical contract; this page uses the v2 API (model_config, field_validator, model_dump). See the pydantic v2 documentation for details.
  • lxml (5+) if you intend to parse LIDO or CIDOC-CRM RDF/XML while probing candidate sources for coverage.
  • A written inventory of your data sources — for each feed, which standard it speaks (LIDO XML, an OAI-PMH Dublin Core feed, a TMS SQL export, a CollectiveAccess JSON dump) and its expected volume.
  • The authoritative specifications, bookmarked: the LIDO schema, the CIDOC-CRM specification, Dublin Core terms from the Dublin Core / DCMI namespace, and — for rights — RightsStatements.org. TMS and CollectiveAccess are software data models rather than open specifications, so their “spec” is the vendor’s field dictionary and the open-source schema respectively.
  • A shortlist of institutional requirements ranked by weight: the criteria in the next section only produce a useful ranking once you know which ones your museum actually cares about.

Schema & Standards Reference

The five standards are not competitors on a single axis; they answer different questions. LIDO is a harvesting and interchange format — an event-centric XML envelope designed to carry records between institutions and into aggregators. CIDOC-CRM is a formal ontology: it does not prescribe fields at all but a semantic graph of entities and relationships (E22 Human-Made Object, E12 Production, E52 Time-Span) into which any record can be expressed. Dublin Core is a deliberately minimal descriptive baseline of fifteen core elements, optimized for cross-domain discovery, not curatorial depth. TMS and CollectiveAccess are operational collection-management data models — the relational schemas real museums run day to day, rich in workflow fields but tied to their platform.

The table below scores each standard across the six criteria that most often decide a canonical-schema choice.

Criterion LIDO CIDOC-CRM Dublin Core TMS CollectiveAccess
Primary purpose Interchange / harvesting envelope Formal semantic ontology Minimal descriptive baseline Vendor collection-management model Open-source collection-management model
Granularity High — descriptive + administrative Very high — arbitrarily fine Low — 15 core elements High — hundreds of operational fields High — configurable metadata elements
Event modeling First-class (lido:eventSet) Native and central (E5 Event) None (flat properties) Implicit (dated activity tables) Configurable (relationship + reln types)
Controlled-vocab support Strong (concept IDs + sources) Strong (E55 Type, external URIs) Weak (free text or DCMI encoding) Moderate (authority tables) Strong (built-in vocabulary lists)
Interoperability High — built for exchange High — but requires RDF expertise Very high — universal lowest common denominator Low — export needed to leave platform Moderate — open schema, custom profiles
Tooling (Python) lxml, LIDO XSDs rdflib, SPARQL endpoints dcxml, most OAI libraries ODBC / SQL export, no open library REST/GraphQL API, direct DB access

Two reading notes matter more than any single cell. First, granularity and interoperability trade off: Dublin Core interoperates everywhere precisely because it says almost nothing, while CIDOC-CRM can express anything at the cost of requiring RDF fluency to consume. Second, TMS and CollectiveAccess are systems of record, not interchange formats — you will almost always read from them and write a canonical record, rather than adopting their schema wholesale, because their field sets are entangled with platform-specific workflow tables. The dedicated TMS versus CollectiveAccess schema mapping guide works through that particular reconciliation field by field.

To make the comparison programmatic rather than impressionistic, model each standard as a scorecard. A strict Pydantic v2 model enforces that every criterion is scored on the same 0–5 scale, so the ranking that follows is auditable.

python
from pydantic import BaseModel, ConfigDict, field_validator

CRITERIA = (
    "purpose_fit",
    "granularity",
    "event_modeling",
    "vocab_support",
    "interoperability",
    "tooling",
)

class StandardScore(BaseModel):
    """A weighted evaluation of one schema standard against local needs."""
    model_config = ConfigDict(extra="forbid")

    name: str
    purpose_fit: int
    granularity: int
    event_modeling: int
    vocab_support: int
    interoperability: int
    tooling: int

    @field_validator(*CRITERIA)
    @classmethod
    def in_range(cls, value: int) -> int:
        # Every criterion shares one scale so totals are comparable.
        if not 0 <= value <= 5:
            raise ValueError("each criterion scores 0 to 5")
        return value

With extra="forbid", adding a criterion in one scorecard but forgetting it in another is caught immediately rather than silently skewing the ranking.

Step-by-Step Implementation

1. Score each standard against institutional needs

The comparison table gives defensible baseline scores, but the weights are yours. A regional history museum digitizing photographs weights interoperability and vocabulary support highly and event modeling barely at all; an archaeology collection tracking excavation context weights event modeling above everything. Encode both the scores and the weights, then collapse each scorecard to a single comparable number.

python
from collections.abc import Mapping

DEFAULT_WEIGHTS: dict[str, float] = {
    "purpose_fit": 0.30,
    "granularity": 0.15,
    "event_modeling": 0.20,
    "vocab_support": 0.15,
    "interoperability": 0.15,
    "tooling": 0.05,
}

def weighted_total(score: StandardScore, weights: Mapping[str, float]) -> float:
    """Collapse a scorecard into one comparable number in [0, 5]."""
    data = score.model_dump()
    return round(sum(data[k] * w for k, w in weights.items()), 3)

def rank(
    scores: list[StandardScore], weights: Mapping[str, float]
) -> list[tuple[str, float]]:
    """Return (name, total) pairs sorted best-first for a given weighting."""
    totals = [(s.name, weighted_total(s, weights)) for s in scores]
    return sorted(totals, key=lambda pair: pair[1], reverse=True)

Edge cases. Keep the weights in version control alongside the code: a schema decision that cannot be re-derived from written criteria is a decision no successor can defend. Run the ranking under two or three weightings (discovery-led, provenance-led, operations-led) and note which standard wins under each — if one standard leads across all of them, the choice is easy; if the winner flips, that tension is exactly what to discuss before committing.

2. Pick a canonical internal schema

Ranking rarely selects your internal schema outright. In practice the highest-scoring standard informs the internal model but does not become it, because your canonical schema must serve reads, writes, indexing, and access control — concerns no interchange format optimizes for. The usual outcome is a purpose-built internal schema that borrows LIDO’s event structure and CIDOC-CRM’s identifier discipline while flattening both into a shape your database and API can serve efficiently.

Express the canonical schema as its own strict contract. This is the model the rest of the system depends on; every external standard is translated into it.

python
from datetime import date
from pydantic import BaseModel, ConfigDict, field_validator

RIGHTS_URI_PREFIX = "https://rightsstatements.org/vocab/"

class CanonicalObject(BaseModel):
    """The single internal record every external standard maps to."""
    model_config = ConfigDict(str_strip_whitespace=True, extra="forbid")

    object_id: str
    title: str
    creator: str | None = None
    rights_status: str
    date_modified: date

    @field_validator("rights_status")
    @classmethod
    def must_be_rights_uri(cls, value: str) -> str:
        # Rights is a resolvable URI internally, whatever the source called it.
        if not value.startswith(RIGHTS_URI_PREFIX):
            raise ValueError(f"rights_status must be a {RIGHTS_URI_PREFIX} URI")
        return value

Edge cases. Resist the urge to make the canonical schema a superset of every standard’s fields — that reproduces TMS’s sprawl and makes every consumer pay for coverage it never uses. Model the fields your workflows read, and preserve everything else as an opaque source_payload blob you can re-crosswalk later if a requirement emerges. For XML sources you extract these fields with lxml; for API sources they usually arrive as JSON already; for flat exports they come through the CSV path — but they all converge on this one model.

3. Define crosswalks in Python

A crosswalk is a table: for each canonical field, the source path in every standard that can supply it. Modeling it as data rather than scattered if branches makes coverage measurable and keeps the mapping reviewable by curators who do not read Python fluently.

python
from dataclasses import dataclass

@dataclass(frozen=True)
class FieldMap:
    """One canonical field and its source path in each standard."""
    canonical: str
    lido: str | None = None
    cidoc_crm: str | None = None
    dublin_core: str | None = None
    tms: str | None = None
    collectiveaccess: str | None = None

CROSSWALK: tuple[FieldMap, ...] = (
    FieldMap(
        canonical="object_id",
        lido="lido:lidoRecID",
        cidoc_crm="crm:E42_Identifier",
        dublin_core="dc:identifier",
        tms="Objects.ObjectID",
        collectiveaccess="ca_objects.idno",
    ),
    FieldMap(
        canonical="title",
        lido="lido:titleSet/lido:appellationValue",
        cidoc_crm="crm:P102_has_title",
        dublin_core="dc:title",
        tms="Objects.Title",
        collectiveaccess="ca_objects.preferred_labels.name",
    ),
    FieldMap(
        canonical="creator",
        lido="lido:eventSet/lido:actorInRole/lido:actor",
        cidoc_crm="crm:P14_carried_out_by",
        dublin_core="dc:creator",
        tms="ConXrefs.DisplayName",
        collectiveaccess="ca_entities.preferred_labels.name",
    ),
    FieldMap(
        canonical="rights_status",
        lido="lido:rightsWorkSet/lido:rightsType",
        cidoc_crm="crm:P104_is_subject_to",
        dublin_core="dc:rights",
        tms="Objects.RightsType",
        collectiveaccess="ca_objects.access",
    ),
    FieldMap(
        canonical="date_modified",
        lido="lido:recordWrap/lido:recordInfoSet/lido:recordMetadataDate",
        cidoc_crm="crm:P4_has_time-span",
        dublin_core="dc:modified",
        tms="Objects.EditDate",
        collectiveaccess="ca_objects.lastModified",
    ),
)

def source_path(field: str, standard: str) -> str | None:
    """Look up the source path for a canonical field in one standard."""
    for entry in CROSSWALK:
        if entry.canonical == field:
            return getattr(entry, standard)
    return None

Edge cases. Some mappings are one-to-many or lossy in one direction. CIDOC-CRM’s P4_has_time-span points at an E52 Time-Span node, not a literal date, so the crosswalk entry marks the anchor and the extraction code resolves the literal underneath. A None in a cell is meaningful: it records that a standard genuinely cannot supply that field, which is exactly the signal the coverage test in the verification section reports on. Once titles, creators, and place names land in canonical form, resolving their free-text values to authority URIs is the job of implementing Getty AAT and TGN.

Scoring five schema standards down to one canonical internal schema Five candidate schema standards are listed on the left: LIDO as an interchange envelope, CIDOC-CRM as a formal ontology, Dublin Core as a descriptive baseline, TMS as a vendor collection-management model, and CollectiveAccess as an open collection-management model. Each feeds a scoring stage that weights them against institutional needs across six criteria. The scoring stage selects one canonical internal schema, which is the single source of truth. From that schema, crosswalks defined in Python map each external standard back to the canonical fields. Choose a canonical schema, then crosswalk every standard into it LIDO interchange envelope CIDOC-CRM formal ontology Dublin Core descriptive baseline TMS vendor CMS model CollectiveAccess open CMS model Score against institutional needs 6 weighted criteria Canonical internal schema single source of truth Crosswalks per standard defined in Python selects maps

Rights and Access Routing

Every standard carries rights, but each carries it differently, and those differences decide how much automated access routing the canonical schema can drive. LIDO holds rights in a structured lido:rightsWorkSet with a typed lido:rightsType and a source attribution, which maps cleanly to a controlled URI. CIDOC-CRM expresses rights as a relationship — an object P104_is_subject_to an E30 Right node — so the value lives in the graph rather than a field. Dublin Core’s dc:rights is a single free-text element with no vocabulary constraint, which is why Dublin Core feeds so often arrive with rights strings like "in copyright" that must be resolved before they are usable. TMS stores a RightsType code plus free-text rights TextEntries, and CollectiveAccess couples an access flag with configurable rights metadata.

The canonical schema resolves all of this to one thing: a resolvable RightsStatements.org URI in rights_status, validated by the must_be_rights_uri check above. That normalization is what makes rights routable. A record resolving to InC/1.0/ commits at a metadata-only access tier — descriptive record public, media gated — while NoC-US/1.0/ may expose full-resolution media. Because the canonical URI is standard-agnostic, the downstream policy engine never needs to know whether a record originated in LIDO, TMS, or a Dublin Core harvest. Free-text values that no crosswalk can resolve to a URI must be quarantined for a curator, never defaulted to public. The full policy layer — embargo dates, license routing, jurisdiction — lives in rights metadata mapping and licensing automation; the canonical schema’s only job here is to guarantee that whatever standard a record came from, its rights leave this stage as one machine-readable URI.

Verification and Testing

The crosswalk is only trustworthy if it covers the whole canonical schema. Two checks catch the two ways it can rot: a canonical field that no crosswalk row defines (a silent gap that drops data), and a drifting sense of how much each standard actually supplies. The coverage report answers the second; the assertion guards the first.

python
CANONICAL_FIELDS = {"object_id", "title", "creator", "rights_status", "date_modified"}
STANDARDS = ("lido", "cidoc_crm", "dublin_core", "tms", "collectiveaccess")

def coverage_report(crosswalk: tuple[FieldMap, ...]) -> dict[str, float]:
    """Fraction of canonical fields each standard supplies a source path for."""
    mapped = {std: 0 for std in STANDARDS}
    for entry in crosswalk:
        for std in STANDARDS:
            if getattr(entry, std) is not None:
                mapped[std] += 1
    total = len(CANONICAL_FIELDS)
    return {std: round(count / total, 2) for std, count in mapped.items()}

def test_no_canonical_gap() -> None:
    # Every canonical field must have a crosswalk row, or data is silently lost.
    covered = {entry.canonical for entry in CROSSWALK}
    missing = CANONICAL_FIELDS - covered
    assert not missing, f"canonical fields with no crosswalk row: {sorted(missing)}"

if __name__ == "__main__":
    test_no_canonical_gap()
    print(coverage_report(CROSSWALK))

Run it to see coverage per standard before trusting any import:

bash
python -m taxonomy.crosswalk_check

A healthy report shows every standard at or near full coverage for the fields it can supply, and test_no_canonical_gap passing. A standard that suddenly drops to partial coverage after a schema edit is your early warning that a canonical field lost its mapping — fix the crosswalk before that field starts arriving empty in production.

Where to Go Deeper

  • TMS versus CollectiveAccess Schema Mapping — takes the two operational data models this guide scores side by side and works the reconciliation field by field: how TMS’s Objects/ConXrefs relational tables and CollectiveAccess’s ca_objects bundles each translate into the canonical schema, where the two disagree on cardinality, and which fields have no clean equivalent.

FAQ

Should I just adopt LIDO as my internal schema since it is standards-based?

Usually no. LIDO is an interchange envelope optimized for harvesting between institutions, not for the reads, writes, and access control your application performs constantly. Borrow LIDO’s event structure and identifier discipline, but keep your canonical schema purpose-built for your workflows and treat LIDO as a format you crosswalk on the way in and out. The LIDO mapping guide covers that translation.

What is the difference between using CIDOC-CRM and using LIDO?

CIDOC-CRM is a formal ontology — a semantic graph of entities and relationships with no prescribed fields — while LIDO is a concrete XML schema with defined elements. CIDOC-CRM can express anything but requires RDF fluency to consume; LIDO is far easier to parse with lxml but less expressive. Many institutions use LIDO as the wire format and treat it as a lightweight, field-oriented profile over CIDOC-CRM’s semantics.

Do I need Getty vocabularies before choosing a schema standard?

No — vocabulary support is one scoring criterion, not a prerequisite. Choose the canonical schema first, then attach authority identifiers to its free-text fields. Resolving materials, roles, and place names to controlled URIs is a separate stage covered in implementing Getty AAT and TGN.

How do I handle a field that only one standard can supply?

Keep it in the crosswalk with None in the cells where a standard has no equivalent, and decide whether the canonical schema needs the field at all. If only your CollectiveAccess source carries it and your workflows depend on it, the field belongs in the canonical model and simply arrives empty from other sources. If nothing downstream reads it, preserve it in the opaque source_payload blob rather than widening the canonical contract.

Can I change my canonical schema later without re-importing everything?

Yes, if you retained each record’s original source_payload. Because every import is a deterministic crosswalk from a preserved source representation, adding or reshaping a canonical field is a re-crosswalk over stored payloads rather than a fresh harvest. This is the main reason to keep the raw source alongside the canonical record instead of discarding it after import.