This document describes the OBO Foundry synchronization functionality added to the KG-Registry.
The OBO Foundry sync feature automatically creates and updates KG-Registry resources for ontologies from the OBO Foundry registry (https://obofoundry.org/). This ensures that the KG-Registry stays up-to-date with the latest ontologies from this important biomedical ontology resource.
util/sync_obo_foundry.py - Main synchronization scriptMakefile with new sync targetsobo-foundry collection# Sync all active OBO Foundry ontologies
uv run python util/sync_obo_foundry.py --verbose
# Test sync with limited number of ontologies
uv run python util/sync_obo_foundry.py --limit 5 --verbose
# Dry run to see what would be synced
uv run python util/sync_obo_foundry.py --dry-run --verbose
# Sync all active OBO Foundry ontologies
make sync-obo-foundry
# Test sync with 5 ontologies
make sync-obo-test
# Dry run to see what would be synced
make sync-obo-dry-run
The OBO Foundry sync is integrated into the main build process via the all target in the Makefile. This means that every time the full build is run, the OBO Foundry ontologies will be synchronized.
The sync process:
resource/ directoryobo-foundry collectionThe OBO Foundry record is the source for these page fields: name, description,
homepage_url, repository, activity_status, category, layout and license.
Each synced product has its own source fields: name, description, format,
category and product_url. Local edits to any of them take priority.
cache/obo_sync_state.yml records what OBO Foundry said for each ontology at the
last sync. On each run, every field is decided by comparing three values: the
page, the recorded value, and the incoming OBO value.
| Page vs. recorded | Incoming vs. recorded | Result |
|---|---|---|
| same | same | nothing changes |
| same | changed | the OBO value is applied |
| edited | same | the local edit is kept |
| edited | changed | the local edit is kept and the clash is reported |
A license is compared as a whole object, URL and label together. Fields missing
from the page are always filled in. A product that was synced before and has
since been removed from the page is not re-added. Clashes are logged, counted in
the summary, and written to reports/obo_sync_conflicts.tsv by
make sync-obo-foundry and make sync-obo-foundry-dry-run. When a clash means
the OBO Foundry record itself is out of date, the lasting fix is a pull request to
that ontology’s ontology/<id>.md in
OBOFoundry.github.io.
The state is written after each full sync, but not by a dry run.
make sync-obo-foundry-seed-state records current OBO values without changing any
page. Use it only when the pages already match OBO Foundry, as they did when the
state file was introduced (#901). OBO registry records carry no modification
dates, and per-record commit dates in the OBO repository are bumped by bulk edits,
which is why the sync compares values rather than dates.
Ontologies an OBO record lists under dependencies become sources on its products
(#421). Each synced product lists the ontology itself as prov:hadPrimarySource.
Products other than base modules also list each dependency the same way. Base
modules (ids such as cl-base.owl) hold only the ontology’s own axioms, so they
don’t get the dependencies. Dependencies without a registry resource are skipped,
as are bridge files such as go/extensions/go-bridge-to-nifstd.owl.
Sources are merged, not replaced. A dependency OBO Foundry added since the last sync is added, one it dropped is removed, and sources curated on the page are left alone.
OBO Foundry domains are mapped to KG-Registry domains as follows:
anatomy and development → anatomy and developmentbiological systems → biological systemschemistry and biochemistry → chemistry and biochemistrydiet, metabolomics, and nutrition → nutritionhealth → biomedicalinformation → information technologyinformation technology → information technologyinvestigations → generalorganisms → organismsphenotype → phenotypesimulation → information technologyupper → generalenvironment → environmentagriculture → agricultureUnknown domains default to biological systems.
The OBO Foundry sync has been enhanced with four key improvements to better align with the KG-Registry schema:
Based on the most recent sync (September 2024):
Added the obo-foundry collection to CollectionEnum in the schema:
obo-foundry:
description: >-
This entity is an ontology from the OBO Foundry,
a collaborative effort to create reference ontologies
in the biomedical domain.
meaning: https://obofoundry.org/
The sync script includes comprehensive error handling:
Failed ontologies are logged but don’t stop the overall sync process.
The script provides detailed logging at INFO and DEBUG levels:
Potential future improvements: