kg_microbe.transform_utils.ontologies_stubs package

Submodules

kg_microbe.transform_utils.ontologies_stubs.ncit_category_projection module

Project two reviewed native NCIT concepts, never a whole semantic-type vocabulary.

kg_microbe.transform_utils.ontologies_stubs.ncit_category_projection.native_contract(selected, resolved, sha256): Serialize the producer’s selected locator, exact bytes and complete absent-sidecar set.

kg_microbe.transform_utils.ontologies_stubs.ncit_category_projection.project_category(adapter, curie, disposition): Authorize a finite projection only when every reviewed native fact still agrees.

kg_microbe.transform_utils.ontologies_stubs.ncit_category_projection.read_dispositions(stream): Require the complete finite policy, rejecting duplicate or expanded scope.

kg_microbe.transform_utils.ontologies_stubs.ncit_category_projection.validate_native_contract(contract, raw_dir, snapshots, ncit_nodes, *, admission=None, expected_member=None): Validate the finite NCIT contract without adapters, retaining guards in public admission.

kg_microbe.transform_utils.ontologies_stubs.ontologies_stubs_transform module

Ontologies-stubs transform.

KG-Microbe deliberately does NOT load the full NCIT, MESH, BTO, PO, or MICRO ontologies — those would each add thousands of unrelated nodes for what is in practice a small per-mapping reference footprint. But the chemical-mapping consolidator and the BacDive isolation-source mapper reference ~73 distinct NCIT IDs, ~95 distinct mesh IDs, a handful of distinct BTO/PO IDs, and ~34 distinct MICRO IDs (across ~150 total reference rows) as canonical xrefs for ingredients, growth media, plant-anatomy isolation sources, and microbial conditions (e.g. NCIT:C29298 'Oatmeal', mesh:D011136 'Tween', PO:0009046 'flower', MICRO:0000082 'nutrient broth'). Without this transform those CURIEs would appear as dangling node ids in the merged KG: edges point at them but no node row carries the label.

This transform:

Calls collect_stub_curies() to discover every NCIT / mesh / BTO / PO / MICRO CURIE referenced anywhere under mappings/.
For each prefix, dispatches on STUB_ONTOLOGY_SOURCES[prefix]["source_type"]:
- "semsql" — NCIT, mesh, BTO. Queries the local SemSQL DB (data/raw/{ncit,mesh,bto}.db) via OAK and emits one labelled stub node per referenced CURIE; no edges. The mesh entry sets db_prefix = "MESH" to bridge the casing gap between our lowercase mesh: mapping ids and the DB’s uppercase MESH: subject form.
- "owl_mireot" — PO, MICRO. Runs robot extract --method MIREOT against the source OWL (data/raw/{po,micro}.owl from download.yaml) under a curated upper term and parses the resulting OBO Graph JSON. Emits both nodes (stubs + ancestors up to the upper term, with label/synonyms/xrefs) and biolink:subclass_of edges (one per is_a axiom in the module). MICRO’s two IRI shapes — standard obo/MICRO_xxxx and legacy obo/MicrO.owl/MICRO_xxxx — are handled by passing both forms to ROBOT for every lower term. Both OWLs are pinned together with the modules they import, which resolve through a local catalog while ROBOT runs with network access refused.
Writes per-ontology TSVs to data/transformed/ontologies_stubs/{ncit,mesh,bto,po,micro}_nodes.tsv carrying the canonical KGX node columns. For PO and MICRO, {po,micro}_edges.tsv is also written with the canonical KGX edge columns.

Note for downstream consumers: if a KG built with this transform is ever merged with a kg-microbe-biomedical KG that loads any of these ontologies fully, biolink merge semantics will union nodes — the stub node here is a strict subset of what the full ontology would emit (label/synonym/xref + optional rdfs:subClassOf ancestors), so the union will simply pick the fuller record.

class kg_microbe.transform_utils.ontologies_stubs.ontologies_stubs_transform.OntologiesStubsTransform(input_dir=None, output_dir=None)

Bases: Transform

Emit one labelled stub node per referenced NCIT / mesh / BTO / PO / MICRO CURIE.

DATA_INPUTS: tuple = ('mappings/ncit_category_dispositions.tsv', 'mappings/canonical/chemical_mappings.tsv', 'mappings/canonical/enzyme_mappings.tsv', 'mappings/canonical/enzyme_name_to_go.tsv', 'mappings/canonical/metpo_alias_mappings.tsv', 'mappings/canonical/pathway_mappings.tsv', 'mappings/canonical/phenotype_mappings.tsv', 'mappings/canonical/special_chemical_mappings.tsv', 'mappings/ingredient_mappings.sssom.tsv', 'mappings/isolation_source_to_ontology.tsv', 'mappings/kgmicrobe_unified_entity_mappings.sssom.tsv.gz', 'mappings/madin_environment_id_corrections.tsv')

Derived from the collector’s own list rather than restated, so the two cannot drift (#839). Hand-maintaining it declared 1 of the 11 files this transform reads, which made kgm-freshness-check — which consults nothing but DATA_INPUTS — report the output fresh after a change to any of the other ten. That is the #812 blind spot, and this was the last transform still carrying it.

Note this makes the SSSOM dependency visible: DEFAULT_MAPPING_PATHS[0] is the unified mapping set, so ontologies_stubs is not independent of an incoming MIM release, which the one-file declaration implied it was. Paths outside the repo are skipped rather than raising. This runs at class-definition time, so relative_to raising would surface as an ImportError from a tuple comprehension — breaking every importer, and pointing nowhere near the entry that caused it (#841). An out-of-repo path is not git-checkable anyway, so dropping it loses no signal.

TRANSFORM_INPUTS: tuple = ('ontologies',): Finalization reads the ontologies output’s declarations, so CHEBI terms MicrO’s edges reference are not declared again here (#1511).

begin_consumed_inputs(): Start the native authority guard afresh only when a new producer run begins.

classmethod missing_prerequisites(input_dir=None)

Report a missing pinned ROBOT, java, pinned import module or pinned release, or an unusable ncit.db.

ROBOT builds the PO and MICRO hierarchies, and their imports must be on disk because they are no longer fetched; any of these missing used to fail only after the hours-long ontologies source. ncit.db gets the ontologies source’s check: this source reads an existing copy as it is, so one left beside a newer ncit.db.gz would be read silently (#1536).

Return type:: List[str]

run(data_file=None, **kwargs)

Collect stub CURIEs and emit per-ontology node (and, where configured, edge) TSVs.

Dispatches per STUB_ONTOLOGY_SOURCES source_type:

semsql — _write_stub_nodes_from_semsql() writes {prefix}_nodes.tsv only.
semsql_mireot — _write_stub_module_from_semsql_walk() writes {prefix}_nodes.tsv AND {prefix}_edges.tsv after walking rdfs:subClassOf ancestors via OAK to the curated upper_terms.
owl_mireot — _write_stub_module_from_mireot() writes {prefix}_nodes.tsv AND {prefix}_edges.tsv after running robot extract --method MIREOT against the source OWL.

Parameters:

data_file – Unused (kept for the base-class signature). The transform discovers its inputs from the mapping TSVs and the input files in input_base_dir.
kwargs – Absorbs forward-compatible kwargs (e.g. show_status) that the transform dispatcher in kg_microbe.transform.transform passes to non-ontologies transforms. The stub transform has no progress bar to toggle, so the flag is intentionally ignored.

Return type:

None

verify_native_inputs(*, byte_verified=False, output_dir=None): Check native guards; only the byte-verifying consumed-input checkpoint may skip rehashing.

classmethod verify_recorded_native_inputs(report, report_path, *, admission=None): Verify saved NCIT provenance and retain locator/absence checks for public merge admission.

Module contents

Ontologies-stubs transform package.

class kg_microbe.transform_utils.ontologies_stubs.OntologiesStubsTransform(input_dir=None, output_dir=None)

Bases: Transform

Emit one labelled stub node per referenced NCIT / mesh / BTO / PO / MICRO CURIE.

DATA_INPUTS: tuple = ('mappings/ncit_category_dispositions.tsv', 'mappings/canonical/chemical_mappings.tsv', 'mappings/canonical/enzyme_mappings.tsv', 'mappings/canonical/enzyme_name_to_go.tsv', 'mappings/canonical/metpo_alias_mappings.tsv', 'mappings/canonical/pathway_mappings.tsv', 'mappings/canonical/phenotype_mappings.tsv', 'mappings/canonical/special_chemical_mappings.tsv', 'mappings/ingredient_mappings.sssom.tsv', 'mappings/isolation_source_to_ontology.tsv', 'mappings/kgmicrobe_unified_entity_mappings.sssom.tsv.gz', 'mappings/madin_environment_id_corrections.tsv')

Derived from the collector’s own list rather than restated, so the two cannot drift (#839). Hand-maintaining it declared 1 of the 11 files this transform reads, which made kgm-freshness-check — which consults nothing but DATA_INPUTS — report the output fresh after a change to any of the other ten. That is the #812 blind spot, and this was the last transform still carrying it.

Note this makes the SSSOM dependency visible: DEFAULT_MAPPING_PATHS[0] is the unified mapping set, so ontologies_stubs is not independent of an incoming MIM release, which the one-file declaration implied it was. Paths outside the repo are skipped rather than raising. This runs at class-definition time, so relative_to raising would surface as an ImportError from a tuple comprehension — breaking every importer, and pointing nowhere near the entry that caused it (#841). An out-of-repo path is not git-checkable anyway, so dropping it loses no signal.

TRANSFORM_INPUTS: tuple = ('ontologies',): Finalization reads the ontologies output’s declarations, so CHEBI terms MicrO’s edges reference are not declared again here (#1511).

begin_consumed_inputs(): Start the native authority guard afresh only when a new producer run begins.

classmethod missing_prerequisites(input_dir=None)

Report a missing pinned ROBOT, java, pinned import module or pinned release, or an unusable ncit.db.

ROBOT builds the PO and MICRO hierarchies, and their imports must be on disk because they are no longer fetched; any of these missing used to fail only after the hours-long ontologies source. ncit.db gets the ontologies source’s check: this source reads an existing copy as it is, so one left beside a newer ncit.db.gz would be read silently (#1536).

Return type:: List[str]

run(data_file=None, **kwargs)

Collect stub CURIEs and emit per-ontology node (and, where configured, edge) TSVs.

Dispatches per STUB_ONTOLOGY_SOURCES source_type:

semsql — _write_stub_nodes_from_semsql() writes {prefix}_nodes.tsv only.
semsql_mireot — _write_stub_module_from_semsql_walk() writes {prefix}_nodes.tsv AND {prefix}_edges.tsv after walking rdfs:subClassOf ancestors via OAK to the curated upper_terms.
owl_mireot — _write_stub_module_from_mireot() writes {prefix}_nodes.tsv AND {prefix}_edges.tsv after running robot extract --method MIREOT against the source OWL.

Parameters:

data_file – Unused (kept for the base-class signature). The transform discovers its inputs from the mapping TSVs and the input files in input_base_dir.
kwargs – Absorbs forward-compatible kwargs (e.g. show_status) that the transform dispatcher in kg_microbe.transform.transform passes to non-ontologies transforms. The stub transform has no progress bar to toggle, so the flag is intentionally ignored.

Return type:

None

verify_native_inputs(*, byte_verified=False, output_dir=None): Check native guards; only the byte-verifying consumed-input checkpoint may skip rehashing.

classmethod verify_recorded_native_inputs(report, report_path, *, admission=None): Verify saved NCIT provenance and retain locator/absence checks for public merge admission.