kg_microbe.transform_utils.ontologies package

Submodules

kg_microbe.transform_utils.ontologies.ontologies_transform module

Ontology transform module.

kg_microbe.transform_utils.ontologies.ontologies_transform.METAMODEL_NODE_PREFIXES = frozenset({'dc', 'dct', 'dcterms', 'doap', 'foaf', 'oio', 'owl', 'pav', 'rdf', 'rdfs', 'skos', 'terms'})

Prefixes of the annotation and metadata vocabulary an OWL file annotates with – never anything it defines. KGX’s OBO-JSON loader emits a node for each one it encounters, so rdfs:label, owl:deprecated, dc:title and dcterms:license arrived as biolink:OntologyClass nodes connected to nothing: 55 distinct ids, 172 rows across ten ontologies, 0 edges, 36 of them in the shipped graph (#1023). Matching on the id column of a nodes file only – skos:closeMatch as a relation value is legitimate and lives in a different column of a different file.

class kg_microbe.transform_utils.ontologies.ontologies_transform.OntologiesTransform(input_dir=None, output_dir=None)

Bases: Transform

OntologyTransform parses an Obograph JSON form of an Ontology into nodes nad edges.

DATA_INPUTS: tuple = ('mappings/foodon_model_dispositions.tsv', 'mappings/ontology_self_loop_exclusions.tsv', 'mappings/native_ro_predicate_dispositions.json', 'mappings/foodon_kernel_quarantine.json', 'mappings/ontology_parent_dispositions.json', 'mappings/upa_reaction_projection_dispositions.json')

Repo-relative curation files this transform reads, beyond its own data/raw/ download.

Declared so freshness tooling can tell that an output is stale against its data rather than only its code. Without it a mapping correction lands, every consumer keeps reporting FRESH, and a re-merge silently ships the old groundings: #778 corrected 16 isolation-source ids and #786 rewrote the unified chemical SSSOM, and the merged KG built afterwards still asserted 75 organisms isolated from a “Cell Line”, because nothing re-ran the transforms that read those files (#812).

Paths are relative to the repo root. Keep them tracked in git — the freshness check uses commit time, not mtime, because git checkout rewrites mtimes without changing content (#797).

List every curation file read, not a representative one. A partial declaration fails silently and looks identical to a complete one: ontologies_stubs declared 1 of the 11 files it read and was reported fresh after changes to the other ten (#839). Where the set comes from a constant, derive this from it rather than restating it.

ONTOLOGY_KNOWLEDGE_SOURCES = {'chebi': 'infores:chebi', 'ec': 'infores:ec', 'envo': 'infores:envo', 'foodon': 'infores:foodon', 'go': 'infores:go', 'hp': 'infores:hp', 'metpo': 'infores:metpo', 'mondo': 'infores:mondo', 'ncbitaxon': 'infores:ncbitaxon', 'pato': 'infores:pato', 'ro': 'infores:ro', 'taxrank': 'infores:taxrank', 'uberon': 'infores:uberon', 'upa': 'infores:upa'}

REQUIRED_AUDIT_FILES: tuple = ('ontology_metadata_exclusions.tsv', 'native_relation_dispositions.tsv', 'foodon_kernel_quarantine.tsv', 'ontology_parent_dispositions.tsv', 'upa_route_review.tsv')

Producer-owned disposition reports whose emitted bytes must survive finalization/merge.

REQUIRED_CONSUMED_INPUTS: tuple = ('ontology_native_relation_policy', 'ontology_foodon_kernel_policy', 'ontology_parent_disposition_policy', 'upa_route_policy')

Named generated inputs that must actually be read before fresh finalization. Unlike DATA_INPUTS, these are checked after upstream producers have run.

bind_finalization_members(names)

Record the graph files of the record being finalized; the directory may hold others.

decompress(data_file)

Unzip file.

finalize(*, file_prefix='', fresh_run=False)

Make the successful source-publication dialect boundary explicit for native verification.

classmethod missing_prerequisites(input_dir=None, *, ontology=None)

Report each reviewed upstream input that is missing, stale or not the reviewed release.

Reviewed policies pin the OWL and JSON releases their content comes from, and the Biolink model, but check them only when their source finalizes, after hours of NCBITaxon and ChEBI work. Checking first makes a stale download fail in seconds (#1420). A single ontology runs under this source and is checked for the files its own run reads (#1534).

Parameters:

• input_dir (Optional[Path]) – Effective raw input directory; None means the default.

• ontology (Optional[str]) – The single ontology selected, or None for the whole source.

Return type:

List[str]

parse(name, data_file, source)

Process the data_file.

Parameters:

• name (str) – Name of the ontology.

• data_file (Optional[Path]) – data file to parse.

• source (str) – Source name.

Return type:

None

Returns:

None.

post_process(name)

Post process specific nodes and edges files.

property producer_native_inputs

Expose the versioned audit-driven native contract for source finalization.

run(data_file=None, show_status=True)

Transform an ontology.

Parameters:

data_file (Union[Path, None, str]) – data file to parse

Return type:

None

Returns:

None.

verify_native_inputs(*, byte_verified=False, output_dir=None)

Preserve native RO role identities after run completion and during finalization.

classmethod verify_recorded_native_inputs(report, report_path, *, admission=None)

Validate persisted audit/native bindings; never rewrite graph rows at merge time.

Module contents

Ontologies transform module.

class kg_microbe.transform_utils.ontologies.OntologiesTransform(input_dir=None, output_dir=None)

Bases: Transform

OntologyTransform parses an Obograph JSON form of an Ontology into nodes nad edges.

DATA_INPUTS: tuple = ('mappings/foodon_model_dispositions.tsv', 'mappings/ontology_self_loop_exclusions.tsv', 'mappings/native_ro_predicate_dispositions.json', 'mappings/foodon_kernel_quarantine.json', 'mappings/ontology_parent_dispositions.json', 'mappings/upa_reaction_projection_dispositions.json')

Repo-relative curation files this transform reads, beyond its own data/raw/ download.

Declared so freshness tooling can tell that an output is stale against its data rather than only its code. Without it a mapping correction lands, every consumer keeps reporting FRESH, and a re-merge silently ships the old groundings: #778 corrected 16 isolation-source ids and #786 rewrote the unified chemical SSSOM, and the merged KG built afterwards still asserted 75 organisms isolated from a “Cell Line”, because nothing re-ran the transforms that read those files (#812).

Paths are relative to the repo root. Keep them tracked in git — the freshness check uses commit time, not mtime, because git checkout rewrites mtimes without changing content (#797).

List every curation file read, not a representative one. A partial declaration fails silently and looks identical to a complete one: ontologies_stubs declared 1 of the 11 files it read and was reported fresh after changes to the other ten (#839). Where the set comes from a constant, derive this from it rather than restating it.

ONTOLOGY_KNOWLEDGE_SOURCES = {'chebi': 'infores:chebi', 'ec': 'infores:ec', 'envo': 'infores:envo', 'foodon': 'infores:foodon', 'go': 'infores:go', 'hp': 'infores:hp', 'metpo': 'infores:metpo', 'mondo': 'infores:mondo', 'ncbitaxon': 'infores:ncbitaxon', 'pato': 'infores:pato', 'ro': 'infores:ro', 'taxrank': 'infores:taxrank', 'uberon': 'infores:uberon', 'upa': 'infores:upa'}

REQUIRED_AUDIT_FILES: tuple = ('ontology_metadata_exclusions.tsv', 'native_relation_dispositions.tsv', 'foodon_kernel_quarantine.tsv', 'ontology_parent_dispositions.tsv', 'upa_route_review.tsv')

Producer-owned disposition reports whose emitted bytes must survive finalization/merge.

REQUIRED_CONSUMED_INPUTS: tuple = ('ontology_native_relation_policy', 'ontology_foodon_kernel_policy', 'ontology_parent_disposition_policy', 'upa_route_policy')

Named generated inputs that must actually be read before fresh finalization. Unlike DATA_INPUTS, these are checked after upstream producers have run.

bind_finalization_members(names)

Record the graph files of the record being finalized; the directory may hold others.

decompress(data_file)

Unzip file.

finalize(*, file_prefix='', fresh_run=False)

Make the successful source-publication dialect boundary explicit for native verification.

classmethod missing_prerequisites(input_dir=None, *, ontology=None)

Report each reviewed upstream input that is missing, stale or not the reviewed release.

Reviewed policies pin the OWL and JSON releases their content comes from, and the Biolink model, but check them only when their source finalizes, after hours of NCBITaxon and ChEBI work. Checking first makes a stale download fail in seconds (#1420). A single ontology runs under this source and is checked for the files its own run reads (#1534).

Parameters:

• input_dir (Optional[Path]) – Effective raw input directory; None means the default.

• ontology (Optional[str]) – The single ontology selected, or None for the whole source.

Return type:

List[str]

parse(name, data_file, source)

Process the data_file.

Parameters:

• name (str) – Name of the ontology.

• data_file (Optional[Path]) – data file to parse.

• source (str) – Source name.

Return type:

None

Returns:

None.

post_process(name)

Post process specific nodes and edges files.

property producer_native_inputs

Expose the versioned audit-driven native contract for source finalization.

run(data_file=None, show_status=True)

Transform an ontology.

Parameters:

data_file (Union[Path, None, str]) – data file to parse

Return type:

None

Returns:

None.

verify_native_inputs(*, byte_verified=False, output_dir=None)

Preserve native RO role identities after run completion and during finalization.

classmethod verify_recorded_native_inputs(report, report_path, *, admission=None)

Validate persisted audit/native bindings; never rewrite graph rows at merge time.