kg_microbe.transform_utils.madin_etal package

Submodules

kg_microbe.transform_utils.madin_etal.madin_etal module

Transform the Madin etal data from NCBI and GTDB.

kg_microbe.transform_utils.madin_etal.madin_etal.ENVIRONMENT_ID_CORRECTIONS_FILE = PosixPath('/home/runner/work/kg-microbe/kg-microbe/mappings/madin_environment_id_corrections.tsv')

Corrections to environments.csv, which is downloaded rather than tracked, so a fix to data/raw/ is erased by the next kg download.

class kg_microbe.transform_utils.madin_etal.madin_etal.MadinEtAlTransform(input_dir, output_dir, nlp=True)

Bases: Transform

Ingest Madin et al dataset (NCBI/GTDB).

Essentially just ingests and transforms this file: https://github.com/bacteria-archaea-traits/bacteria-archaea-traits/blob/master/output/condensed_traits_NCBI.csv And extracts the following columns:

  • tax_id

  • org_name

  • metabolism

  • pathways

  • carbon_substrates

  • cell_shape

  • range_salinity

  • motility

  • gram_stain

  • sporulation

  • isolation_source

Also implements:

  • OAK to run NLP via the ‘ner_utils’ module and

  • ROBOT using ‘robot_utils’ module.

DATA_INPUTS: tuple = ('mappings/kgmicrobe_unified_entity_mappings.sssom.tsv.gz',)

Repo-relative curation files this transform reads, beyond its own data/raw/ download.

Declared so freshness tooling can tell that an output is stale against its data rather than only its code. Without it a mapping correction lands, every consumer keeps reporting FRESH, and a re-merge silently ships the old groundings: #778 corrected 16 isolation-source ids and #786 rewrote the unified chemical SSSOM, and the merged KG built afterwards still asserted 75 organisms isolated from a “Cell Line”, because nothing re-ran the transforms that read those files (#812).

Paths are relative to the repo root. Keep them tracked in git — the freshness check uses commit time, not mtime, because git checkout rewrites mtimes without changing content (#797).

List every curation file read, not a representative one. A partial declaration fails silently and looks identical to a complete one: ontologies_stubs declared 1 of the 11 files it read and was reported fresh after changes to the other ten (#839). Where the set comes from a constant, derive this from it rather than restating it.

run(data_file=None, show_status=True)

Call method and perform needed transformations for trait data (NCBI/GTDB).

Parameters:

data_file (Union[Path, None, str]) – Input file name.

Module contents

MadinEtAl transform.

class kg_microbe.transform_utils.madin_etal.MadinEtAlTransform(input_dir, output_dir, nlp=True)

Bases: Transform

Ingest Madin et al dataset (NCBI/GTDB).

Essentially just ingests and transforms this file: https://github.com/bacteria-archaea-traits/bacteria-archaea-traits/blob/master/output/condensed_traits_NCBI.csv And extracts the following columns:

  • tax_id

  • org_name

  • metabolism

  • pathways

  • carbon_substrates

  • cell_shape

  • range_salinity

  • motility

  • gram_stain

  • sporulation

  • isolation_source

Also implements:

  • OAK to run NLP via the ‘ner_utils’ module and

  • ROBOT using ‘robot_utils’ module.

DATA_INPUTS: tuple = ('mappings/kgmicrobe_unified_entity_mappings.sssom.tsv.gz',)

Repo-relative curation files this transform reads, beyond its own data/raw/ download.

Declared so freshness tooling can tell that an output is stale against its data rather than only its code. Without it a mapping correction lands, every consumer keeps reporting FRESH, and a re-merge silently ships the old groundings: #778 corrected 16 isolation-source ids and #786 rewrote the unified chemical SSSOM, and the merged KG built afterwards still asserted 75 organisms isolated from a “Cell Line”, because nothing re-ran the transforms that read those files (#812).

Paths are relative to the repo root. Keep them tracked in git — the freshness check uses commit time, not mtime, because git checkout rewrites mtimes without changing content (#797).

List every curation file read, not a representative one. A partial declaration fails silently and looks identical to a complete one: ontologies_stubs declared 1 of the 11 files it read and was reported fresh after changes to the other ten (#839). Where the set comes from a constant, derive this from it rather than restating it.

run(data_file=None, show_status=True)

Call method and perform needed transformations for trait data (NCBI/GTDB).

Parameters:

data_file (Union[Path, None, str]) – Input file name.