kg_microbe.transform_utils.microbedecoder package

Submodules

kg_microbe.transform_utils.microbedecoder.microbedecoder module

MicrobeDecoder transform (Hackmann & Zhang, Sci Adv 2023).

Ingests the wide per-LPSN-strain CSV MicrobeDecoder publishes on GitHub — Shiny/MicrobeDecoder/data/database/database.zip — into KGX-format nodes and edges. The database is the actively-maintained successor to FermentationExplorer and pre-joins four curated fermentation-metabolism sources KG-Microbe does not otherwise cover:

  • Bergey’s Manual of Systematics of Archaea and Bacteria — expert curation of Type_of_metabolism, Major_end_products, Minor_end_products, Substrates_for_end_products

  • VPI Anaerobe Laboratory Manual — independent second-opinion fermentation profiles for anaerobes

  • Primary literature — hand-curated end-products with DOI/PMID citations

  • FAPROTAX — functional labels joined at strain granularity

Also emits a biolink:close_match crosswalk from every lpsn:<LPSN_ID> row to NCBITaxon:, ncbi.assembly: (or GTDB: taxonomy strings), gold:, and IMG:. BacDive crosswalks use kgmicrobe.strain:bacdive_* subsumption.

Design decisions locked from the plan-mode Q&A:

  1. Node identity: attach every edge to the existing lpsn:<LPSN_ID> node emitted by the LPSN transform. This transform emits edges only (plus terminal nodes for end-products / cross-refs / provisional placeholders); it does not re-emit LPSN taxon nodes.

  2. BacDive_* replay: ingested with primary_knowledge_source = infores:microbedecoder so the paper’s 2024-11-07 BacDive snapshot stays reproducible. The fresher bacdive transform is authoritative in the merged KG; merge-time dedup keeps both edges around, distinguishable by provenance.

License: CC BY 4.0 (matches the upstream repo).

class kg_microbe.transform_utils.microbedecoder.microbedecoder.MicrobeDecoderTransform(input_dir=None, output_dir=None, chemical_loader=None)

Bases: Transform

Transform the MicrobeDecoder wide CSV into KGX nodes and edges.

DATA_INPUTS: tuple = ('mappings/kgmicrobe_unified_entity_mappings.sssom.tsv.gz',)

Repo-relative curation files this transform reads, beyond its own data/raw/ download.

Declared so freshness tooling can tell that an output is stale against its data rather than only its code. Without it a mapping correction lands, every consumer keeps reporting FRESH, and a re-merge silently ships the old groundings: #778 corrected 16 isolation-source ids and #786 rewrote the unified chemical SSSOM, and the merged KG built afterwards still asserted 75 organisms isolated from a “Cell Line”, because nothing re-ran the transforms that read those files (#812).

Paths are relative to the repo root. Keep them tracked in git — the freshness check uses commit time, not mtime, because git checkout rewrites mtimes without changing content (#797).

List every curation file read, not a representative one. A partial declaration fails silently and looks identical to a complete one: ontologies_stubs declared 1 of the 11 files it read and was reported fresh after changes to the other ten (#839). Where the set comes from a constant, derive this from it rather than restating it.

TRANSFORM_INPUTS: tuple = ('lpsn', 'gold', 'gtdb')

Reads this transform’s output; see Transform.TRANSFORM_INPUTS (#845).

TSV_QUOTING = 3
run(data_file=None, show_status=True)

Emit nodes.tsv and edges.tsv from database.csv.

Return type:

None

Parameters

data_file:

Optional override for the input path. Accepts either a zip (extracted here) or an already-unzipped CSV. Default resolves to <input_base_dir>/microbedecoder_database.zip when the zip is present, else <input_base_dir>/database.csv.

show_status:

Accepted for CLI compatibility; unused because the parse pass is fast enough (~8 K rows) that a progress bar isn’t worth pulling tqdm in for.

kg_microbe.transform_utils.microbedecoder.utils module

Helpers for the MicrobeDecoder transform.

Column definitions and light per-cell parsers so microbedecoder.py stays focused on row → node/edge emission and cross-ref wiring. All heavy semantic work (label → CURIE resolution, NER fallback) reuses existing utilities in kg_microbe.utils — see the transform module docstring for the full list.

kg_microbe.transform_utils.microbedecoder.utils.LPSN_GENUS_COLUMN = 'LPSN_Genus'

Naming columns, used to label stubs for ids the lpsn transform cannot supply (botanical-code cyanobacteria absent from lpsn_gss.csv, #811).

kg_microbe.transform_utils.microbedecoder.utils.crosswalk_curie(local_id, prefix, source_prefix=None)

Normalize a crosswalk token to the identifier its owner transform emits.

Return type:

str

kg_microbe.transform_utils.microbedecoder.utils.format_citation(raw)

Return all recognized PMID: / doi: CURIEs, pipe-separated, or None.

Accepts free text; extracts each explicit DOI or PMID it finds. Returns None when the cell is empty (per is_empty_cell()) or contains no recognisable citation token — the caller can then decide whether to drop the citation attribute or store the raw prose.

Return type:

Optional[str]

kg_microbe.transform_utils.microbedecoder.utils.is_empty_cell(cell)

Report whether a cell value should be treated as absent.

Return type:

bool

kg_microbe.transform_utils.microbedecoder.utils.iter_metabolism_columns(row)

Yield (group_label, fields) for every metabolism source with data.

fields carries the raw cell values for each column-role the group declares (see METABOLISM_GROUPS). Groups whose columns are all empty (via is_empty_cell()) are skipped so the caller only iterates over sources that actually contributed to this row. Missing columns (source doesn’t declare that role) simply are absent from fields.

Return type:

Iterable[Tuple[str, Dict[str, object]]]

kg_microbe.transform_utils.microbedecoder.utils.slugify_label(label)

Produce a stable CURIE-safe slug from a raw source label.

Return type:

str

kg_microbe.transform_utils.microbedecoder.utils.split_multivalue(cell)

Split outside brackets, preserving numeric chemical locants such as 2,3-butanediol.

Parameters:

cell (object) – Raw cell value.

Return type:

List[str]

Returns:

Trimmed, non-empty tokens.

kg_microbe.transform_utils.microbedecoder.utils.split_multivalue_comma_only(cell)

Split a multi-value cell on comma ONLY (preserves in-value semicolons).

Use for identity-crosswalk cells (IMG_Genome_ID and friends) where a raw semicolon is part of the identifier (GTDB_ID rank separator) rather than a value separator.

Return type:

List[str]

kg_microbe.transform_utils.microbedecoder.utils.split_snapshot_values(column, cell)

Preserve a lone assay sign without decoding its biological interpretation.

The generic parser treats ‘-’ as missing, but these two source columns report ‘+’ and ‘-’ assay tokens. A mixed ‘+,-’ already preserved both; the lone ‘-’ must also survive. Other columns retain their existing missing-value and multivalue conventions.

Return type:

List[str]

Module contents

MicrobeDecoder transform (Hackmann & Zhang 2023, Sci Adv).