kg_microbe.transform_utils.bacdive package

Submodules

kg_microbe.transform_utils.bacdive.ancestry_authority module

Own a private, byte-bound BacDive ancestor-query authority.

The caller/shared OAK adapter identifies its actual selected SQLite file only; no caller engine/session is queried, reset, or closed. Immutable SQLite mode does not provide a filesystem snapshot: retained byte and locator checks bind the independently owned query connection throughout its lifecycle.

class kg_microbe.transform_utils.bacdive.ancestry_authority.FreshAncestryAuthority(selected_path, *, admission=None)

Bases: object

Keep one owned immutable-mode SQLite connection fixed for a query batch.

immutable=1 tells SQLite to avoid file locks and journals. It does NOT make the filesystem immutable. Hashes before opening and after the complete query batch, per-query file stamps, original lexical bindings, and absent sidecar checks are mandatory. Unsupported adapters are refused, not guessed.

ancestors(identifier): Run the unchanged OAK query and retain its computed, not asserted-native, result.

close(): Close only this object’s owned connection/session on every normal or failure path.

property contract: Return only a successfully completed actual-query contract.

finish(): Seal the exact query batch and rehash authority bytes after all queries, before publication.

classmethod from_recorded_authority(recorded, *, admission=None)

Re-admit the original authority and replay with byte-identical installed query code.

A different interpreter can have the same pinned OAK file at a different path. Preserve and rehash the original producer’s implementation path; separately bind the current interpreter’s actual imported code. Never describe that current replay origin as the original producer’s import.

classmethod from_resolved_adapter(adapter, *, admission=None): Select the real resolved adapter engine’s SQLite file, not configured ncbitaxon.db.

verify(*, full=False): Retain fatal failures, exact native identity and monotonic lifecycle after closure.

kg_microbe.transform_utils.bacdive.bacdive module

BacDive KG transform.

Input: any file in data/raw/ (that was downloaded by placing a URL in incoming.txt/yaml and running run.py download.

Output: transformed data in data/raw/bacdive_strains.json:

Output these two files: - nodes.tsv - edges.tsv

class kg_microbe.transform_utils.bacdive.bacdive.BacDiveTransform(input_dir=None, output_dir=None)

Bases: Transform

Template for how the transform class would be designed.

DATA_INPUTS: tuple = ('mappings/isolation_source_to_ontology.tsv', 'mappings/kgmicrobe_unified_entity_mappings.sssom.tsv.gz', 'mappings/canonical/bacdive_ec_substrate_corrections.tsv', 'mappings/bacdive_isolation_category_policy.json', 'mappings/bacdive_clinical_mesh_policy.json', 'mappings/bacdive_ncit_category_claims.json', 'mappings/bacdive_supplied_taxid_dispositions.json')

Repo-relative curation files this transform reads, beyond its own data/raw/ download.

Declared so freshness tooling can tell that an output is stale against its data rather than only its code. Without it a mapping correction lands, every consumer keeps reporting FRESH, and a re-merge silently ships the old groundings: #778 corrected 16 isolation-source ids and #786 rewrote the unified chemical SSSOM, and the merged KG built afterwards still asserted 75 organisms isolated from a “Cell Line”, because nothing re-ran the transforms that read those files (#812).

Paths are relative to the repo root. Keep them tracked in git — the freshness check uses commit time, not mtime, because git checkout rewrites mtimes without changing content (#797).

List every curation file read, not a representative one. A partial declaration fails silently and looks identical to a complete one: ontologies_stubs declared 1 of the 11 files it read and was reported fresh after changes to the other ten (#839). Where the set comes from a constant, derive this from it rather than restating it.

OPTIONAL_RAW_CONSUMED_INPUTS: tuple = (('bacdive_taxonomy_lpsn_gss', 'lpsn_gss.csv'),): Named optional reads relative to the effective input_base_dir, including explicit absence. Kept separate so repository-relative optional inputs never silently relocate.

REQUIRED_AUDIT_FILES: tuple = ('ec_substrate_corrections.tsv', 'isolation_category_identity_dispositions.tsv', 'clinical_isolation_category_dispositions.tsv', 'ncit_isolation_category_dispositions.tsv', 'taxonomy_assertion_dispositions.tsv'): Producer-owned disposition reports whose emitted bytes must survive finalization/merge.

REQUIRED_CONSUMED_INPUTS: tuple = ('ec_substrate_correction_policy', 'ec_substrate_legacy_mapping', 'bacdive_isolation_category_policy', 'bacdive_isolation_category_mapping', 'bacdive_clinical_mesh_policy', 'bacdive_clinical_mesh_native', 'bacdive_ncit_category_claim_policy', 'bacdive_isolation_category_source', 'bacdive_supplied_taxid_dispositions'): Named generated inputs that must actually be read before fresh finalization. Unlike DATA_INPUTS, these are checked after upstream producers have run.

TRANSFORM_INPUTS: tuple = ('ontologies',): Reads ontologies/ncbitaxon_nodes.tsv and ontologies/chebi_nodes.tsv via NCBITAXON_NODES_FILE / CHEBI_NODES_FILE. Undeclared until #1035: the path lives in constants.py, so the cross-transform guard never saw it.

begin_consumed_inputs(): Reset source-local claim state only when starting a genuine producer run.

finalize(*, file_prefix='', fresh_run=False): Record the canonical TSV dialect only after successful source publication.

property producer_native_inputs: Retain mandatory native claim evidence even for genuinely zero-target runs.

run(data_file=None, show_status=True): Run the transformation.

verify_native_inputs(*, byte_verified=False, output_dir=None): Keep native roles and lossless source-claim dispositions fixed through publication.

classmethod verify_recorded_native_inputs(report, report_path, *, admission=None): Enforce native cohort conservation during read-only merge admission.

kg_microbe.transform_utils.bacdive.clinical_categories module

Retain four finite native clinical classifications without inventing external site identity.

kg_microbe.transform_utils.bacdive.clinical_categories.emit_occurrence(transform, category_index, category, field, value, *, isolation, organism_id, native_record_id, node_writer, edge_writer, audit_writer, policy): Intercept exactly one original raw leaf; never mutate its group or PRIDE coordinates.

kg_microbe.transform_utils.bacdive.clinical_categories.prepare_policy(transform, *, policy_path=PosixPath('/home/runner/work/kg-microbe/kg-microbe/mappings/bacdive_clinical_mesh_policy.json'), native_path=None): Require actual native evidence and exact admitted mappings before opening graph outputs.

kg_microbe.transform_utils.bacdive.csv_field_limit module

Read BacDive audits whose cells can outgrow CSV’s field limit (#1571, #1575).

kg_microbe.transform_utils.bacdive.csv_field_limit.csv_field_limit_at_least(size)

Raise CSV’s process-wide field limit to at least size for the block, then restore it.

The audits embed native BacDive records, and a cell can exceed CSV’s default of 131,072 characters. Pass the size in bytes of the file being read: no cell is longer than its file. The caller’s limit is never lowered, and is restored even when the read fails.

Consume the reader inside the block. A generator such as graph_rows resumed after it would parse at the caller’s limit again.

Parameters:: size – The size in bytes of the file the block reads.

kg_microbe.transform_utils.bacdive.ec_substrate_corrections module

Apply finite reviewed legacy EC/substrate corrections without raw or global aliases.

kg_microbe.transform_utils.bacdive.ec_substrate_corrections.apply_corrections(source_rows, policies, *, knowledge_source): Correct only complete reviewed rows; known-scope drift requires a new review.

kg_microbe.transform_utils.bacdive.ec_substrate_corrections.load_policy(stream): Read explicit complete-row curation, never partial names or global old-ID replacements.

kg_microbe.transform_utils.bacdive.ec_substrate_corrections.prepare_corrections(transform, legacy_path, *, policy_path=PosixPath('/home/runner/work/kg-microbe/kg-microbe/mappings/canonical/bacdive_ec_substrate_corrections.tsv')): Bind actual policy/source reads before opening graph outputs, retaining the original admission.

kg_microbe.transform_utils.bacdive.emission module

KGX row-emission helpers for the BacDive transform.

kg_microbe.transform_utils.bacdive.emission.DEPOSIT_CONFLICT_HEADER = ['strain_id', 'parent_count', 'parents', 'bacdive_ids', 'resolution', 'asserted_parent', 'withheld_supplied_taxids', 'other_bacdive_ids']: Header of the deposit-claim report written alongside nodes.tsv / edges.tsv. withheld_supplied_taxids lists citing records whose ambiguous supplied taxids were withheld, as <BacDive key>=<CURIE>,<CURIE> joined by |; other_bacdive_ids lists citing records that contributed neither a claim nor withheld candidates (#1421).

kg_microbe.transform_utils.bacdive.emission.RESOLUTION_COLLAPSED = 'collapsed': The claims sat on one lineage; the shared ancestor was asserted (#898).

kg_microbe.transform_utils.bacdive.emission.RESOLUTION_SUPPRESSED = 'suppressed': The claims were disjoint; no parent was asserted.

kg_microbe.transform_utils.bacdive.emission.RESOLUTION_SUPPRESSED_NO_ANCESTRY = 'suppressed_ancestry_unavailable': Ancestry for at least one claim could not be read, so the claims could not be compared and the deposit was suppressed as a precaution rather than a verdict. Distinguishing this matters – a degraded NCBITaxon adapter otherwise looks exactly like a data problem in BacDive (#897).

kg_microbe.transform_utils.bacdive.emission.RESOLUTION_SUPPRESSED_WITHHELD = 'suppressed_withheld_unresolved': A record whose ambiguous supplied taxids were withheld cites the deposit, and its candidates could not all be shown to be, or lie beneath, one parent that the other citing records support. That is not proof the records disagree, only that no parent can be asserted for all of them (#1421).

class kg_microbe.transform_utils.bacdive.emission.StrainProvenanceWriter(inner_writer, *, knowledge_source, ks_column_index, publications_column_index)

Bases: object

Keep PKS scalar and attach a BacDive record page to strain-derived evidence.

writerow(row, *, publications=None)

Attach explicit item citations and record evidence without changing the primary resource.

Return type:: None

writerows(rows)

Apply provenance augmentation to a sequence of rows.

Return type:: None

kg_microbe.transform_utils.bacdive.emission.contested_deposit_description(parents, withheld=None, others=(), resolution='suppressed')

Explain, on the node itself, why a contested deposit has no edges (#907).

A deposit whose claimants disagreed gets neither a parent nor a link to the records that cited it, which leaves an organism-typed node with a culture number for a label and nothing else – indistinguishable from an ingest gap. The description says which taxa were claimed so the emptiness reads as a decision.

Scoped to BacDive on purpose: 55 of these deposits are the object of an LPSN close_match edge, so a sentence claiming nothing links to the node would be false in the merged graph (#908). This transform cannot see LPSN’s output and should not try to – it states only what it did itself.

Parameters:

parents – {NCBITaxon CURIE: [BacDive record key, ...]} for the deposit.
withheld – {BacDive record key: [NCBITaxon CURIE, ...]} for citing records whose ambiguous supplied taxids were withheld (#1421).
others – BacDive record keys of citing records that contributed neither a claim nor withheld candidates.
resolution – The deposit’s RESOLUTION_* value. A withheld-record suppression is not a proven disagreement, so it is worded as such.

Returns:

One-sentence description naming the conflicting claims.

kg_microbe.transform_utils.bacdive.emission.deposit_conflict_rows(contested, others=None)

Render contested deposit claims as report rows.

The report is the record that a claim existed and what became of it, so a consumer can recover the taxonomy that was suppressed, and see which deposits had theirs coarsened to a shared ancestor.

Parameters:

contested – contested as returned by resolve_deposit_parents().
others – {strain CURIE: [BacDive record key, ...]}, citing records that contributed neither a claim nor withheld candidates.

Returns:

List of rows matching DEPOSIT_CONFLICT_HEADER.

kg_microbe.transform_utils.bacdive.emission.entailed_by_every_claim(parents, ancestors_of)

Return the one claimed taxon that every other claim entails, or None.

Claims can differ without contradicting each other: one record records the species (Brevundimonas vesicularis) and another the strain beneath it (Brevundimonas vesicularis NBRC 12165). The species claim is true either way, so it is the strongest statement all claimants support, and asserting it privileges neither record. Only a claim that was actually made is eligible – falling back to a computed common ancestor would put a taxon nobody claimed (often just Bacteria) on the node.

Parameters:

parents – Iterable of claimed NCBITaxon CURIEs.
ancestors_of – Callable mapping a CURIE to its proper is_a ancestors, or None when no ontology is available.

Returns:

The entailed CURIE, or None when the claims are disjoint.

kg_microbe.transform_utils.bacdive.emission.is_contested(parents, withheld=None, citing=None)

Report whether a deposit’s claimants must be resolved rather than trusted.

Two claimed parents always must. A record whose ambiguous supplied taxids were withheld claims no parent, but its candidates are still evidence about the organism (#1421). It contests the deposit whenever any other record cites it, whether or not that record contributed a claim: a record whose parent came only from a late, name-derived route could otherwise be linked to an unrelated withheld record. Cited by that record alone, it has nothing to contradict.

Parameters:

parents – {NCBITaxon CURIE: [BacDive record key, ...]} for the deposit.
withheld – {BacDive record key: [NCBITaxon CURIE, ...]} for the deposit.
citing – Every BacDive record key that cites the deposit. When None, only claimants and withheld records are counted.

Returns:

True when the deposit belongs in the claims report.

kg_microbe.transform_utils.bacdive.emission.resolution_asserts_a_parent(resolution)

Report whether a resolution left the deposit with a parent taxon.

Used to decide whether a record may be linked to the deposit at all. A deposit that got a parent is one identifier its claimants agree about, so linking to it is sound. One that got none is a designation two unrelated organisms happen to share, and biolink:close_match – symmetric, and mapped to SEMMEDDB:same_as – would put those organisms two hops apart (#899).

Parameters:: resolution – One of the RESOLUTION_* values.
Returns:: True when a parent was asserted.

kg_microbe.transform_utils.bacdive.emission.resolve_deposit_parents(claims, ancestors_of=None, ancestry_failed=None, withheld=None, citing=None)

Decide which culture-collection deposits may assert a parent taxon (#892).

A deposit number such as ATCC 13722 identifies a physical deposit, not a BacDive record, so several records can cite the same one. When they disagree about the taxon, emitting one subclass_of edge per claiming record leaves the deposit node with several parents and nothing to tell them apart, so a consumer that takes “the” parent gets whichever one file order put first.

A parent is asserted when the claimants agree outright, and when they differ only in depth along one lineage – there the shared ancestor is asserted, per entailed_by_every_claim(). Genuinely disjoint claims get no edge.

A record whose ambiguous supplied taxids were withheld supports no parent, but its candidates contest one: the claimed parent survives only if every candidate is that taxon or lies beneath it, so the record belongs to it whichever candidate is right. Otherwise, and whenever the withheld record shares the deposit with a record that claimed nothing, the deposit gets no parent and no record links, with RESOLUTION_SUPPRESSED_WITHHELD (#1421). A candidate is never asserted, not even as a coarser shared parent.

Every contested deposit is reported, whichever way it went, so the report answers “what happened here and why” rather than listing only the half that lost its edge (#898).

Parameters:

claims – {strain CURIE: {NCBITaxon CURIE: [BacDive record key, ...]}}.
ancestors_of – Callable mapping an NCBITaxon CURIE to its proper is_a ancestors. When None, any disagreement is treated as a conflict.
ancestry_failed – Callable reporting whether a CURIE’s ancestry could not be read, used to mark a suppression that is a precaution rather than a verdict (#897). When None, no suppression is marked that way.
withheld – {strain CURIE: {BacDive record key: [NCBITaxon CURIE, ...]}} for citing records whose ambiguous supplied taxids were withheld.
citing – {strain CURIE: [BacDive record key, ...]}, every record that cites each withheld-cited deposit. See is_contested().

Returns:

(resolved, contested). resolved is a sorted list of (strain CURIE, NCBITaxon CURIE) pairs to emit. contested is a sorted list of (strain CURIE, claims, resolution, asserted parent, withheld) for every deposit is_contested() selects; asserted parent is empty when none was, and withheld is empty when no withheld record cited it.

kg_microbe.transform_utils.bacdive.fallback_evidence module

Bind source-derived fallback choices to the existing BacDive taxonomy lifecycle.

This is composed into taxonomy_evidence, not a second producer/finalizer or a graph-repair pass. All native choices are prepared before emission; no output/audit inventory serves as its independent input authority.

class kg_microbe.transform_utils.bacdive.fallback_evidence.FallbackEvidence(entries, snapshots, *, raw_role, selected_path, opener=None, adapter_selector=None, recorded=None, admission)

Bases: object

Retain one completed, source-derived inference plan and its owned native query guard.

property contract: Return detached evidence only after every actual inference record has a plan.

plan(ordinal): Expose a detached selected record plan without permitting cached evidence mutation.

select_ordinary(ordinal, producer_cache): Validate a run-local cache against the independently fixed native decision.

property selected_roles: Return only this child’s immutable selected-input role for the shared row audit.

verify(snapshots): Retain exact source roles, parsed witnesses and owned native authority after completion.

kg_microbe.transform_utils.bacdive.fallback_evidence.recorded_selected_opener(snapshots, admission, role, path): Re-read only the originally bound transformed authority; never fabricate a consumed role.

kg_microbe.transform_utils.bacdive.fallback_native_authority module

Own literal native-statement queries for BacDive taxon inference (#1364).

This reuses1361’s owned-connection/locator/code/sidecar admission design by subclassing it into a NEW instance, never borrowing a deposit child’s session. The base ancestry ledger remains empty; this child’s direct native statement queries have a separate contract composed into source-local taxonomy admission.

class kg_microbe.transform_utils.bacdive.fallback_native_authority.FreshFallbackAuthority(selected_path, *, admission=None)

Bases: FreshAncestryAuthority

Own a distinct actual-read connection and an independent native-query ledger.

candidate_facts(identifier): Rejoin class/rank and asserted native ancestry, not a generic computed closure.

declaration(identifier): Return every literal statement about a requested candidate or asserted ancestor.

exact_label_candidates(label): Retain all exact native labels, not OAK first-result order or synonym guesses.

finish_fallback(): Return this child’s independent complete query contract, not a deposit contract.

prime_exact_labels(names): Prepare at most one exact-name label stream for this independently owned batch.

kg_microbe.transform_utils.bacdive.fallback_selection module

Assess native candidate eligibility for BacDive name-derived taxonomy (#1364).

The caller must supply facts from the separately owned actual native reader and retain this complete decision against the original raw record. Actual-read, complete-source and output accounting belong to the composed fallback/taxonomy guards; these pure scientific rules do not substitute for those contracts.

class kg_microbe.transform_utils.bacdive.fallback_selection.Selection(status, identifier, terminal, evidence_json)

Bases: object

Keep a rejected match distinct from an ordinary lookup miss.

evidence_json: str

identifier: str | None

status: str

terminal: bool

kg_microbe.transform_utils.bacdive.fallback_selection.assess_candidates(query, source_field, source_classification, candidates, *, route='ordinary_rank', match_mode='native_exact_label')

Assess actual native candidates without selecting an alternative synonym or parent.

candidates must include the complete selected tier’s result set, not its first member. A fast selected-label result does not claim a full-database homonym search. Native-reader and original-record bindings are separate mandatory contracts. Every nonempty-but-rejected set is terminal at this selector: it cannot be converted into a lookup miss that silently permits broader family fallthrough. This conservatively covers the reviewed175523 failure without hardcoding a replacement biological identity or trusting a source field’s rank as evidence.

kg_microbe.transform_utils.bacdive.isolation_categories module

Retain reviewed BacDive headings and Agriculture context without false site identity.

kg_microbe.transform_utils.bacdive.isolation_categories.emit_category_attributes(transform, categories, isolation, organism_id, node_writer, edge_writer, audit_writer, policy, *, native_record_id, native_record=None, clinical_handler=None): Emit each reviewed raw occurrence as an attribute and return every other literal value unchanged.

kg_microbe.transform_utils.bacdive.isolation_categories.load_policy(stream): Require the exact finite reviewed scope and immutable primary authority locator.

kg_microbe.transform_utils.bacdive.isolation_categories.prepare_policy(transform, *, policy_path=PosixPath('/home/runner/work/kg-microbe/kg-microbe/mappings/bacdive_isolation_category_policy.json'), mapping_path=PosixPath('/home/runner/work/kg-microbe/kg-microbe/mappings/isolation_source_to_ontology.tsv')): Bind actual policy/mapping reads and reject reintroduced external identities before output.

kg_microbe.transform_utils.bacdive.ncit_category_claims module

Retain four reviewed native leaf observations without unsupported NCIT site identity.

class kg_microbe.transform_utils.bacdive.ncit_category_claims.ClaimGuard(raw_dir, policy, selected, bindings, *, completed=False, admission=None)

Bases: object

Keep immutable actual-read evidence through run, finalization and repeated publication.

property contract: Return a copy, not the mutable live contract or scientific policy.

emit(transform, index, category, field, value, *, record, organism_id, node_writer, edge_writer, audit_writer): Intercept only the exact native leaf while preserving all original array indices.

finish(snapshots): Require every eligible native observation before atomic output contexts publish.

mark_output_ready(snapshots): Seal output readiness after successful atomic writer contexts have exited.

property outputs_ready: Expose a read-only phase view, checked against the retained admission observation.

verify(snapshots, *, output_dir=None, quoting=3, output_ready=None): Do not trust a deleted role, changed cached record or fabricated zero receipt.

verify_output(directory, *, quoting=3): Rejoin the full audit and retained graph fields; preserve exact selected member scope.

kg_microbe.transform_utils.bacdive.ncit_category_claims.load_policy(stream): Require all four reviewed records and both complete original mapping claims.

kg_microbe.transform_utils.bacdive.ncit_category_claims.prepare(transform, *, native_observer=None): Own the actual raw read; a caller cannot substitute cached payload for native bytes.

kg_microbe.transform_utils.bacdive.ncit_category_claims.verify_recorded(contract, snapshots, raw_dir, output_dir, *, members=None, admission=None): Rebuild a finite contract from actual native bytes, never a receipt’s claimed occurrence count.

kg_microbe.transform_utils.bacdive.references module

Resolve explicit BacDive item references without spreading record-wide citations.

kg_microbe.transform_utils.bacdive.references.item_publications(item, references): Return only the publications named by this item’s explicit reference pointer(s).

kg_microbe.transform_utils.bacdive.references.publication_doi(value): Normalize a complete DOI field, excluding catalogue and BacDive record identifiers.

kg_microbe.transform_utils.bacdive.references.reference_dois(record): Index unambiguous publication DOI references; contradictory duplicate IDs stay unresolved.

kg_microbe.transform_utils.bacdive.supplied_taxid_policy module

Finite approved withholding of 97 same-priority, normalized-multivalent records.

kg_microbe.transform_utils.bacdive.supplied_taxid_policy.disposition(record, policy): Retain source claims/order, withholding only reviewed distinct normalized candidates.

kg_microbe.transform_utils.bacdive.supplied_taxid_policy.load_policy(stream): Admit only the reviewed finite policy; no output-derived or global ambiguity rule.

kg_microbe.transform_utils.bacdive.taxonomy_evidence module

Bind source-local BacDive taxonomy evidence for #1353, #1354, #1361 and #1364.

Native supplied-taxid and LPSN selection and inferred native-candidate eligibility are independently replayable. Name-derived parent claims remain predictions, never native supplied taxids.

class kg_microbe.transform_utils.bacdive.taxonomy_evidence.LpsnIndex(stream)

Bases: object

A run-local index built only from the actual optional immutable GSS read.

lookup(block): Keep original multiplicity but decide uniqueness on resolved target IDs.

preview_index(): Return independent copies for legacy narrow preview callers, not production read receipts.

state(): Expose only the parsed scientific state for retained checkpoint verification.

verify(): Reject mutable parsed-cache drift once per checkpoint, not once per record.

class kg_microbe.transform_utils.bacdive.taxonomy_evidence.TaxonomyEvidence(raw_dir, gss, bindings, optional_state, *, supplied_policy, admission=None)

Bases: object

Retain source-local support classifications; withheld candidates contest deposit parents, never support them.

capture_native(payload, snapshots): Observe the exact parsed payload while its original immutable reader is still open.

capture_records(records, snapshots): Use the same complete stream projection during recorded-source rejoining.

classify_deposit(subject, target, claims): Direct support must name the selected parent; mixed ancestor derivations stay derived.

complete(buffered_claims, resolved, contested, audit_writer=None): Reconcile every original claim and write the mandatory audit before publication.

property contract: Return a copy only after complete producer taxonomy observation.

emit_fallback_hierarchy(ordinal, transform, node_writer, edge_writer): Emit the already reconstructed late hierarchy using the existing node writers.

fallback_choice(ordinal, producer_cache): Use actual source/native planning before any inferred parent node or deposit is emitted.

lookup(ordinal, block): Use the captured native block and consumed GSS rather than a constructor-time index.

mark_output_ready(snapshots): Permit graph checks only after writers close, and make that transition irreversible.

native_deposit_claims(): Recreate exact native token multiplicity and the already observed early parent choices.

observe_parent_choice(ordinal, record, target, inferred): Retain the existing selected target and prove whether a supplied native taxid supports it.

observe_selected_parent(ordinal, entry, target, inferred): Retain native supplied choices or an explicitly identified producer derivation.

property outputs_ready: Return a read-only output phase; callers may not erase completion to skip checks.

prepare_fallback(snapshots, *, selected_path, opener, adapter_selector=None, recorded=None): Compose one source-derived child after the existing native observer, before output.

require_fallback(): Refuse a lost child instead of downgrading inference to old unaudited choices.

resolve_deposits(claims, adapter_selector=None, *, recorded_ancestry=None)

Delegate scientific selection to the unchanged resolver with a fresh actual-read callback.

Demand comes from actual resolver callbacks, including repeated calls across deposits. It is not guessed as the union of every candidate ID. The private per-batch cache preserves the old callback’s query reuse.

supplied_parent_withheld(ordinal, record): Keep unresolved supplied IDs distinct from the missing-ID fallback cohort.

verify(snapshots, *, output_dir=None, quoting=3, finalized=False): Retain a failed output checkpoint as well as every mandatory native-role checkpoint.

verify_native(snapshots): Check original roles and complete parsed state; failure remains sticky until a real rerun.

verify_output(directory, *, quoting=3, finalized=False): Require every exact lookup/deposit row while leaving other graph assertions outside this scope.

withheld_deposit_contests()

Return the candidates of withheld records per cited deposit, for contest only.

A record whose same-priority supplied taxids were withheld supports no deposit parent. Its normalized candidates still say which organism it is, so they contest the parent of every deposit it cites; dropping them made a deposit shared with an unrelated organism look uncontested (#1421).

Returns:: {deposit CURIE: {BacDive record ID: [sorted candidate CURIEs]}}.

kg_microbe.transform_utils.bacdive.taxonomy_evidence.prepare(transform): Consume GSS before output creation and wait for the existing actual raw-read observer.

kg_microbe.transform_utils.bacdive.taxonomy_evidence.prepare_fallback(transform, selected_path, adapter_selector): Track an actual immutable selected-label read without retaining constructor lookup caches.

kg_microbe.transform_utils.bacdive.taxonomy_evidence.split_native_contract(recorded): Require the new child while preserving the exact independently reviewed NCIT contract.

kg_microbe.transform_utils.bacdive.taxonomy_evidence.verify_recorded(contract, snapshots, raw_dir, output_dir, *, optional_state, members=None, admission=None)

Rejoin the mandatory complete native cohort and exact lookup decisions during admission.

The mandatory fallback child re-reads the original selected/native inputs, rederives all eligible records and predicts exact current parent rows. This does not upgrade name-derived inference into native supplied-ID observations.

Module contents

BacDive transform.

class kg_microbe.transform_utils.bacdive.BacDiveTransform(input_dir=None, output_dir=None)

Bases: Transform

Template for how the transform class would be designed.

DATA_INPUTS: tuple = ('mappings/isolation_source_to_ontology.tsv', 'mappings/kgmicrobe_unified_entity_mappings.sssom.tsv.gz', 'mappings/canonical/bacdive_ec_substrate_corrections.tsv', 'mappings/bacdive_isolation_category_policy.json', 'mappings/bacdive_clinical_mesh_policy.json', 'mappings/bacdive_ncit_category_claims.json', 'mappings/bacdive_supplied_taxid_dispositions.json')

Repo-relative curation files this transform reads, beyond its own data/raw/ download.

Declared so freshness tooling can tell that an output is stale against its data rather than only its code. Without it a mapping correction lands, every consumer keeps reporting FRESH, and a re-merge silently ships the old groundings: #778 corrected 16 isolation-source ids and #786 rewrote the unified chemical SSSOM, and the merged KG built afterwards still asserted 75 organisms isolated from a “Cell Line”, because nothing re-ran the transforms that read those files (#812).

Paths are relative to the repo root. Keep them tracked in git — the freshness check uses commit time, not mtime, because git checkout rewrites mtimes without changing content (#797).

List every curation file read, not a representative one. A partial declaration fails silently and looks identical to a complete one: ontologies_stubs declared 1 of the 11 files it read and was reported fresh after changes to the other ten (#839). Where the set comes from a constant, derive this from it rather than restating it.

OPTIONAL_RAW_CONSUMED_INPUTS: tuple = (('bacdive_taxonomy_lpsn_gss', 'lpsn_gss.csv'),): Named optional reads relative to the effective input_base_dir, including explicit absence. Kept separate so repository-relative optional inputs never silently relocate.

REQUIRED_AUDIT_FILES: tuple = ('ec_substrate_corrections.tsv', 'isolation_category_identity_dispositions.tsv', 'clinical_isolation_category_dispositions.tsv', 'ncit_isolation_category_dispositions.tsv', 'taxonomy_assertion_dispositions.tsv'): Producer-owned disposition reports whose emitted bytes must survive finalization/merge.

REQUIRED_CONSUMED_INPUTS: tuple = ('ec_substrate_correction_policy', 'ec_substrate_legacy_mapping', 'bacdive_isolation_category_policy', 'bacdive_isolation_category_mapping', 'bacdive_clinical_mesh_policy', 'bacdive_clinical_mesh_native', 'bacdive_ncit_category_claim_policy', 'bacdive_isolation_category_source', 'bacdive_supplied_taxid_dispositions'): Named generated inputs that must actually be read before fresh finalization. Unlike DATA_INPUTS, these are checked after upstream producers have run.

TRANSFORM_INPUTS: tuple = ('ontologies',): Reads ontologies/ncbitaxon_nodes.tsv and ontologies/chebi_nodes.tsv via NCBITAXON_NODES_FILE / CHEBI_NODES_FILE. Undeclared until #1035: the path lives in constants.py, so the cross-transform guard never saw it.

assay_edges_generated: Optional[List]

assay_nodes_generated: Optional[List]

assay_raw_data: Optional[dict]

assay_target_nodes_generated: Optional[List]

begin_consumed_inputs(): Reset source-local claim state only when starting a genuine producer run.

chebi_categories: Dict[str, str]

finalize(*, file_prefix='', fresh_run=False): Record the canonical TSV dialect only after successful source publication.

isolation_source_mappings: Dict[str, tuple]

ncbitaxon_fallback_cache: Dict[tuple, Optional[str]]

ncbitaxon_labels: Dict[str, str]

ncbitaxon_name_to_id: Dict[str, str]

ncbitaxon_synonyms: Dict[str, frozenset]

pathogenicity_mappings: Dict[str, tuple]

phenotype_routing: List[Dict[str, str]]

property producer_native_inputs: Retain mandatory native claim evidence even for genuinely zero-target runs.

run(data_file=None, show_status=True): Run the transformation.

verify_native_inputs(*, byte_verified=False, output_dir=None): Keep native roles and lossless source-claim dispositions fixed through publication.

classmethod verify_recorded_native_inputs(report, report_path, *, admission=None): Enforce native cohort conservation during read-only merge admission.