kg_microbe.transform_utils.lpsn_api package
Submodules
kg_microbe.transform_utils.lpsn_api.lpsn_api module
LPSN JSON API transform.
Enriches the GSS-based lpsn transform with the fields that only live
in LPSN’s authenticated JSON API and not in the bulk CSV:
Publication provenance:
publication_doi,publication_pmid,ijsem_list_doi→doi:*/PMID:*publication subjects mentioning the LPSN taxon (biolink:mentions,IAO:0000142).Full above-genus taxonomy:
lpsn_parent_id→ additionalbiolink:subclass_ofedges walking up to family / order / class / phylum / domain, which the GSS format doesn’t expose.Nomenclatural genealogy:
basonym_id→biolink:same_asedge (relationskos:exactMatch) from a comb. nov. name to its basonym.Node-level detail:
is_legitimate(boolean) and richernomenclatural_status(free-text) rolled into the description.16S rRNA sequence provenance:
molecules→biolink:in_taxon(RO:0002162) from each INSDC accession to its LPSN taxon, plus abiolink:NucleicAcidEntitynode per accession.Hierarchy and basionym triples already present in the selected sibling GSS
edges.tsvare omitted. Missing and API-only links remain. Both selected GSS files are required native inputs for completion.
Access model
The API is auth-gated (free registration at
https://lpsn.dsmz.de/register). We look for LPSN_USERNAME + LPSN_PASSWORD
in the environment (typically loaded from .env via python-dotenv,
same pattern the BacDive transform uses). If either is missing, the
transform raises a clear RuntimeError with the exact instructions
rather than falling back to silent no-op.
Rate limits
LPSN doesn’t publish an explicit rate limit. Empirically the lpsn
Python client tolerates ~1–2 requests/second, which projects to 6–10
hours for a full 34K-record pull. Every response is cached to
data/raw/lpsn/api_cache/<record_no>.json (gitignored) so re-runs
skip already-fetched records and a partial run resumes cleanly.
Design notes
This is a SEPARATE transform (registered as lpsn_api in
DATA_SOURCES) instead of an inline step in the main lpsn
transform, so the fast 16-second GSS-only path stays the default. Users
who want the API enrichment run both:
poetry run kg transform -s lpsn # fast GSS-only (16 s), after gtdb and ontologies poetry run kg transform -s lpsn_api # slow enrichment (hours)
The merge step then combines both nodes.tsv / edges.tsv files.
- class kg_microbe.transform_utils.lpsn_api.lpsn_api.LPSNAPITransform(input_dir=None, output_dir=None, client=None, page_fetch=None)
Bases:
TransformFetch per-record LPSN JSON, emit enrichment nodes + edges.
-
REQUIRED_AUDIT_FILES:
tuple= ('lpsn_api_records.tsv',) Producer-owned disposition reports whose emitted bytes must survive finalization/merge.
-
REQUIRED_CONSUMED_INPUTS:
tuple= ('lpsn_api_gss_nodes', 'lpsn_api_gss_edges') Named generated inputs that must actually be read before fresh finalization. Unlike DATA_INPUTS, these are checked after upstream producers have run.
-
TRANSFORM_INPUTS:
tuple= ('lpsn',) Reads this transform’s output; see Transform.TRANSFORM_INPUTS (#845).
- property producer_native_inputs
Return the selected API record inventory without exposing mutable producer state.
- run(data_file=None, show_status=True)
Emit enrichment nodes + edges from LPSN JSON API responses.
- Return type:
None
Parameters
- data_file:
Optional override for the GSS-transform
nodes.tsvwe read to know which record_no’s to enrich.- show_status:
Accepted for compatibility with the
kg transformCLI. Progress is printed in periodic batches regardless.
- verify_native_inputs(*, byte_verified=False, output_dir=None)
Require selected API record bytes, locators and the mandatory producer audit.
- classmethod verify_recorded_native_inputs(report, report_path, *, admission=None)
Bind saved API reads to their selected cache paths during read-only merge admission.
-
REQUIRED_AUDIT_FILES:
- kg_microbe.transform_utils.lpsn_api.lpsn_api.WEB_RECORD_SOURCE_KEY = 'kgmicrobe_source'
Marks a record dict assembled from the web page rather than the API.
- kg_microbe.transform_utils.lpsn_api.lpsn_api.fetch_taxon_page(record_no)
Return the HTML of LPSN’s web page for
record_no, orNone.Only reached for records the JSON API has no result for; the page is public and needs no credentials. Any transport or HTTP failure is reported and yields
Noneso the caller falls back to a bare stub rather than aborting the run.- Return type:
Optional[str]
- kg_microbe.transform_utils.lpsn_api.lpsn_api.parse_taxon_page(record_no, page)
Build an API-shaped record from an LPSN taxon page, or
None.The page title is
<rank>: <name>(Species: Bacterium sonnei); that is the whole reason the page is worth fetching, so no title means no record. Nomenclatural and taxonomic status are taken when present. The dict carriesWEB_RECORD_SOURCE_KEYso consumers can tell it from an API record; it never carries ids to link to.- Return type:
Optional[dict]
kg_microbe.transform_utils.lpsn_api.native_inputs module
Bind LPSN API citation labels to consumed record bytes and selected locators (#1838).
- kg_microbe.transform_utils.lpsn_api.native_inputs.audit_rows(contract)
Serialize deterministic evidence for every selected API record, including absent cache entries.
- kg_microbe.transform_utils.lpsn_api.native_inputs.iter_gss_record_nos(handle)
Read the exact root-selection rule shared by production and independent admission.
- kg_microbe.transform_utils.lpsn_api.native_inputs.linked_record_nos(record_no, record)
Preserve production’s ordered parent/basonym links, excluding nonnumeric and self links.
- kg_microbe.transform_utils.lpsn_api.native_inputs.read_gss_edge_triples(handle)
Read exact selected GSS triples; malformed TSV cannot silently suppress API evidence.
- kg_microbe.transform_utils.lpsn_api.native_inputs.validate_record(record, record_no)
Reject missing, malformed or unrelated API responses before attributing their fields.
- kg_microbe.transform_utils.lpsn_api.native_inputs.validate_record_no(record_no)
Require a literal numeric record identifier before constructing a cache path.
- kg_microbe.transform_utils.lpsn_api.native_inputs.verify_native_contract(contract, snapshots, raw_dir, *, audit_path=None, admission=None)
Revalidate selected lexical paths as well as bytes before completion and merge admission.
- kg_microbe.transform_utils.lpsn_api.native_inputs.verify_record_audit(contract, audit_path, *, admission=None)
Check required audit bytes, retaining them only for a read-only admission epoch.
- kg_microbe.transform_utils.lpsn_api.native_inputs.verify_selection_inventory(contract, guard, raw)
Derive all requested records from verified GSS roots and API links, never ledger membership.
Module contents
LPSN JSON API transform (enriches the GSS transform’s output).