kg_microbe.transform_utils package
Subpackages
- kg_microbe.transform_utils.bacdive package
- Submodules
- kg_microbe.transform_utils.bacdive.ancestry_authority module
- kg_microbe.transform_utils.bacdive.bacdive module
BACDIVE_PREDICATE_TO_RELATIONBacDiveTransformBacDiveTransform.DATA_INPUTSBacDiveTransform.OPTIONAL_RAW_CONSUMED_INPUTSBacDiveTransform.REQUIRED_AUDIT_FILESBacDiveTransform.REQUIRED_CONSUMED_INPUTSBacDiveTransform.TRANSFORM_INPUTSBacDiveTransform.begin_consumed_inputs()BacDiveTransform.bind_ontology_dependency_declarations()BacDiveTransform.finalize()BacDiveTransform.ontology_dependency_declarationsBacDiveTransform.producer_native_inputsBacDiveTransform.run()BacDiveTransform.verify_native_inputs()BacDiveTransform.verify_recorded_native_inputs()
relation_for_predicate()
- kg_microbe.transform_utils.bacdive.clinical_categories module
- kg_microbe.transform_utils.bacdive.csv_field_limit module
- kg_microbe.transform_utils.bacdive.ec_substrate_corrections module
- kg_microbe.transform_utils.bacdive.emission module
DEPOSIT_CONFLICT_HEADERRESOLUTION_COLLAPSEDRESOLUTION_SUPPRESSEDRESOLUTION_SUPPRESSED_NO_ANCESTRYRESOLUTION_SUPPRESSED_WITHHELDStrainProvenanceWritercontested_deposit_description()deposit_conflict_rows()entailed_by_every_claim()is_contested()resolution_asserts_a_parent()resolve_deposit_parents()
- kg_microbe.transform_utils.bacdive.fallback_evidence module
- kg_microbe.transform_utils.bacdive.fallback_native_authority module
- kg_microbe.transform_utils.bacdive.fallback_selection module
- kg_microbe.transform_utils.bacdive.isolation_categories module
- kg_microbe.transform_utils.bacdive.ncit_category_claims module
- kg_microbe.transform_utils.bacdive.references module
- kg_microbe.transform_utils.bacdive.source_declaration_evidence module
- kg_microbe.transform_utils.bacdive.supplied_taxid_policy module
- kg_microbe.transform_utils.bacdive.taxonomy_evidence module
LpsnIndexTaxonomyEvidenceTaxonomyEvidence.bind_ontology_dependencies()TaxonomyEvidence.capture_native()TaxonomyEvidence.capture_records()TaxonomyEvidence.classify_deposit()TaxonomyEvidence.complete()TaxonomyEvidence.contractTaxonomyEvidence.emit_fallback_hierarchy()TaxonomyEvidence.fallback_choice()TaxonomyEvidence.lookup()TaxonomyEvidence.mark_output_ready()TaxonomyEvidence.native_deposit_claims()TaxonomyEvidence.observe_parent_choice()TaxonomyEvidence.observe_selected_parent()TaxonomyEvidence.outputs_readyTaxonomyEvidence.prepare_fallback()TaxonomyEvidence.prepare_source_declarations()TaxonomyEvidence.require_fallback()TaxonomyEvidence.resolve_deposits()TaxonomyEvidence.supplied_parent_withheld()TaxonomyEvidence.verify()TaxonomyEvidence.verify_native()TaxonomyEvidence.verify_output()TaxonomyEvidence.withheld_deposit_contests()
prepare()prepare_fallback()prepare_source_declarations()split_native_contract()verify_recorded()
- Module contents
BacDiveTransformBacDiveTransform.DATA_INPUTSBacDiveTransform.OPTIONAL_RAW_CONSUMED_INPUTSBacDiveTransform.REQUIRED_AUDIT_FILESBacDiveTransform.REQUIRED_CONSUMED_INPUTSBacDiveTransform.TRANSFORM_INPUTSBacDiveTransform.assay_edges_generatedBacDiveTransform.assay_nodes_generatedBacDiveTransform.assay_raw_dataBacDiveTransform.assay_target_nodes_generatedBacDiveTransform.begin_consumed_inputs()BacDiveTransform.bind_ontology_dependency_declarations()BacDiveTransform.chebi_categoriesBacDiveTransform.finalize()BacDiveTransform.isolation_source_mappingsBacDiveTransform.ncbitaxon_fallback_cacheBacDiveTransform.ncbitaxon_labelsBacDiveTransform.ncbitaxon_name_to_idBacDiveTransform.ncbitaxon_synonymsBacDiveTransform.ontology_dependency_declarationsBacDiveTransform.pathogenicity_mappingsBacDiveTransform.phenotype_routingBacDiveTransform.producer_native_inputsBacDiveTransform.run()BacDiveTransform.verify_native_inputs()BacDiveTransform.verify_recorded_native_inputs()
- kg_microbe.transform_utils.bactotraits package
- kg_microbe.transform_utils.bakta package
- Submodules
- kg_microbe.transform_utils.bakta.bakta module
BaktaTransformBaktaTransform.add_cog_annotation()BaktaTransform.add_ec_annotation()BaktaTransform.add_edge()BaktaTransform.add_functional_annotations()BaktaTransform.add_gene_node()BaktaTransform.add_go_annotation()BaktaTransform.add_kegg_annotation()BaktaTransform.add_organism_node()BaktaTransform.add_protein_node()BaktaTransform.get_ncbitaxon_id()BaktaTransform.get_organism_id()BaktaTransform.missing_optional_input()BaktaTransform.process_gene()BaktaTransform.process_genome()BaktaTransform.run()BaktaTransform.write_output()
- kg_microbe.transform_utils.bakta.create_samn_mapping module
- kg_microbe.transform_utils.bakta.utils module
- Module contents
BaktaTransformBaktaTransform.add_cog_annotation()BaktaTransform.add_ec_annotation()BaktaTransform.add_edge()BaktaTransform.add_functional_annotations()BaktaTransform.add_gene_node()BaktaTransform.add_go_annotation()BaktaTransform.add_kegg_annotation()BaktaTransform.add_organism_node()BaktaTransform.add_protein_node()BaktaTransform.get_ncbitaxon_id()BaktaTransform.get_organism_id()BaktaTransform.missing_optional_input()BaktaTransform.process_gene()BaktaTransform.process_genome()BaktaTransform.run()BaktaTransform.write_output()
- kg_microbe.transform_utils.cog package
- kg_microbe.transform_utils.ctd package
- kg_microbe.transform_utils.disbiome package
- kg_microbe.transform_utils.example_transform package
- kg_microbe.transform_utils.gold package
- Submodules
- kg_microbe.transform_utils.gold.alias_observations module
- kg_microbe.transform_utils.gold.crosswalk_quarantine module
- kg_microbe.transform_utils.gold.fold_restoration module
- kg_microbe.transform_utils.gold.gold module
- kg_microbe.transform_utils.gold.selected_taxon_aliases module
- kg_microbe.transform_utils.gold.taxon_dispositions module
- Module contents
- kg_microbe.transform_utils.gtdb package
- kg_microbe.transform_utils.kegg package
- kg_microbe.transform_utils.lpsn package
- kg_microbe.transform_utils.lpsn_api package
- kg_microbe.transform_utils.madin_etal package
- kg_microbe.transform_utils.mediadive package
- Submodules
- kg_microbe.transform_utils.mediadive.bulk_inputs module
- kg_microbe.transform_utils.mediadive.material_scope_audit module
- kg_microbe.transform_utils.mediadive.mediadive module
MediaDiveTransformMediaDiveTransform.DATA_INPUTSMediaDiveTransform.DEFAULT_INPUT_DIRMediaDiveTransform.OPTIONAL_RAW_CONSUMED_INPUTSMediaDiveTransform.REQUIRED_AUDIT_FILESMediaDiveTransform.REQUIRED_CONSUMED_INPUTSMediaDiveTransform.TRANSFORM_INPUTSMediaDiveTransform.consume_bulk_input()MediaDiveTransform.download_yaml_and_get_json()MediaDiveTransform.finalize()MediaDiveTransform.get_compounds_of_solution()MediaDiveTransform.get_json_object()MediaDiveTransform.get_solution_recipe_occurrences()MediaDiveTransform.missing_prerequisites()MediaDiveTransform.producer_native_inputsMediaDiveTransform.run()MediaDiveTransform.standardize_compound_id()MediaDiveTransform.verify_native_inputs()MediaDiveTransform.verify_recorded_native_inputs()
- kg_microbe.transform_utils.mediadive.recipe_identity module
- Module contents
MediaDiveTransformMediaDiveTransform.DATA_INPUTSMediaDiveTransform.DEFAULT_INPUT_DIRMediaDiveTransform.OPTIONAL_RAW_CONSUMED_INPUTSMediaDiveTransform.REQUIRED_AUDIT_FILESMediaDiveTransform.REQUIRED_CONSUMED_INPUTSMediaDiveTransform.TRANSFORM_INPUTSMediaDiveTransform.chebi_categoriesMediaDiveTransform.chebi_labelsMediaDiveTransform.chebi_role_edgesMediaDiveTransform.chebi_rolesMediaDiveTransform.consume_bulk_input()MediaDiveTransform.download_yaml_and_get_json()MediaDiveTransform.finalize()MediaDiveTransform.get_compounds_of_solution()MediaDiveTransform.get_json_object()MediaDiveTransform.get_solution_recipe_occurrences()MediaDiveTransform.missing_prerequisites()MediaDiveTransform.producer_native_inputsMediaDiveTransform.run()MediaDiveTransform.standardize_compound_id()MediaDiveTransform.verify_native_inputs()MediaDiveTransform.verify_recorded_native_inputs()
- kg_microbe.transform_utils.metatraits package
- Submodules
- kg_microbe.transform_utils.metatraits.io module
- kg_microbe.transform_utils.metatraits.metatraits module
- Module contents
MetaTraitsTransformMetaTraitsTransform.DATA_INPUTSMetaTraitsTransform.MEASUREMENT_TRAITSMetaTraitsTransform.OPTIONAL_CONSUMED_INPUTSMetaTraitsTransform.TRANSFORM_INPUTSMetaTraitsTransform.discovered_data_inputs()MetaTraitsTransform.indeterminate_traits_fileMetaTraitsTransform.ncbitaxon_name_to_idMetaTraitsTransform.run()MetaTraitsTransform.trait_mapping
- kg_microbe.transform_utils.metatraits_gtdb package
- Submodules
- kg_microbe.transform_utils.metatraits_gtdb.metatraits_gtdb module
- Module contents
MetaTraitsGTDBTransformMetaTraitsGTDBTransform.CODE_INPUTSMetaTraitsGTDBTransform.DATA_INPUTSMetaTraitsGTDBTransform.REQUIRED_AUDIT_FILESMetaTraitsGTDBTransform.REQUIRED_CONSUMED_INPUTSMetaTraitsGTDBTransform.TRANSFORM_INPUTSMetaTraitsGTDBTransform.accession_to_gtdb_speciesMetaTraitsGTDBTransform.accession_to_ncbiMetaTraitsGTDBTransform.gtdb_preferred_ncbiMetaTraitsGTDBTransform.gtdb_to_ncbiMetaTraitsGTDBTransform.ncbitaxon_name_to_idMetaTraitsGTDBTransform.run()MetaTraitsGTDBTransform.synthetic_nodes_metadataMetaTraitsGTDBTransform.trait_mapping
- kg_microbe.transform_utils.microbedecoder package
- Submodules
- kg_microbe.transform_utils.microbedecoder.bacdive_join_dispositions module
- kg_microbe.transform_utils.microbedecoder.chemical_curation module
- kg_microbe.transform_utils.microbedecoder.crosswalk_quarantine module
CrosswalkDecisionCrosswalkDecision.agent_typeCrosswalkDecision.dispositionCrosswalkDecision.keyCrosswalkDecision.knowledge_levelCrosswalkDecision.objectCrosswalkDecision.predicateCrosswalkDecision.primary_knowledge_sourceCrosswalkDecision.rationaleCrosswalkDecision.raw_lpsn_idCrosswalkDecision.raw_record_sha256CrosswalkDecision.relationCrosswalkDecision.rule_idCrosswalkDecision.source_cell_base64CrosswalkDecision.source_columnCrosswalkDecision.source_recordCrosswalkDecision.source_tokenCrosswalkDecision.subjectCrosswalkDecision.witness_id
CrosswalkQuarantineQuarantinePolicyQuarantinePolicy.decisions_fileQuarantinePolicy.decisions_rowsQuarantinePolicy.decisions_sha256QuarantinePolicy.evidence_fileQuarantinePolicy.evidence_sha256QuarantinePolicy.evidence_uncompressed_bytesQuarantinePolicy.evidence_uncompressed_sha256QuarantinePolicy.load()QuarantinePolicy.source_recordsQuarantinePolicy.source_sha256QuarantinePolicy.version
canonical_record_json()crosswalk_raw_record_sha256()
- kg_microbe.transform_utils.microbedecoder.crosswalk_witnesses module
- kg_microbe.transform_utils.microbedecoder.curation module
ProcessCurationProcessMappingProcessMapping.agent_typeProcessMapping.curation_rationaleProcessMapping.evidence_typeProcessMapping.evidence_uriProcessMapping.knowledge_levelProcessMapping.predicateProcessMapping.relationProcessMapping.source_columnProcessMapping.source_keyProcessMapping.source_literalProcessMapping.target_categoryProcessMapping.target_curieProcessMapping.target_label
- kg_microbe.transform_utils.microbedecoder.faprotax_dispositions module
- kg_microbe.transform_utils.microbedecoder.gold_crosswalk_dispositions module
- kg_microbe.transform_utils.microbedecoder.material_dispositions module
MaterialDispositionMaterialDisposition.agent_typeMaterialDisposition.curation_rationaleMaterialDisposition.dispositionMaterialDisposition.evidence_uriMaterialDisposition.identity_approvedMaterialDisposition.identity_statusMaterialDisposition.keyMaterialDisposition.knowledge_levelMaterialDisposition.object_curieMaterialDisposition.predicateMaterialDisposition.primary_knowledge_sourceMaterialDisposition.raw_record_sha256MaterialDisposition.relationMaterialDisposition.source_citationMaterialDisposition.source_columnMaterialDisposition.source_literalMaterialDisposition.source_recordMaterialDisposition.subjectMaterialDisposition.value_encoding
MaterialDispositionCuration
- kg_microbe.transform_utils.microbedecoder.microbedecoder module
MicrobeDecoderTransformMicrobeDecoderTransform.DATA_INPUTSMicrobeDecoderTransform.REQUIRED_AUDIT_FILESMicrobeDecoderTransform.REQUIRED_CONSUMED_INPUTSMicrobeDecoderTransform.TRANSFORM_INPUTSMicrobeDecoderTransform.TSV_QUOTINGMicrobeDecoderTransform.run()MicrobeDecoderTransform.verify_native_inputs()MicrobeDecoderTransform.verify_recorded_native_inputs()
- kg_microbe.transform_utils.microbedecoder.observation_audit module
- kg_microbe.transform_utils.microbedecoder.phenotype_curation module
- kg_microbe.transform_utils.microbedecoder.process_scopes module
- kg_microbe.transform_utils.microbedecoder.source_annotations module
- kg_microbe.transform_utils.microbedecoder.utils module
- Module contents
- kg_microbe.transform_utils.mim_ingredients package
- kg_microbe.transform_utils.ontologies package
- Submodules
- kg_microbe.transform_utils.ontologies.ontologies_transform module
METAMODEL_NODE_PREFIXESOntologiesTransformOntologiesTransform.DATA_INPUTSOntologiesTransform.ONTOLOGY_KNOWLEDGE_SOURCESOntologiesTransform.REQUIRED_AUDIT_FILESOntologiesTransform.REQUIRED_CONSUMED_INPUTSOntologiesTransform.bind_finalization_members()OntologiesTransform.decompress()OntologiesTransform.finalize()OntologiesTransform.missing_prerequisites()OntologiesTransform.parse()OntologiesTransform.post_process()OntologiesTransform.producer_native_inputsOntologiesTransform.run()OntologiesTransform.verify_native_inputs()OntologiesTransform.verify_recorded_native_inputs()
- Module contents
OntologiesTransformOntologiesTransform.DATA_INPUTSOntologiesTransform.ONTOLOGY_KNOWLEDGE_SOURCESOntologiesTransform.REQUIRED_AUDIT_FILESOntologiesTransform.REQUIRED_CONSUMED_INPUTSOntologiesTransform.bind_finalization_members()OntologiesTransform.decompress()OntologiesTransform.finalize()OntologiesTransform.missing_prerequisites()OntologiesTransform.parse()OntologiesTransform.post_process()OntologiesTransform.producer_native_inputsOntologiesTransform.run()OntologiesTransform.verify_native_inputs()OntologiesTransform.verify_recorded_native_inputs()
- kg_microbe.transform_utils.ontologies_stubs package
- Submodules
- kg_microbe.transform_utils.ontologies_stubs.ncit_category_projection module
- kg_microbe.transform_utils.ontologies_stubs.ontologies_stubs_transform module
OntologiesStubsTransformOntologiesStubsTransform.DATA_INPUTSOntologiesStubsTransform.TRANSFORM_INPUTSOntologiesStubsTransform.begin_consumed_inputs()OntologiesStubsTransform.missing_prerequisites()OntologiesStubsTransform.run()OntologiesStubsTransform.verify_native_inputs()OntologiesStubsTransform.verify_recorded_native_inputs()
- Module contents
OntologiesStubsTransformOntologiesStubsTransform.DATA_INPUTSOntologiesStubsTransform.TRANSFORM_INPUTSOntologiesStubsTransform.begin_consumed_inputs()OntologiesStubsTransform.missing_prerequisites()OntologiesStubsTransform.run()OntologiesStubsTransform.verify_native_inputs()OntologiesStubsTransform.verify_recorded_native_inputs()
- kg_microbe.transform_utils.prego package
- Submodules
- kg_microbe.transform_utils.prego.bto_metadata module
- kg_microbe.transform_utils.prego.calibration module
- kg_microbe.transform_utils.prego.envo_identity module
- kg_microbe.transform_utils.prego.native_inputs module
- kg_microbe.transform_utils.prego.prego module
- kg_microbe.transform_utils.prego.quality module
- kg_microbe.transform_utils.prego.utils module
- Module contents
- kg_microbe.transform_utils.rhea_mappings package
- Submodules
- kg_microbe.transform_utils.rhea_mappings.label_inputs module
- kg_microbe.transform_utils.rhea_mappings.legacy_xrefs module
- kg_microbe.transform_utils.rhea_mappings.native_graph module
- kg_microbe.transform_utils.rhea_mappings.native_inputs module
- kg_microbe.transform_utils.rhea_mappings.native_rdf module
- kg_microbe.transform_utils.rhea_mappings.reaction_families module
- kg_microbe.transform_utils.rhea_mappings.rhea_mappings module
RheaMappingsTransformRheaMappingsTransform.DATA_INPUTSRheaMappingsTransform.REQUIRED_AUDIT_FILESRheaMappingsTransform.REQUIRED_CONSUMED_INPUTSRheaMappingsTransform.TRANSFORM_INPUTSRheaMappingsTransform.TSV_QUOTINGRheaMappingsTransform.producer_native_inputsRheaMappingsTransform.run()RheaMappingsTransform.verify_native_inputs()RheaMappingsTransform.verify_recorded_native_inputs()
- kg_microbe.transform_utils.rhea_mappings.stage_snapshot module
- Module contents
RheaMappingsTransformRheaMappingsTransform.DATA_INPUTSRheaMappingsTransform.REQUIRED_AUDIT_FILESRheaMappingsTransform.REQUIRED_CONSUMED_INPUTSRheaMappingsTransform.TRANSFORM_INPUTSRheaMappingsTransform.TSV_QUOTINGRheaMappingsTransform.producer_native_inputsRheaMappingsTransform.run()RheaMappingsTransform.verify_native_inputs()RheaMappingsTransform.verify_recorded_native_inputs()
- kg_microbe.transform_utils.uniprot_functional_microbes package
- kg_microbe.transform_utils.uniprot_human package
- kg_microbe.transform_utils.uniprot_trembl package
- kg_microbe.transform_utils.wallen_etal package
Submodules
kg_microbe.transform_utils.constants module
Constants for transform_utilities.
- kg_microbe.transform_utils.constants.GTDB_NCBI_POOLING_REPORT = 'gtdb_ncbi_pooling_report.tsv'
Report of NCBI taxa that several GTDB taxa map onto (#883). Written on every gtdb run, empty or not.
kg_microbe.transform_utils.transform module
Transform utility module.
- class kg_microbe.transform_utils.transform.Transform(source_name, input_dir=None, output_dir=None, nlp=False)
Bases:
objectParent class for transforms, that sets up a lot of default file info.
-
CODE_INPUTS:
tuple= () Additional repo-relative Python packages/files whose behavior this producer inherits. This is code provenance, never a dependency on another producer’s graph outputs.
- DATA_DIR = PosixPath('/home/runner/work/kg-microbe/kg-microbe/kg_microbe/transform_utils/data')
-
DATA_INPUTS:
tuple= () Repo-relative curation files this transform reads, beyond its own
data/raw/download.Declared so freshness tooling can tell that an output is stale against its data rather than only its code. Without it a mapping correction lands, every consumer keeps reporting FRESH, and a re-merge silently ships the old groundings: #778 corrected 16 isolation-source ids and #786 rewrote the unified chemical SSSOM, and the merged KG built afterwards still asserted 75 organisms isolated from a “Cell Line”, because nothing re-ran the transforms that read those files (#812).
Paths are relative to the repo root. Keep them tracked in git — the freshness check uses commit time, not mtime, because git checkout rewrites mtimes without changing content (#797).
List every curation file read, not a representative one. A partial declaration fails silently and looks identical to a complete one: ontologies_stubs declared 1 of the 11 files it read and was reported fresh after changes to the other ten (#839). Where the set comes from a constant, derive this from it rather than restating it.
- DEFAULT_INPUT_DIR = PosixPath('/home/runner/work/kg-microbe/kg-microbe/kg_microbe/transform_utils/data/raw')
- DEFAULT_OUTPUT_DIR = PosixPath('/home/runner/work/kg-microbe/kg-microbe/kg_microbe/transform_utils/data/transformed')
-
OPTIONAL_CONSUMED_INPUTS:
tuple= () Named optional repository-relative reads; absence is evidence, never a required file. Unlike DATA_INPUTS these locators do not relocate to input_base_dir.
-
OPTIONAL_RAW_CONSUMED_INPUTS:
tuple= () Named optional reads relative to the effective input_base_dir, including explicit absence. Kept separate so repository-relative optional inputs never silently relocate.
-
REQUIRED_AUDIT_FILES:
tuple= () Producer-owned disposition reports whose emitted bytes must survive finalization/merge.
-
REQUIRED_CONSUMED_INPUTS:
tuple= () Named generated inputs that must actually be read before fresh finalization. Unlike DATA_INPUTS, these are checked after upstream producers have run.
-
TRANSFORM_INPUTS:
tuple= () Registered source names whose output this transform reads.
DATA_INPUTS covers curation files under
mappings/. It does not cover a dependency on another transform’s output, which several sources have. For example, gold readsontologies/ncbitaxon_nodes.tsv(and refuses to run without it) plusontologies_stubs/po_nodes.tsv, lpsn readsgtdb/nodes.tsv, lpsn_api and microbedecoder readlpsn/nodes.tsv, prego readsontologies/, and bacdive, mediadive and metatraits reachontologies/through theNCBITAXON_NODES_FILE/CHEBI_NODES_FILEconstants (metatraits_gtdb inherits metatraits’). Those last three went undeclared for months precisely because the path is spelled inconstants.pyrather than in the transform, which the guard below could not see until #1035.Source finalization reads the ontologies output’s declarations too, but only for a source that lists
ontologieshere. gtdb, lpsn, madin_etal, rhea_mappings and ontologies_stubs declare it for that reason alone (#1511).kg transformrefuses to start a source whose declared upstream has no completion marker, unless the same run builds it first.Undeclared, re-running an upstream leaves every downstream genuinely stale while all three freshness signals report fresh — the #812 shape, across transforms rather than within one (#845). The ordering was already encoded in
DATA_SOURCEScomments (“Run gold after ontologies…”); this makes it machine-readable so the fingerprint can fold the upstream in.tests/test_cross_transform_inputs.py derives the real dependency map from the source and fails on anything undeclared, because every previous version of this contract was opt-in and was forgotten (#812, #839, #876).
- begin_consumed_inputs()
Start a real producer run with no inherited input-consumption claims.
- begin_dependency_admission()
Bind discovered curation and inherited code before a producer loads either.
- consume_input(name, path)
Read a generated UTF-8 input from an immutable, exact-byte tracked snapshot.
- consume_optional_input(name)
Read one declared optional input immutably, retaining its locator and explicit absence.
- property consumed_input_snapshots
Return copied named path/digest snapshots so callers cannot mutate recorded evidence.
- finalize(*, file_prefix='', fresh_run=False)
Validate and finalize produced TSVs before publication as a current source.
The CLI invokes this after
runand before writing the source fingerprint. Direct Python callers must invoke it explicitly afterrun; producer writes themselves are not a bundle transaction.
- classmethod missing_optional_input(input_dir=None)
Return why an optional source has nothing to transform, or None to run it.
An optional source (Bakta) has no download and is excluded from the standard merge, so a machine without its data is a normal machine. The default batch skips such a source instead of letting
runreturn early into a finalization error that fails the whole batch; an explicit-srequest still fails, because requested output was not created. Required sources keep this default and always run.- Parameters:
input_dir (
Optional[Path]) – Effective raw input directory;Nonemeans the default.- Return type:
Optional[str]
- classmethod missing_prerequisites(input_dir=None)
Return a message for each external prerequisite this transform needs but cannot find.
Covers what
DATA_INPUTScannot: executables, and raw downloads that are not git-tracked curation. Checked before any source runs, so a missing prerequisite fails in seconds instead of after hours of upstream work.- Parameters:
input_dir (
Optional[Path]) – Effective raw input directory;Nonemeans the default.- Return type:
List[str]
- property optional_consumed_inputs
Return independent serialized state without replacing original read evidence.
- pass_through(nodes_file, edges_file)
Copy nodes and edges files to output directory.
- Parameters:
nodes_file (
str) – nodes files to take from raw directory and put in transform directoryedges_file (
str) – edges files to take from raw directory and put in transform directory
- Return type:
None
- property producer_audit_snapshots
Return copied exact-byte identities recorded by the successful producer.
- record_producer_audit(name)
Bind a mandatory sidecar after its successful write, before source finalization.
- run(data_file=None)
Run the transform.
- Parameters:
data_file (
Union[Path,None,str]) – Input data file, defaults to None
- verify_consumed_inputs()
Reject missing required reads or changed/deleted bytes consumed by this run.
- verify_declared_dependencies()
Refuse drift without replacing the original producer-time dependency snapshot.
-
CODE_INPUTS:
Module contents
Transform utilities module.