Evaluation for ctkp

Evaluator: Automated Evaluation

Evaluated on: 2026-05-30

⚠️ Automated Evaluation: This evaluation was generated automatically using an AI-based system. It is distinct from manual evaluations curated by human experts. Please review findings carefully and report any inaccuracies.

Evaluation Criteria: This evaluation uses the KG-Registry evaluation rubric as described in Cortes et al. (2025) . The rubric assesses knowledge graphs across multiple dimensions including access, provenance, documentation, maintenance, and fitness for purpose.

Access Level and Types

Question	Answer	Comment
Access to data outside of the knowledge graph	Y	Translator wiki, SmartAPI registration, and source repository provide public documentation outside the graph artifact itself.
API or online access to the knowledge graph	Y	SmartAPI documents a public TRAPI 1.5 endpoint for the Multiomics Clinical Trials KP.
Multiple access options available	Y	The page documents a release artifact, Translator wiki documentation, and SmartAPI registration.
Source code availability	Y	Public source repository exists at multiomicsKP/clinical_trials_kp.
Downloadable knowledge graph	Y	KGX JSONL release artifact is listed on the resource page.

Section Score: 5/5

Provenance of Nodes and Edges

Question	Answer	Comment
Source list provided	Y	The primary graph product explicitly records AACT and ClinicalTrials.gov provenance.
Source versions information	Y	The page records Translator release version, build identifier, source version, and Biolink version.
Import dependencies	Y	Reviewed docs explain that CTKP depends on AACT-mediated ClinicalTrials.gov content.
Node and edge sources	Y	Wiki documents trial identifiers, interventions, conditions, and adverse events as core graph content.
Edges deduplication	N	No explicit deduplication policy was identified.
Triples source details	Y	Wiki names representative predicates and node categories used by the provider.
Edge type schema	Y	Biolink predicates and Translator TRAPI exposure are documented.

Section Score: 6/7

Documented standards, schema, construction

Question	Answer	Comment
Biological usable data	Y	The resource exposes clinically relevant intervention-condition-trial associations.
Resolvable IDs	Y	NCT identifiers and standard Translator/Biolink categories are documented.
Construction documentation	Y	Translator wiki explains the source and the broad association types exposed by the graph.
Transformation documentation	Y	Docs make clear that AACT is the transformation layer between ClinicalTrials.gov and the KP graph.
Schema used	Y	Biolink predicates and TRAPI/Biolink compatibility are documented.

Section Score: 5/5

Update frequency and versioning

Question	Answer	Comment
Stable versions	Y	Stable Translator release and build identifiers are listed.
Public tracker information	Y	Public GitHub repository provides issue tracking and code history.
Knowledge graph contact information	Y	Resource page now includes a maintainer contact via the public repository identity.
Updated annually	Y	Source repository shows recent updates and the resource page has current release metadata.
Prior versions access	Y	Release/build version fields are preserved on the graph product.

Section Score: 5/5

Evaluation - Metrics and Fitness for Purpose

Question	Answer	Comment
Use case provided	Y	Provider is designed for Translator-facing clinical trial evidence and intervention-condition associations.
Evaluation against other models	N	No comparative benchmark against other trial KGs was identified.
Defined scope	Y	Scope is clearly bounded to ClinicalTrials.gov data mediated through AACT.
Multiple evaluation methods	N	No explicit evaluation framework was identified in public docs.
Accuracy metrics	N	No quantitative accuracy metrics were captured during this pass.

Section Score: 2/5

License Information

Question	Answer	Comment
License		MIT license is recorded on the graph release products.