Evaluation for geneticskp
Evaluator: Automated Evaluation
Evaluated on: 2026-05-30
⚠️ Automated Evaluation: This evaluation was generated automatically using an AI-based system. It is distinct from manual evaluations curated by human experts. Please review findings carefully and report any inaccuracies.
Evaluation Criteria: This evaluation uses the KG-Registry evaluation rubric as described in Cortes et al. (2025) . The rubric assesses knowledge graphs across multiple dimensions including access, provenance, documentation, maintenance, and fitness for purpose.
Access Level and Types
| Question | Answer | Comment |
|---|---|---|
| Access to data outside of the knowledge graph | Y | Translator wiki, public repository, and release artifact page document the provider outside the graph itself. |
| API or online access to the knowledge graph | Y | Public TRAPI endpoint is listed on the resource page. |
| Multiple access options available | Y | Resource page documents a TRAPI endpoint, release artifact download, repository, and Translator wiki documentation. |
| Source code availability | Y | Source repository is public at broadinstitute/genetics-kp-dev. |
| Downloadable knowledge graph | Y | KGX JSONL release artifact is listed as a graph product. |
Section Score: 5/5
Provenance of Nodes and Edges
| Question | Answer | Comment |
|---|---|---|
| Source list provided | Y | The page now lists Genebass, GenCC, ClinVar, and ClinGen as upstream sources for the primary graph product. |
| Source versions information | Y | The graph product records Translator release version, source version, and Biolink version. |
| Import dependencies | Y | Public docs describe MAGMA, Richards, ABC, and integrated-genetics methods layered over curated upstream sources. |
| Node and edge sources | Y | Repository and wiki document both curated data resources and method-generated association layers. |
| Edges deduplication | N | No explicit deduplication policy was identified in reviewed docs. |
| Triples source details | Y | Reviewed docs specify available method families and phenotype coverage for several association sets. |
| Edge type schema | Y | TRAPI/Biolink compatibility is recorded for the graph release. |
Section Score: 6/7
Documented standards, schema, construction
| Question | Answer | Comment |
|---|---|---|
| Biological usable data | Y | The resource is explicitly intended for genetic association and disease-biology analysis. |
| Resolvable IDs | Y | Phenotype lists in the public repository include EFO and MONDO identifiers, and the release is Translator/TRAPI-oriented. |
| Construction documentation | Y | Translator wiki and public repository document source inputs and major method families. |
| Transformation documentation | Y | Public docs describe MAGMA, Richards, ABC, and integrated genetics transformations over upstream datasets. |
| Schema used | Y | Biolink compatibility is recorded for the graph artifact and TRAPI endpoint. |
Section Score: 5/5
Update frequency and versioning
| Question | Answer | Comment |
|---|---|---|
| Stable versions | Y | The page records a stable Translator release and build identifier. |
| Public tracker information | Y | Public GitHub repository exposes issues and development history. |
| Knowledge graph contact information | Y | Marc Duby and other team members are recorded on the resource page. |
| Updated annually | Y | Current resource metadata and release history indicate active maintenance within the last year. |
| Prior versions access | Y | Versioned release and build identifiers are exposed on the graph product. |
Section Score: 5/5
Evaluation - Metrics and Fitness for Purpose
| Question | Answer | Comment |
|---|---|---|
| Use case provided | Y | Use case is Translator-facing gene-disease analysis from integrated genetic evidence. |
| Evaluation against other models | N | No comparative benchmark against other genetic KGs was identified during this pass. |
| Defined scope | Y | Scope is focused genetic association evidence, especially GWAS-derived signals and curated clinical genetics resources. |
| Multiple evaluation methods | N | Multiple association-generation methods are documented, but not framed as a published evaluation suite. |
| Accuracy metrics | N | No explicit quantitative accuracy metric was captured in the reviewed docs. |
Section Score: 2/5
License Information
| Question | Answer | Comment |
|---|---|---|
| License | MIT license is recorded on the graph release products. |