Evaluation for spoke-okn
Evaluator: Automated Evaluation (GPT-5.4)
Evaluated on: 2026-05-30
⚠️ Automated Evaluation: This evaluation was generated automatically using an AI-based system. It is distinct from manual evaluations curated by human experts. Please review findings carefully and report any inaccuracies.
Evaluation Criteria: This evaluation uses the KG-Registry evaluation rubric as described in Cortes et al. (2025) . The rubric assesses knowledge graphs across multiple dimensions including access, provenance, documentation, maintenance, and fitness for purpose.
Access Level and Types
| Question | Answer | Comment |
|---|---|---|
| Access to data outside of the knowledge graph | Y | SPOKE website and publications describe the graph and its intended uses. |
| API or online access to the knowledge graph | Y | SPARQL and TPF endpoints are listed for the OKN deployment. |
| Multiple access options available | Y | Both FRINK query interfaces are documented. |
| Source code availability | Y | Resource page links to the public SPOKE repository. |
| Downloadable knowledge graph | N | No downloadable SPOKE-OKN dump was identified in this pass. |
Section Score: 4/5
Provenance of Nodes and Edges
| Question | Answer | Comment |
|---|---|---|
| Source list provided | Y | Resource page now states that the OKN graph is derived from SPOKE, whose publications describe integrated sources. |
| Source versions information | Y | 2023 Bioinformatics paper reports weekly rebuilds of the underlying SPOKE graph. |
| Import dependencies | Y | OKN deployment is explicitly modeled as derived from SPOKE. |
| Node and edge sources | Y | SPOKE publications describe integration of dozens of biomedical databases and ontologies. |
| Edges deduplication | N | No explicit deduplication policy was confirmed for the OKN deployment. |
| Triples source details | Y | Reviewed publications provide source counts, node/edge types, and representative upstream resources. |
| Edge type schema | Y | Publications document SPOKE node and edge types even though a separate OKN schema page was not found. |
Section Score: 6/7
Documented standards, schema, construction
| Question | Answer | Comment |
|---|---|---|
| Biological usable data | Y | The graph is explicitly intended for biomedical and precision-medicine applications. |
| Resolvable IDs | Y | SPOKE integrates established biomedical resources and identifiers. |
| Construction documentation | Y | Both reviewed publications describe graph assembly and the integrated source landscape. |
| Transformation documentation | Y | 2023 paper describes weekly ETL-style graph builds from upstream resources. |
| Schema used | N | No separate formal schema artifact for the OKN deployment was identified. |
Section Score: 4/5
Update frequency and versioning
| Question | Answer | Comment |
|---|---|---|
| Stable versions | N | No versioned OKN releases were identified. |
| Public tracker information | N | No public issue tracker specific to SPOKE-OKN was identified. |
| Knowledge graph contact information | Y | Sergio Baranzini is listed as a contact on the resource page. |
| Updated annually | Y | Underlying SPOKE publications report weekly updates. |
| Prior versions access | N | No archived OKN snapshots were found. |
Section Score: 2/5
Evaluation - Metrics and Fitness for Purpose
| Question | Answer | Comment |
|---|---|---|
| Use case provided | Y | Publications describe applications in drug discovery, disease analysis, and clinical knowledge integration. |
| Evaluation against other models | N | No benchmark comparison for the OKN deployment was identified. |
| Defined scope | Y | Scope is a biomedical and environmental health knowledge graph derived from SPOKE. |
| Multiple evaluation methods | N | No explicit multi-method evaluation framework was found. |
| Accuracy metrics | N | No specific quantitative accuracy metric was identified. |
Section Score: 2/5
License Information
| Question | Answer | Comment |
|---|---|---|
| License | Multiple upstream licenses apply; SPOKE publishes a source-license page. |