Evaluation for genomickb
Evaluator: Cortes et al.
Evaluated on: 2025-08-26
This is a manual evaluation intended to identify potential barriers to reuse.
Evaluation Criteria: This evaluation uses the KG-Registry evaluation rubric as described in Cortes et al. (2025) . The rubric assesses knowledge graphs across multiple dimensions including access, provenance, documentation, maintenance, and fitness for purpose.
Access Level and Types
| Question | Answer | Comment |
|---|---|---|
| Access to data outside of the knowledge graph | Y | Can submit an online query and download subgraphs that meet criteria |
| API or online access to the knowledge graph | Y | Website with UI for creating queries, API supposely coming in v2, Neo4j dump available but not hosted by creators |
| Multiple access options available | Y | Can use online query tool or download Neo4j dump |
| Source code availability | Y | Https://github.com/tinalee-tech/KG-dataloader |
| Downloadable knowledge graph | Y | Neo4j dump |
Section Score: 5/5
Provenance of Nodes and Edges
| Question | Answer | Comment |
|---|---|---|
| Source list provided | Y | Supplementary note 1 |
| Source versions information | N | |
| Import dependencies | N | |
| Node and edge sources | Y | Tables 1 (nodes) & 2 (edges), query allows users to restrict by datasource |
| Edges deduplication | Y | Considered but not de-duplicated. Users can restric querys to specific datasources and duplicate edges can be used for validation. EX: ‘enhancer regulate gene’ with restriction ‘cell_line=GM12878’ and ‘data_source=EnhancerAtlas’ -> 118,610 node pairs. A second query for ‘variant overlap enhancer’, ‘enhancer regulate gene’, and ‘variant correlate_with gene’ -> 16,871 enhancer-gene pairs from EnhancerAtlas can be validated by GTEx eQTLs |
| Triples source details | Y | Table 2 |
| Edge type schema | Y | Table 2 |
Section Score: 5/7
Documented standards, schema, construction
| Question | Answer | Comment |
|---|---|---|
| Biological usable data | Y | They have a web interface (https://gkb.dcmb.med.umich.edu/) presenting the entities and edges in a human-readable format. It can be used to answer genomics-related questions and conduct biological analysis |
| Resolvable IDs | Y | Uses one external ID when possible and defines globaly unique IDs for entities without such as ChIP-seq peaks |
| Construction documentation | Y | Github and supplement |
| Transformation documentation | Y | Sup Note 1 Data Collection and Processing |
| Schema used | Y | Uses custom schema not an existing one but does provide documentation |
Section Score: 5/5
Update frequency and versioning
| Question | Answer | Comment |
|---|---|---|
| Stable versions | Y | But still on version 1.0 despite multiple upates listed |
| Public tracker information | N | FAQ says to email the team |
| Knowledge graph contact information | Y | Website has a contacts button with name/emal for both technical details and accessing the data/web server |
| Updated annually | Y | During 2022-2023 yes but not since then |
| Prior versions access | Y | Clearly documented chages with dates but prior versions not available. *Paper states that they store all versions internally and can revert if unexpected changes are ever made to their graph but these are not accessible publicaly |
Section Score: 4/5
Evaluation - Metrics and Fitness for Purpose
| Question | Answer | Comment |
|---|---|---|
| Use case provided | N | The paper provides examples of specific queries that can be performed through the website that would normally require coding and integrating multiple data sources, but it is limited to knowledge extraction and does not show how this could be used to make novel biological discoveries |
| Evaluation against other models | N | |
| Defined scope | Y | Coding free queries of human genomic information including both functional, strutural, epigenetic and other relationships |
| Multiple evaluation methods | N | |
| Accuracy metrics | Y | Kind of but not explicitly. Since this tool is all about data querying, confidence could be assesed by creating multiple queries like the example of using eQTLs to "validate" EnhancerAtlas relationships |
Section Score: 2/5
License Information
| Question | Answer | Comment |
|---|---|---|
| License | CC BY-NC 4.0 |