Evaluation for hra-kg
Evaluator: Cortes et al.
Evaluated on: 2025-08-26
This is a manual evaluation intended to identify potential barriers to reuse.
Evaluation Criteria: This evaluation uses the KG-Registry evaluation rubric as described in Cortes et al. (2025) . The rubric assesses knowledge graphs across multiple dimensions including access, provenance, documentation, maintenance, and fitness for purpose.
Access Level and Types
| Question | Answer | Comment |
|---|---|---|
| Access to data outside of the knowledge graph | Y | Users can ask biological questions via queries (Basic Usage: https://cns-iu.github.io/hra-kg-supporting-information/#basic-usage, Advanced Usage: https://github.com/cns-iu/hra-kg-supporting-information/blob/main/README.md) |
| API or online access to the knowledge graph | Y | Can access by SPARQL endpoint, HRA API, and RESTful-API. |
| Multiple access options available | Y | Dynamic queries through its SPARQL, RESTful-API, and UIs. |
| Source code availability | Y | Github (https://github.com/hubmapconsortium/hra-kg) |
| Downloadable knowledge graph | Y | Available on Zenodo and HRA CDN |
Section Score: 5/5
Provenance of Nodes and Edges
| Question | Answer | Comment |
|---|---|---|
| Source list provided | Y | Table 2 |
| Source versions information | Y | The type, name, and version of each DO is documented (https://lod.humanatlas.io/asct-b/). |
| Import dependencies | Y | In the hra-do-processor. |
| Node and edge sources | Y | Both nodes and edges have source information |
| Edges deduplication | N | Not explicit. As each digital object produces its own graph, the triples from different objects may have similar relationships, but they are stored in separate contexts. |
| Triples source details | Y | The triples are generated by Digital Objects in in RDF format. |
| Edge type schema | Y | The KG is built based on RDF graph (CCF ontology) and standard ontologies to define relationship types between the entities. |
Section Score: 6/7
Documented standards, schema, construction
| Question | Answer | Comment |
|---|---|---|
| Biological usable data | Y | The data is human readable and usable in various biological applications. |
| Resolvable IDs | Y | It uses resolvable IDs for the entity resolution such as UBERON:0013702 and ENSG00000175899. |
| Construction documentation | Y | It provides comprehensive documentation on its construction process (HRA KG Construction and Deployment). |
| Transformation documentation | Y | It documents data transfroms such as data normalization and enrichment. |
| Schema used | Y | ASCT+B tables and Common Coordinate Framework (CCF) Ontology. |
Section Score: 5/5
Update frequency and versioning
| Question | Answer | Comment |
|---|---|---|
| Stable versions | Y | E.g., v2.3. |
| Public tracker information | N | Not explicitly mentioned, but there are multiple Github Issues portals for users to post requests, reports bugs, and discussions. |
| Knowledge graph contact information | Y | Contact information is available to reach out to the team (https://humanatlas.io/about#editorial-board) |
| Updated annually | Y | About 2 time a year (https://github.com/hubmapconsortium/hra-kg/releases). |
| Prior versions access | Y | Prior versions are accessible on Github (https://github.com/hubmapconsortium/hra-kg/releases). Changed are documented (https://humanatlas.io/release-notes/v2.3). |
Section Score: 4/5
Evaluation - Metrics and Fitness for Purpose
| Question | Answer | Comment |
|---|---|---|
| Use case provided | Y | They have several use cases and applications (Usage Notes). |
| Evaluation against other models | Y | The HRA-KG is compared with other KGs only from technical perspective, such as the number of nodes, edges, and technologies used. |
| Defined scope | Y | HRA-KG is focused on representing the healthy human body in semantics terms including anatomical structures, cell types, and biomarker and spatial terms such as 3D reference organs. |
| Multiple evaluation methods | Y | It has hra-ols used by OLS for validation (https://lod.humanatlas.io/collection/hra-ols/latest/) and has Validation report (https://github.com/hubmapconsortium/3d-hra-ref-object-validation) (Supplementary). |
| Accuracy metrics | N | Not explicit, but it has HRAlit which links the HRA DOs to publications, experts, experimental datasets (Supplementary). |
Section Score: 4/5
License Information
| Question | Answer | Comment |
|---|---|---|
| License | CC BY 4.0 |