Evaluation for rtx-kg2
Evaluator: Not specified
Evaluated on: 2025-08-26
This is a manual evaluation intended to identify potential barriers to reuse.
Access Level and Types
| Question | Answer | Comment |
|---|---|---|
| Access to data outside of the knowledge graph | Y | RTX-KG2 is able to provide various biomedical information through queries. It is also used as backend to support ARAX's path reasoning and path ranking (https://github.com/RTXteam/RTX) . |
| API or online access to the knowledge graph | Y | Can access by API query, and Neo4j. |
| Multiple access options available | Y | Multiple ways to access including downloadable versions, API (SmartAPI), web browser user interface (seems not currently working). |
| Source code availability | Y | Github (https://github.com/RTXteam/RTX-KG2) |
| Downloadable knowledge graph | Y | Downloadable versions are available on Github |
Section Score: 5/5
Provenance of Nodes and Edges
| Question | Answer | Comment |
|---|---|---|
| Source list provided | Y | 70 sources Table 1 (UMLS, SemMedDB, ChEMBL, DrugBank, Reactome, SMPDB, and 64 additional knowledge sources). |
| Source versions information | Y | It documents the versions of the upstream sources used (https://github.com/RTXteam/RTX-KG2/blob/master/docs/kg2-versions.md). |
| Import dependencies | Y | In the requirements file. |
| Node and edge sources | Y | Node's ID contains source information and edge contains primary knowledge source. |
| Edges deduplication | Y | It provides a pre-canonicalized graph version (RTX-KG2pre, with semantically duplicated concepts) and a canonicalized version (RTX-KG2c, withthout semantically duplicated concepts) |
| Triples source details | Y | In the final output KG, each edge includes the source that reated that triple. |
| Edge type schema | Y | It uses Biolink Model for as the schema standard for both nodes and edges. |
Section Score: 7/7
Documented standards, schema, construction
| Question | Answer | Comment |
|---|---|---|
| Biological usable data | Y | It is used for other biological applications such as answering translational science questions, drug repositioning, identifying new therapeutic targets, and understanding drug mechanisms. |
| Resolvable IDs | Y | It uses resolvable IDs for the entities. |
| Construction documentation | Y | It has clear and step by step documentation on construction on its Github repo |
| Transformation documentation | Y | In Appendix |
| Schema used | Y | Biolink model and extract-transform-load (ETL) approach for construction. |
Section Score: 5/5
Update frequency and versioning
| Question | Answer | Comment |
|---|---|---|
| Stable versions | Y | It is using semantic versioning (e.g., KG2.7.3) |
| Public tracker information | Y | Provides public tracker for requests, bug reports on it Github repo. |
| Knowledge graph contact information | Y | It provides contact information of the KG2 Team. |
| Updated annually | Y | Once per month (mentioned in Discussion). |
| Prior versions access | Y | The prior versions are accessible (https://github.com/ncats/translator-lfs-artifacts/blob/main/README.md) with documented changes (https://github.com/RTXteam/RTX-KG2/blob/master/docs/kg2-versions.md). |
Section Score: 5/5
Evaluation - Metrics and Fitness for Purpose
| Question | Answer | Comment |
|---|---|---|
| Use case provided | Y | It is currently being used by multiple Translator reasoning agents such as ARAX (Autonomous Relay Agent X). |
| Evaluation against other models | Y | It is compared to four other KGs (Hetionet, SPOKE, the SRI Reference Knowledge Graph, and ROBOKOP) |
| Defined scope | Y | It is a part of NCATS Biomedical Data Translator project to support automated biomedical reasoning and question answering. It aims to create a semantically standardized, computable, and interoperable biomedical KG that supports translational reasoning and biomedical discovery. |
| Multiple evaluation methods | Y | It is not only evaluated with other KGs, but also evaluated on the tools that utilize it such as ARAX, mediKanren, BioThings Explorer, and ARAGORN. |
| Accuracy metrics | Y | The nodes and edges contain evidence, provenance, and other information for measuring accuracy and confidence. |
Section Score: 5/5
License Information
| Question | Answer | Comment |
|---|---|---|
| License | CC BY 4.0 |