Evaluation for drugmechdb
Evaluator: Not specified
Evaluated on: 2025-08-26
This is a manual evaluation intended to identify potential barriers to reuse.
Access Level and Types
| Question | Answer | Comment |
|---|---|---|
| Access to data outside of the knowledge graph | Y | Paths that describe the mechanism of action for drug/disease indication pair |
| API or online access to the knowledge graph | Y | Website, can view all paths or only those related to a specific drug or disease |
| Multiple access options available | Y | Can download from zenodo or use website |
| Source code availability | Y | Https://github.com/SuLab/DrugMechDB |
| Downloadable knowledge graph | Y | Kind of on zenodo, but what they are calling a KG is a collection of entries for drug/disease pairs and the paths connecting them. Raw KG containing all edges between all entries does not appear to exist |
Section Score: 5/5
Provenance of Nodes and Edges
| Question | Answer | Comment |
|---|---|---|
| Source list provided | Y | Table with node sources. Indications: DrugCentral ( according to GH, randomly sampled from availble drug-disease pairs, does not include all of them), paths: Mechanism of Action section from DrugBank, description section within Inxight Drugs, review articles, GeneOntology , UniProt, Reactome, and well-sources Wikipedia articles. No Primary experimental results are included. All are verified by curators |
| Source versions information | N | Date given for DrugCentral download, cannot find info for other sources |
| Import dependencies | Y | There are requirements.txt and .yaml files in the KG's downloadable .zip file on Zenodo (https://zenodo.org/records/8139357) showing the dependencies. |
| Node and edge sources | Y | Each entry consist of ‘graph’, ‘links’, ‘nodes’, and ‘reference’ keys. Graph contains the drug/disease pair, links contains relationships informative to the indication, nodes contains information about all entities in link, and reference contains all the sources for the entire relationship but not specifically for individual nodes/edges |
| Edges deduplication | N | Paths are manually curated |
| Triples source details | N | Edges between informative nodes in a path do not contain orignal source infromation. Informative paths have citations for the entire path, not indivudal triples within them. |
| Edge type schema | Y | BioLink, but does not specify exactly what sources are used for each edge type |
Section Score: 4/7
Documented standards, schema, construction
| Question | Answer | Comment |
|---|---|---|
| Biological usable data | Y | Json files |
| Resolvable IDs | Y | All nodes come from existing sources |
| Construction documentation | N | Not really. Everything is "manually curated" |
| Transformation documentation | N | |
| Schema used | Y | BioLink |
Section Score: 3/5
Update frequency and versioning
| Question | Answer | Comment |
|---|---|---|
| Stable versions | Y | |
| Public tracker information | Y | GitHub Issues. Also includes a guide for how users can submit their own curated paths. |
| Knowledge graph contact information | Y | Indirectly. website indicated the lab who owns it, but does not provide a point of contact |
| Updated annually | N | V1 released in 2019 with one update less than a year later, same with V2 in 2023, no other updates |
| Prior versions access | N | All versions availble, V2 has publication associated but detailed change logs for minor updates not available |
Section Score: 3/5
Evaluation - Metrics and Fitness for Purpose
| Question | Answer | Comment |
|---|---|---|
| Use case provided | N | |
| Evaluation against other models | Y | They used an external KG to evaluate agreement between types of associations present in both graphs and calculate a p-value representing when the simulated percentage of matching was greater than or equal to the observed percentage |
| Defined scope | Y | Mechanisms of action by which a drug treats a disease |
| Multiple evaluation methods | N | |
| Accuracy metrics | Y | Agreement with MechRepoNet |
Section Score: 3/5
License Information
| Question | Answer | Comment |
|---|---|---|
| License | CC0 1.0 Universal |