Evaluation for emikg
Evaluator: Automated Evaluation
Evaluated on: 2026-01-15
⚠️ Automated Evaluation: This evaluation was generated automatically using an AI-based system. It is distinct from manual evaluations curated by human experts. Please review findings carefully and report any inaccuracies.
Evaluation Criteria: This evaluation uses the KG-Registry evaluation rubric as described in Cortes et al. (2025) . The rubric assesses knowledge graphs across multiple dimensions including access, provenance, documentation, maintenance, and fitness for purpose.
Access Level and Types
| Question | Answer | Comment |
|---|---|---|
| Access to data outside of the knowledge graph | Y | Zenodo v5 package includes inputs/processed data and ontology alongside KG (Zenodo 10.5281/zenodo.17079767). |
| API or online access to the knowledge graph | Y | SPARQL endpoint at https://qlever.earthmetabolome.org/api/metrin-kg and hosted query UI. |
| Multiple access options available | Y | SPARQL endpoint, web query editor, and RDF dump on Zenodo. |
| Source code availability | Y | Code in GitHub repos earth-metabolome-initiative/metrin-kg and earth_metabolome_ontology. |
| Downloadable knowledge graph | Y | RDF dump provided via Zenodo (v1-v5 releases, e.g., 10.5281/zenodo.17079767). |
Section Score: 5/5
Provenance of Nodes and Edges
| Question | Answer | Comment |
|---|---|---|
| Source list provided | Y | Zenodo description lists pf1600, Globi, TRY, structure metadata and ontology inputs. |
| Source versions information | Y | Zenodo provides versioned releases (v1–v5) with DOI per version. |
| Import dependencies | Y | Build tutorial documents MySQL, Ontop, Pipenv and other required tools in repo README. |
| Node and edge sources | Y | KG construction docs map entities/edges from pf1600, Globi, TRY and structure metadata into EMI-based triples. |
| Edges deduplication | N | No explicit deduplication process described. |
| Triples source details | N | Per-triple provenance not documented. |
| Edge type schema | Y | Edge predicates defined by the EMI ontology used to generate the KG. |
Section Score: 5/7
Documented standards, schema, construction
| Question | Answer | Comment |
|---|---|---|
| Biological usable data | Y | Metabolomics, taxonomy, traits, and interactions content (METRIN-KG dataset). |
| Resolvable IDs | Y | Uses w3id.org/emi IRIs and external IDs from source datasets. |
| Construction documentation | Y | README and tutorials describe KG generation workflow and configuration files. |
| Transformation documentation | Y | Step-by-step instructions for mapping, Ontop materialization, and data loading. |
| Schema used | Y | EMI ontology (w3id.org/emi) provides the schema for nodes and predicates. |
Section Score: 5/5
Update frequency and versioning
| Question | Answer | Comment |
|---|---|---|
| Stable versions | Y | Versioned Zenodo releases (v1–v5) with persistent DOIs. |
| Public tracker information | Y | Issues enabled on GitHub (earth-metabolome-initiative/metrin-kg, earth_metabolome_ontology). |
| Knowledge graph contact information | Y | Contact via GitHub issues and project team listed on Zenodo. |
| Updated annually | Y | Multiple releases throughout 2025 (v1–v5) and active development. |
| Prior versions access | Y | Zenodo retains all prior versions (v1–v4) alongside latest. |
Section Score: 5/5
Evaluation - Metrics and Fitness for Purpose
| Question | Answer | Comment |
|---|---|---|
| Use case provided | Y | KG targets metabolomics/traits/interactions for Earth Metabolome Initiative datasets. |
| Evaluation against other models | N | No comparative evaluation reported. |
| Defined scope | Y | Scope defined in README and Zenodo description (METRIN-KG for metabolites, traits, interactions). |
| Multiple evaluation methods | N | No multiple evaluation methodologies described. |
| Accuracy metrics | N | No accuracy or quality metrics provided. |
Section Score: 2/5
License Information
| Question | Answer | Comment |
|---|---|---|
| License | Y | CC0-1.0 (repo LICENSE and Zenodo record). |