Evaluation for biomarkerkg
Evaluator: Automated Evaluation
Evaluated on: 2026-01-06
⚠️ Automated Evaluation: This evaluation was generated automatically using an AI-based system. It is distinct from manual evaluations curated by human experts. Please review findings carefully and report any inaccuracies.
Evaluation Criteria: This evaluation uses the KG-Registry evaluation rubric as described in Cortes et al. (2025) . The rubric assesses knowledge graphs across multiple dimensions including access, provenance, documentation, maintenance, and fitness for purpose.
Access Level and Types
| Question | Answer | Comment |
|---|---|---|
| Access to data outside of the knowledge graph | Y | BKG provides a web interface (BKG Explorer) for querying and exploring biomarker-anatomy, biomarker-compound, biomarker-condition, biomarker-role, and biomarker-variant relationships |
| API or online access to the knowledge graph | N | The BKG web interface does not provide direct API access based on available documentation |
| Multiple access options available | Y | Multiple downloadable data formats available including CSV files organized by node and edge types from S3 bucket |
| Source code availability | N | No public source code repository mentioned for KG construction |
| Downloadable knowledge graph | Y | Individual CSV files for nodes (Anatomy, Biomarker, Compound, Condition, Role, Variant) and edges are available for download |
Section Score: 3/5
Provenance of Nodes and Edges
| Question | Answer | Comment |
|---|---|---|
| Source list provided | Y | Integrates data from well-known sources: GlyGen Biomarker Database, Uber-Anatomy Ontology (Uberon), PubChem, Human Disease Ontology (DOID), and dbSNP |
| Source versions information | N | Specific version information for source databases is not provided in the available documentation |
| Import dependencies | Y | Declares explicit relationships (node edge types): determined_using_sample_from (Anatomy), indicated_by_above/below_normal_level_of (Compound), diagnostic_for/indicates_risk_of/prognostic_for (Condition), has_best_classification (Role) |
| Node and edge sources | Y | Each product file is described with a specific relationship type (e.g., Biomarker to Anatomy relationships, Biomarker to Condition relationships) |
| Edges deduplication | N | No explicit documentation of deduplication mechanisms at the edges level |
| Triples source details | Y | Relationship types are clearly defined with meaningful semantic labels (diagnostic_for, prognostic_for, indicates_risk_of_developing, etc.) |
| Edge type schema | Y | Uses Uberon anatomy, DOID disease ontology, dbSNP variant IDs, and PubChem compound identifiers which are resolvable |
Section Score: 5/7
Documented standards, schema, construction
| Question | Answer | Comment |
|---|---|---|
| Biological usable data | N | Detailed construction documentation is not publicly available |
| Resolvable IDs | N | Data transformation steps and filtering procedures are not documented in available resources |
| Construction documentation | Y | Clear edge type schema with defined semantic relationships between biomarkers and other entity types |
| Transformation documentation | N | No formal versioning scheme mentioned (no v1.0, v2.0, etc.) |
| Schema used | N | No public issue tracker or GitHub repository visible for tracking feature requests or bugs |
Section Score: 1/5
Update frequency and versioning
| Question | Answer | Comment |
|---|---|---|
| Stable versions | Y | Contact information provided: avi.maayan@mssm.edu (MaayanLab) |
| Public tracker information | Y | Partially - KG is actively maintained but no formal update frequency documented (appears to be an active development project) |
| Knowledge graph contact information | N | No archived prior versions or changelog available |
| Updated annually | Y | BKG is designed for biomarker discovery and exploration, relevant for clinical diagnostics and precision medicine applications |
| Prior versions access | N | No comparisons with other biomarker knowledge bases mentioned in available documentation |
Section Score: 3/5
Evaluation - Metrics and Fitness for Purpose
| Question | Answer | Comment |
|---|---|---|
| Use case provided | Y | Focused scope on biomarker relationships across anatomical structures, compounds, conditions, roles, and genetic variants |
| Evaluation against other models | N | No quantitative evaluation metrics or benchmarks provided in available documentation |
| Defined scope | N | Accuracy, precision, recall, or other quality metrics are not reported |
| Multiple evaluation methods | N | No explicit evaluation methods comparison provided in documentation |
| Accuracy metrics |
Section Score: 1/4
License Information
| Question | Answer | Comment |
|---|---|---|
| License | CC-BY-4.0 (inferred from MaayanLab standard practice; not explicitly stated in resource metadata) |