benchmark · task

Zero-shot variant effect prediction

This paper-specific evaluation tests Zero-shot variant effect prediction using ProteinShake VEP datasets.

At a glance

Explanatory profile: limited source coverage · Automated source review, 2026-09-16. This does not change the review status of its results.

Data, procedure and scoring
Property	Description and evidence
Record type	Paper-specific task; protocol incompletely extractedEndowing protein language models with structural knowledge · Table 2, PST row, Zero-shot VEP Mean \|ρ\| column; Table 2, ESM-2 row, Zero-shot VEP Mean \|ρ\| column
Inputs	ProteinShake VEP datasetsEndowing protein language models with structural knowledge · Table 2, PST row, Zero-shot VEP Mean \|ρ\| column; Table 2, ESM-2 row, Zero-shot VEP Mean \|ρ\| column
Assessment	Mean \|Spearman rho\|Endowing protein language models with structural knowledge · Table 2, PST row, Zero-shot VEP Mean \|ρ\| column; Table 2, ESM-2 row, Zero-shot VEP Mean \|ρ\| column
Recorded split or evaluation setting	UnextractedEndowing protein language models with structural knowledge · Table 2, PST row, Zero-shot VEP Mean \|ρ\| column; Table 2, ESM-2 row, Zero-shot VEP Mean \|ρ\| column
Datasets	Not extracted or verified for this record.
Organisms	Not extracted or verified for this record.
Assays	Not extracted or verified for this record.
Adaptation	Not extracted or verified for this record.
Baselines	Not extracted or verified for this record.

How it works

Reported evaluation outline

Outline of the existing paper extraction. Split membership, fitting details and scorer implementation remain incompletely reviewed.

Read the diagram as text

ProteinShake VEP datasets
Recorded fitting or scoring procedure
Assess Mean |Spearman rho|

Endowing protein language models with structural knowledge · Table 2, PST row, Zero-shot VEP Mean |ρ| column; Table 2, ESM-2 row, Zero-shot VEP Mean |ρ| column

Evaluation context

The existing paper extraction describes: Zero-shot VEP; paper averages absolute Spearman correlations. This description is retained with the exact evaluation records; it is not a new protocol reconstruction.

Endowing protein language models with structural knowledge · Table 2, PST row, Zero-shot VEP Mean |ρ| column; Table 2, ESM-2 row, Zero-shot VEP Mean |ρ| column

Tested models and results

Release 2026-09-16-d74d282221a9 · 2 evaluations · 2 metric rows. Different protocols are not a single leaderboard.

Evaluated configurationMetricEvidence origin

Results grouped by the exact reported evaluation
Metric and finding	Coverage and uncertainty	Evidence
PST: Zero-shot variant effect prediction Model: PST · Benchmark: Zero-shot variant effect prediction · Dataset: ProteinShake VEP datasets Zero-shot VEP; paper averages absolute Spearman correlations. Author-reported evaluation · Evaluation metadata: needs review
0.501 Mean \|Spearman rho\| Unit: unitless · Direction: unknown	Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported	source checkedEndowing protein language models with structural knowledge · Table 2, PST row, Zero-shot VEP Mean \|ρ\| column Source checking is not independent reproduction.
ESM-2: Zero-shot variant effect prediction Model: ESM-2 · Benchmark: Zero-shot variant effect prediction · Dataset: ProteinShake VEP datasets Zero-shot VEP; paper averages absolute Spearman correlations. Independent external evaluation · Evaluation metadata: needs review
0.489 Mean \|Spearman rho\| Unit: unitless · Direction: unknown	Uncertainty: not reported in legacy extract Scored: Not reported · Eligible: Not reported	source checkedEndowing protein language models with structural knowledge · Table 2, ESM-2 row, Zero-shot VEP Mean \|ρ\| column Source checking is not independent reproduction.

Strengths and limitations

Strengths and considerations

The linked evaluation identifies the paper-specific dataset and assessment rather than treating the task name as a universal benchmark.Endowing protein language models with structural knowledge · Table 2, PST row, Zero-shot VEP Mean |ρ| column; Table 2, ESM-2 row, Zero-shot VEP Mean |ρ| column

Limitations and conditions

The numerical table check does not establish the full data-processing, fitting or leakage-control protocol.Endowing protein language models with structural knowledge · Table 2, PST row, Zero-shot VEP Mean |ρ| column; Table 2, ESM-2 row, Zero-shot VEP Mean |ρ| column

Profile review details

Catalogue extraction inspected; protocol claims remain limited to the cited evidence. Missing details are not presumed absent from the original paper.

Stable record: reported-task-a5141363b0ee45

Sources and history

Release 2026-09-16-d74d282221a9 · Record review: needs review

Endowing protein language models with structural knowledge · Original source · PMC12603367.1

Download this release

Technical metadata and extraction receipts

Stable ID: reported-task-a5141363b0ee45

areas: proteins-complexes
tasks: Zero-shot variant effect prediction
entity level: task
version: Not reported
task: Zero-shot variant effect prediction
scope note: Paper-specific evaluation task; protocol completeness requires further extraction.
missing metadata: protocol version: not_reported_in_legacy_extract; split: not_reported_in_legacy_extract

Related records

dataset: ProteinShake VEP datasets
benchmark: PST: Zero-shot variant effect prediction
benchmark: ESM-2: Zero-shot variant effect prediction

Suggest a correction