rewire.itbenchmarks
Configuration

Pangolin 1.0.2 + per-gene masking patch · mask True (P1)

Pangolin predicts splice-site strength and changes caused by genetic variants.

Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage

1 evaluation · 4 results

How it worksPangolin workflow
Pangolin workflow1. Variant and reference genome. Then: 2. Construct sequence inputs. Then: 3. Splice-strength prediction. Then: 4. Reference/alternate comparisonPangolin workflow1. Variant and reference genome. Then: 2. Construct sequence inputs. Then: 3. Splice-strength prediction. Then: 4. Reference/alternate comparisonPangolin workflow1. Variant and reference genome. Then: 2. Construct sequence inputs. Then: 3. Splice-strength prediction. Then: 4. Reference/alternate comparison

Conceptual summary of the documented data flow; optional inputs and configured downstream stages must be reported for a reproducible evaluation.

Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage

Overview

Model type

Dilated convolutional splicing predictor

Inputs

VCF or CSV variants, reference FASTA and matching gene annotations; custom sequence inference is also available.

Outputs

Predicted increases/decreases in splice-site strength and their positions.

Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage

limited source coverage · Automated source review, 2026-09-16. All specifications and missing details

Evaluations and results

1 evaluation · 4 results. Different protocols are not a single leaderboard.

Filter evaluations

Applied filters: All linked evaluations

Exact evaluated configurations and original reported results
Tested configurationProtocol and datasetFindingEvidence and details
Configuration: Pangolin 1.0.2 + per-gene masking patch · mask True (P1)Protocol: MFASS: matched GENCODE 44 canonical annotation
Dataset subset: MFASS v2 test: matched canonical annotation coverage
0.873 AUROC
dimensionless · higher

Uncertainty: Not reported

Coverage: 8297/8324

Rewire evaluation · Source checked
Methods, coverage and source

Pangolin 1.0.2 + per-gene masking patch · mask True (P1) on MFASS matched annotation

MFASS matched GENCODE 44 canonical annotation

Aggregation: Not reported

MFASS matched canonical annotation v1: report.json; MFASS matched canonical annotation v1: manifest-v1.json; MFASS matched canonical annotation v1: verification.json; MFASS matched canonical annotation v1: provenance.json; MFASS matched canonical annotation v1: exclusion-verification.json · report.json: conditions.P1.metrics.auroc
Configuration: Pangolin 1.0.2 + per-gene masking patch · mask True (P1)Protocol: MFASS: matched GENCODE 44 canonical annotation
Dataset subset: MFASS v2 test: matched canonical annotation coverage
0.411 Average precision
dimensionless · higher

Uncertainty: Not reported

Coverage: 8297/8324

Rewire evaluation · Source checked
Methods, coverage and source

Pangolin 1.0.2 + per-gene masking patch · mask True (P1) on MFASS matched annotation

MFASS matched GENCODE 44 canonical annotation

Aggregation: Not reported

MFASS matched canonical annotation v1: report.json; MFASS matched canonical annotation v1: manifest-v1.json; MFASS matched canonical annotation v1: verification.json; MFASS matched canonical annotation v1: provenance.json; MFASS matched canonical annotation v1: exclusion-verification.json · report.json: conditions.P1.metrics.average_precision_sklearn
Configuration: Pangolin 1.0.2 + per-gene masking patch · mask True (P1)Protocol: MFASS: matched GENCODE 44 canonical annotation
Dataset subset: MFASS v2 test: matched canonical annotation coverage
0.66 Precision at 100
dimensionless · higher

Uncertainty: Not reported

Coverage: 8297/8324

Rewire evaluation · Source checked
Methods, coverage and source

Pangolin 1.0.2 + per-gene masking patch · mask True (P1) on MFASS matched annotation

MFASS matched GENCODE 44 canonical annotation

Aggregation: Not reported

MFASS matched canonical annotation v1: report.json; MFASS matched canonical annotation v1: manifest-v1.json; MFASS matched canonical annotation v1: verification.json; MFASS matched canonical annotation v1: provenance.json; MFASS matched canonical annotation v1: exclusion-verification.json · report.json: conditions.P1.metrics.precision_at_capacity
Configuration: Pangolin 1.0.2 + per-gene masking patch · mask True (P1)Protocol: MFASS: matched GENCODE 44 canonical annotation
Dataset subset: MFASS v2 test: matched canonical annotation coverage
0.21 Recall at 100
dimensionless · higher

Uncertainty: Not reported

Coverage: 8297/8324

Rewire evaluation · Source checked
Methods, coverage and source

Pangolin 1.0.2 + per-gene masking patch · mask True (P1) on MFASS matched annotation

MFASS matched GENCODE 44 canonical annotation

Aggregation: Not reported

MFASS matched canonical annotation v1: report.json; MFASS matched canonical annotation v1: manifest-v1.json; MFASS matched canonical annotation v1: verification.json; MFASS matched canonical annotation v1: provenance.json; MFASS matched canonical annotation v1: exclusion-verification.json · report.json: conditions.P1.metrics.recall_at_capacity

Source checking is not independent reproduction. Release 2026-09-25-d40cee0abe73.

Use this model

How it works, versions and access

Related profile: Pangolin. This page retains the exact record and its evaluation context.

This configuration

8,297 of 8,324 held-out variants scored in every configuration (314 positives, 460 exon groups). The same 27 rows are excluded: 23 assembly-orientation mismatches and four outside the selected canonical transcript spans. Missing scores are not zero or negative predictions.

record
Pangolin 1.0.2 + per-gene masking patch · mask True (P1)
configuration
Not reported
entity type
Configuration

How it works

How it works

Pangolin predicts splice-site strength and changes caused by genetic variants. Dilated convolutional network with 16 residual blocks and skip connections; separate probability and usage outputs for heart, liver, brain and testis. The documented inputs are VCF or CSV variants, reference FASTA and matching gene annotations; custom sequence inference is also available. The output consists of predicted increases/decreases in splice-site strength and their positions.

Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
Versions and reproducibility

Pangolin implementation; gene-annotation database and selected weights must be recorded with a run. 5,000 bases upstream and downstream each output position; minimum 10,001-base input for one prediction, with 15,000-base training blocks producing 5,000 central outputs.

Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
Strengths, limitations and unresolved questions

Strengths and limitations

Strengths and considerations

Limitations and conditions

  • Only substitutions and simple insertions/deletions are supported. The documented tool skips variants outside annotated genes, near chromosome ends, inconsistent with the reference or beyond supported deletion lengths.
    Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
Profile review details

Inspected pinned official documentation, relevant implementation files and named primary-paper sections. Claims are limited to those artifacts. Remaining field extraction and identity conflicts are explicit; no new performance claims, model runs or human review are implied.

Stable record: discovery-model-pangolin

Specifications

Inputs, training, access and other details

Explanatory profile: limited source coverage · Automated source review, 2026-09-16. Review applies to the cited claims; unresolved fields are listed below. Numerical results retain their own review status.

Inputs, outputs and configuration
PropertyDescription and evidence
Model typeDilated convolutional splicing predictor
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
ArchitectureDilated convolutional network with 16 residual blocks and skip connections; separate probability and usage outputs for heart, liver, brain and testis.
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
InputsVCF or CSV variants, reference FASTA and matching gene annotations; custom sequence inference is also available.
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
OutputsPredicted increases/decreases in splice-site strength and their positions.
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
ParametersThe reviewed architecture section specifies the dilated residual network, but does not give a complete parameter total for the released ensemble. · Not reported in inspected sources
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
Known versionsPangolin implementation; gene-annotation database and selected weights must be recorded with a run.
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
Training dataHuman, rhesus macaque, mouse and rat sequence/splicing data. Human test chromosomes 1, 3, 5, 7 and 9 are held out, with homologous training genes filtered using Ensembl BioMart.
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
Training cutoffTraining annotations are GENCODE 34 (human), Ensembl 100 (rhesus), GENCODE M25 (mouse) and Ensembl 101 (rat). These component releases do not establish one latest RNA-seq collection date.
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
Context limits5,000 bases upstream and downstream each output position; minimum 10,001-base input for one prediction, with 15,000-base training blocks producing 5,000 central outputs.
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
Weights licenceSeparate checkpoint-distribution terms are not stated in the inspected release documentation and licence material. The source-code licence alone is not recorded as an explicit weight grant. · Not reported in inspected sources
Sources (3)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML; tkzeng/Pangolin: LICENSE · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage; LICENSE: licence text
AccessOfficial project documentation and implementation: https://github.com/tkzeng/Pangolin
Sources (2)tkzeng/Pangolin: README.md; pangolin: Journal full-text XML · Paper: Deep neural network architecture and Generating training and test sets; README.md: Usage
Code licenceGPL-3.0; inspect the pinned licence and any file-specific terms.
Sourcestkzeng/Pangolin: LICENSE · LICENSE: licence text

Evidence

Source checking verifies the cited claim or transcription. It does not establish independent reproduction.

Evidence table

Inspect claims, sources and review details

Trace each statement to its source and review. A context-only reference supports the record generally; it does not verify an individual field. Source checking does not reproduce an experiment.

One row per statement and cited source. Multiple citations are not independent evaluations. Shared locators are labelled explicitly.

5 evidence rows matching the loaded filters

Claims, original sources and review scope · Release 2026-09-25-d40cee0abe73
Property and statementOriginal source and locationReview and provenance
Relationship: variant of
discovery-model-pangolin
Individual claims
MFASS matched canonical annotation v1: exclusion-verification.json

Original source ↗

manifest-v1.json: study and conditions

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 4be7a98e2553fa2378c29625b13eb3e8ac2e58fb
Retrieved: 2026-09-25T11:24:56Z

source checked

automated execution evidence review · 2026-09-25T11:24:56Z

Audit details

Computational source, coverage and metric checks only; not independent human review or published-score reproduction.

Field: links:variant_of:discovery-model-pangolin

Claim: rewire-mfass-matched-v1-configuration-p1-variant-of-discovery-model-pangolin

Source artifact SHA-256: 5d2099db4cf6f01432fd88c7db949da89ab9917940705fd0eaa1fb7c9354d46a

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relationship: variant of
discovery-model-pangolin
Individual claims
MFASS matched canonical annotation v1: manifest-v1.json

Original source ↗

manifest-v1.json: study and conditions

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 093fd1ae198c80ce34408d84d6543bca4fc538f2
Retrieved: 2026-09-25T11:24:56Z

source checked

automated execution evidence review · 2026-09-25T11:24:56Z

Audit details

Computational source, coverage and metric checks only; not independent human review or published-score reproduction.

Field: links:variant_of:discovery-model-pangolin

Claim: rewire-mfass-matched-v1-configuration-p1-variant-of-discovery-model-pangolin

Source artifact SHA-256: 3fce70dd0bd96cb92d9d0e41287f31e62ddc4ecb8d188615093a873e952433e8

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relationship: variant of
discovery-model-pangolin
Individual claims
MFASS matched canonical annotation v1: provenance.json

Original source ↗

manifest-v1.json: study and conditions

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 093fd1ae198c80ce34408d84d6543bca4fc538f2
Retrieved: 2026-09-25T11:24:56Z

source checked

automated execution evidence review · 2026-09-25T11:24:56Z

Audit details

Computational source, coverage and metric checks only; not independent human review or published-score reproduction.

Field: links:variant_of:discovery-model-pangolin

Claim: rewire-mfass-matched-v1-configuration-p1-variant-of-discovery-model-pangolin

Source artifact SHA-256: 6304e9625e793f24cccdc2669d092e6cdcc7a508a7f789967e726816fffde062

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relationship: variant of
discovery-model-pangolin
Individual claims
MFASS matched canonical annotation v1: report.json

Original source ↗

manifest-v1.json: study and conditions

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 093fd1ae198c80ce34408d84d6543bca4fc538f2
Retrieved: 2026-09-25T11:24:56Z

source checked

automated execution evidence review · 2026-09-25T11:24:56Z

Audit details

Computational source, coverage and metric checks only; not independent human review or published-score reproduction.

Field: links:variant_of:discovery-model-pangolin

Claim: rewire-mfass-matched-v1-configuration-p1-variant-of-discovery-model-pangolin

Source artifact SHA-256: 259eb542313914b9114c21b62891f022c0ec0d9e3337196b8d7b2c6d5ae54ef3

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Relationship: variant of
discovery-model-pangolin
Individual claims
MFASS matched canonical annotation v1: verification.json

Original source ↗

manifest-v1.json: study and conditions

Shared locator for this statement’s cited sources; not a separate locator for each citation.

Version: 093fd1ae198c80ce34408d84d6543bca4fc538f2
Retrieved: 2026-09-25T11:24:56Z

source checked

automated execution evidence review · 2026-09-25T11:24:56Z

Audit details

Computational source, coverage and metric checks only; not independent human review or published-score reproduction.

Field: links:variant_of:discovery-model-pangolin

Claim: rewire-mfass-matched-v1-configuration-p1-variant-of-discovery-model-pangolin

Source artifact SHA-256: 13356c3f1bd7055b11cc0ab1b2b86a7ee586c51d0c984fad6e39ce8948d662fa

Hash scope: Hash scope not separately documented; inspect source record

Inspected artifact

Sources and history

View linked audit checks and correction history

Release 2026-09-25-d40cee0abe73 · Record review: source checked

5 source records and release historyDownload this release
Technical metadata and extraction receipts

Stable ID: rewire-mfass-matched-v1-configuration-p1

areas
dna-genomes
condition
P1
configuration
tool: pangolin; mask: True; stem: pangolin-gencode44-canonical-maskTrue
distance
50
annotation
GENCODE v44 canonical transcripts
code
pangolin source sha256: 239c6ba9b1883d712d4856985c6eaa07e5b67f99790a718f93d6316ad076c083; pangolin source identity: pangolin-5cf94b8-mask-per-gene-1; pangolin version: 1.0.2+rewire.maskpergene1; pangolin patch: id: pangolin-5cf94b8-mask-per-gene-1; patch sha256: d1c9fb69245f9d256e2547fd8bf61547686cfc3caf6c40c1351ab2d6997cb454; upstream revision: 5cf94b8db938c658391b4305cd7ce33297d44ff7; upstream source sha256: f3f25c4febf64d01ef42f967dc5cf10f6856bf3aa92d26f09803ad408169da9b; pangolin files: init .py: sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855; git blob: e69de29bb2d1d6434b8b29ae775ad8c2e48c5391; model.py: sha256: 4a1c5c2570aafe1452bb43332255321677e6c6c817adf84b9dd438e3ca4be6f8; git blob: 11dfb436b36f9b5783874b2db636a1788bfab619; models/final.1.0.3.v2: sha256: f0478fab173b75f7f7e9fe96688bad6c50fa4a46d70557f423b110caaf565501; git blob: fbe03218df2f781a9713b05bfa42001ae548782a; models/final.2.0.3.v2: sha256: c4c6bb4880fa6fb28b14182ae3ea0600edb07056158f55325b5e6e6e48fc9f26; git blob: ec4ee767bc83fa0397222f42c0da210a47f13fbb; models/final.3.0.3.v2: sha256: ec685a6e7105a4486c1f89a005458a13deb3fe7171f13d434f4877e386d10676; git blob: f31cbae821e38bc4361216c8e87c8db13fd5a9c0; models/final.1.2.3.v2: sha256: 559c05de3e1ce65c2515ca3e92ef85edb0ec2e47686ca58060e25891ce06eb3a; git blob: 39e8756faf1e427cd9a8c6c691532b3683d8916a; models/final.2.2.3.v2: sha256: 48758ba8b95eee9aa9feea52672ef06ca1b34111299c27f8a710f734d8b9aae5; git blob: 6fd0a31789c7de788227e103f33d697ad538def6; models/final.3.2.3.v2: sha256: 7cb576c2b24db4fdd6970c4ca4fb7c20ae1b1d8ae80645ebbe689848b5743129; git blob: c7f2ae10caab8f5bae167309b2b7d3e6905871e5; models/final.1.4.3.v2: sha256: c50b12e0c0af776d5674ca5e346493f8265783494d4df383364de9c1136657f6; git blob: 0af4fadb3f3dd54c49d41042befefa19fd4bd32a; models/final.2.4.3.v2: sha256: e03303bed4fd6f135ec0f6c1b192cce954ea42d0646f44d17b4a6fbb2b1f610e; git blob: 8c003df629c9f1e8072000f8bbc3eeab94850b6c; models/final.3.4.3.v2: sha256: 9476d2e25520d7ff15bece0cd5d3b657e3b1dd3cc5fcab1d9c3b62bea7a0c5b6; git blob: 9ed4fad50ead41761073cdff3d9acafc6f78bdd4; models/final.1.6.3.v2: sha256: 2aae563fa18a8a9b6699c6c96e0d32b8ec7543f8f805fb3bc9de77302cc9f66e; git blob: 98e5539fe339385e9c8b19d2786c4f8c3f186a3b; models/final.2.6.3.v2: sha256: 7d3c0b1b2a60067b940dec315567874fbc8bcd322f1b7c76bf969f51f0f53f7f; git blob: 6a650d629fae6aee8171708241a0f580bb368cfd; models/final.3.6.3.v2: sha256: 756e7721a382cace24e9bfea5b543af5623f2487d9a3efe7385e9c76367005fd; git blob: 4a8e9e33d5832770e6563d0229bdb4bcae5b8398; pangolin upstream files verified: true; pangolin upstream file mismatches: None recorded
resources sha256
cohort: 389702ff4c647d7ce10a90092a6fa811ae777d15997baf39ce9aae0346247bd0; split: 999ebcb7e63a5c5eaa8780fa468e59ac1f934260ad50102814174c396317f052; fasta: e49b92b3e4f321bf254c042f25b726d9931c4d74c7523e8b6bb530e63b0cfd4b; fai: a2c323ea4cff34d7123ace4578f7e122b2d2f5a22f40dc23eb8b97d17723d169; spliceai annotation: 2383cd148b292b9a6af56da160c9c142ddeb5720b1e9ef7c5966a1744f684c4d; pangolin db: 84ae1a18d9d7a32e14e3fe9c8705256a28cd9bde0a55bc46200cbe77178deb27
source locator
manifest-v1.json: conditions.P1; code.pangolin
limitations
8,297 of 8,324 held-out variants scored in every configuration (314 positives, 460 exon groups). The same 27 rows are excluded: 23 assembly-orientation mismatches and four outside the selected canonical transcript spans. Missing scores are not zero or negative predictions.; Exploratory comparison: prior results were known. Paired contrast intervals are unadjusted and do not establish a universal model ranking.; Pangolin uses the recorded per-gene masking patch; these are exact configurations, not unqualified upstream model scores.; Precision at 100 is sensitive to tied-score ordering, especially P1. Numerical source checking is automated, not human review or independent reproduction.; Assembly-orientation issue reported at https://github.com/KosuriLab/MFASS/issues/1. Original v1 outputs remain unchanged; corrections require a new version.
Related records

Suggest a correction