How we check Syndrome

An educational tool can be wrong too. Before we publish a figure, we compare what Syndrome computes with what the reference software used by researchers computes, using a criterion fixed in advance.

What is compared

Syndrome’s core, the same TypeScript code that runs in your browser, is run in Node over a grid of points: distances d = 3, 5 and 7 and eight values of p per noise model. The same grid is simulated by an independent reference in Python that decodes with PyMatching, the reference minimum-weight matching decoder, and uses Stim to check that the noise model with measurement faults is what it claims to be.

Both sides use the same geometry, the same noise model and the same integer weights in the matching graph, so that they compare exactly the same decoder. Each point is simulated until it has at least 100 logical failures and 10⁵ shots, with a cap of 10⁶.

The criterion, fixed before looking

For each point we compute z, the difference between the two estimates divided by their combined error. The validation passes if:

• no point has |z| > 3, and
• the sum Σz² over the 24 points of the grid is below 43 (chi-squared with 24 degrees of freedom, p > 0.01).

One or two points between 2σ and 3σ are what you expect from two correct simulations; requiring every point to fall within 2σ would make two correct simulations fail two times out of three. Every point is published with its 95% Wilson confidence interval.

With independent noise and d = 3 there is also an exact value: the logical error rate is computed by enumerating all 2¹⁸ possible error patterns. The reference simulation is checked against it with the same test, at the 8 values of p: |z| ≤ 3 at each one and Σz² below 20.1 (chi-squared with 8 degrees of freedom). Syndrome’s core has its own test against that exact value.

What is compared without statistics

• Minimum weights: for 10,000 syndromes per model, the weight of the matching Syndrome finds must be exactly equal to PyMatching’s.
• Verdicts with measurement faults: for 10,000 syndromes, the same weight and the same logical effect of the correction (at most 5 discrepancies are allowed, each one listed).

Result of the validation

Validation of 17 October 2026, per Syndrome’s validation report: passes with both models.

• Independent X/Z noise: max |z| = 1.55 and Σz² = 18.51; minimum weights equal in 10,000 out of 10,000.
• Phenomenological noise: max |z| = 2.17 and Σz² = 20.34; minimum weights equal in 10,000 out of 10,000 and the same logical effect in 10,000 out of 10,000.

If the comparison ever fails, the simulation is fixed, never the numbers: the comparator does not adjust, exclude or reweight any point. The validation is repeated every time Syndrome’s core changes.

The code

Syndrome’s code, with the reference and the comparator, is licensed under AGPL-3.0. The repository is not public for now.

Source for all of the above: Syndrome’s validation report, generated on 17 October 2026, and the physical and statistical conventions set in Syndrome’s repository (code geometry, decoder weights, noise models and validation criterion). What the chart says with those figures: What the Syndrome chart says.