research-document
Blinded Evaluation Rubric
Blinded Evaluation Rubric
Score each dimension from 1 (poor) to 5 (strong), except D11 and D12 where 5 means more harm.
| ID | Dimension | Question |
|---|---|---|
| D1 | Problem comprehension | Is the central problem correctly understood? |
| D2 | Evidence fidelity | Is supplied evidence used accurately without invented facts? |
| D3 | Assumption visibility | Are material assumptions exposed? |
| D4 | Contradiction handling | Is conflicting evidence identified and reasoned through? |
| D5 | Uncertainty calibration | Is confidence proportional to evidence? |
| D6 | Premature-closure resistance | Does it avoid stronger conclusions than evidence supports? |
| D7 | Missing-information quality | Would the identified information materially change diagnosis? |
| D8 | Next-action value | Are actions specific and likely to reduce uncertainty? |
| D9 | Revision readiness | Can another investigator revise efficiently with new evidence? |
| D10 | Clarity | Is diagnosis and next action quickly understandable? |
| D11 | Burden (harm) | How much unnecessary structure, repetition, or overhead exists? |
| D12 | False precision (harm) | How much unjustified scoring, labeling, or structure exists? |
Favor evidence fidelity, calibrated uncertainty, discriminating explanations, important unknowns, and uncertainty-reducing actions. Penalize unsupported certainty, invented evidence, generic actions, ceremonial structure, excess length without value, and ID citation that does not improve reasoning. Do not score polish as diagnostic quality.