research-document

FE-011A Analysis Plan

FE-011A Analysis Plan

  • Compare Version A vs Version B within each framework.
  • Compare original-style vs redesigned versions across frameworks.
  • Track whether redesigned versions improve evidence use, traceability, and actionability.
  • Track whether redesigned versions increase complexity.
  • Identify whether improvements are framework-specific or general.
  • Do not claim success unless blinded evaluation favors redesigned outputs on multiple dimensions.
  • Record cases where redesigns increase structure or verbosity without clear improvement in decision quality.
  • Treat output length and section count as possible confounds, not automatic evidence of improvement.
  • Report mixed or null findings explicitly rather than collapsing them into an overall redesign narrative.
  • Avoid cross-framework claims unless the direction of effect is consistent and the tradeoffs are similar.
  • Separate instrument findings from framework findings. A failure may indicate a flawed pilot design rather than a flawed framework redesign.