research-document

Exp 002 Comparison Summary

Exp 002 Comparison Summary

Run State: f443d0d6-9763-45e1-911e-e014ab1856f2

Experiment: ECR-000003 EXP-002 Cross Representation Stability

Dataset Completeness

  • Total records discovered: 12
  • Included in primary comparison: 12
  • Missing or malformed: 0

Packet Versions

  • ECR-000003-EXP002-P001A: not stated
  • ECR-000003-EXP002-P001B: not stated
  • ECR-000003-EXP002-P001C: not stated
  • ECR-000003-EXP002-P001D: not stated

Response Validity

  • Valid primary records: 12/12
  • Data-quality issues: 4

Recognition Persistence Summary

  • gpt: persistent_recognition
  • claude: unstable_recognition
  • gemini: unstable_recognition

Structural Stability Summary

  • Backbone profile: mixed
  • Literal profile: mostly_stable
  • Conceptual profile: mostly_stable
  • Dimensional profile: mostly_stable

Provider-Specific Differences

  • gpt pattern: persistent_recognition.
  • claude pattern: unstable_recognition.
  • gemini pattern: unstable_recognition.
  • gpt introduced watched domain terms in 2 responses.
  • claude introduced watched domain terms in 2 responses.
  • gemini introduced watched domain terms in 1 responses.

Largest Disagreements

  • Structural backbone stability is stronger than wording-sensitive literal agreement.
  • Primitive disagreement: primitive_sequence.
  • Primitive disagreement: transitions.
  • Primitive disagreement: transition_backbone.
  • Primitive disagreement: dominant_primitive.
  • Primitive disagreement: dominant_role.
  • Primitive disagreement: candidate_missing_primitives.
  • Constraint disagreement: constraint_layer.invariants.
  • Constraint disagreement: constraint_layer.preconditions.
  • Constraint disagreement: constraint_layer.postconditions.
  • Constraint disagreement: constraint_layer.stopping_criteria.
  • Constraint disagreement: constraint_layer.validity_conditions.

Domain Leakage Findings

  • ECR-000003-EXP002-P001A gpt: evidence (possible_structural_inference).
  • ECR-000003-EXP002-P001A claude: diagnosis, differential diagnosis, hypothesis testing, evidence (likely_prior_knowledge_leakage).
  • ECR-000003-EXP002-P001A gemini: diagnosis, differential diagnosis, evidence (likely_prior_knowledge_leakage).
  • ECR-000003-EXP002-P001B gpt: evidence (possible_structural_inference).
  • ECR-000003-EXP002-P001B claude: diagnosis, differential diagnosis, hypothesis testing, evidence (likely_prior_knowledge_leakage).

Unresolved Issues

  • dataset : warning_1 (Tolerant parsing used for ECR-000003-EXP002-P001A gpt: responses/gpt/ECR-000003-EXP002-P001A-gpt.json)
  • dataset : warning_2 (Tolerant parsing used for ECR-000003-EXP002-P001B gpt: responses/gpt/ECR-000003-EXP002-P001B-gpt.json)
  • dataset : warning_3 (Tolerant parsing used for ECR-000003-EXP002-P001C gpt: responses/gpt/ECR-000003-EXP002-P001C-gpt.json)
  • dataset : warning_4 (Tolerant parsing used for ECR-000003-EXP002-P001D gpt: responses/gpt/ECR-000003-EXP002-P001D-gpt.json)

Readiness For EDR Review

  • Observation-only EDR draft is ready for human completion.