Premise-doubt: do judges bias toward confident answers?

=== (a) bc mean by unanimity ===
  unanimous (n=140):  bc μ=0.711  σ=0.082
  split     (n=60):  bc μ=0.721  σ=0.080
  Cohen d (unanimous - split): -0.120
  → positive d means confident answers more likely unanimous — judges agree more on confident items (bias concern)

=== (b) bc vs majority-vote-count (0..3) ===
  spearman ρ=+0.232 p=0.0010

=== (c) bc AUC vs each individual judge ===
  groq/llama-3.3-70b-versatile                       n=200  AUC(CORRECT)=0.631  r=+0.230 p=0.0011
  groq/meta-llama/llama-4-scout-17b-16e-instruct     n=200  AUC(CORRECT)=0.576  r=+0.136 p=0.0543
  groq/openai/gpt-oss-120b                           n=200  AUC(CORRECT)=0.654  r=+0.275 p=0.0001

=== (d) judge pairwise agreement by bc quartile ===
  bc quartile 0 (n=50): mean pair agreement = 0.853
  bc quartile 1 (n=50): mean pair agreement = 0.707
  bc quartile 2 (n=50): mean pair agreement = 0.840
  bc quartile 3 (n=50): mean pair agreement = 0.800

== Interpretation ==
  (a) Cohen d (unanimous-split) < 0.2: judges NOT biased to bc
  (b) |ρ| < 0.15: bc NOT strongly monotone with judge count
  (c) Each judge AUC similar to majority AUC: no single-judge hijack
  (d) Agreement roughly constant across bc quartiles: no bc-bias
