bc extraction quality audit across providers × datasets

dataset                                          n  cor   bc_μ   bc_σ   min   max   0.5%    μC    μI       Δ
------------------------------------------------------------------------------------------------------------
Cerebras-llama-3.1-8b (TruthfulQA)             413  254  0.715  0.080  0.45  1.00   0.0%  0.72  0.72  -0.000
NIM-llama-3.1-8b (TruthfulQA)                  100   62  0.374  0.170  0.06  1.00   0.0%  0.36  0.40  -0.040
Cohere-command-a (TruthfulQA)                  100   79  0.524  0.110  0.36  0.91   0.0%  0.53  0.52  +0.006
Mistral-small (TruthfulQA)                     100   63  0.371  0.095  0.03  0.70   2.0%  0.37  0.38  -0.012
Groq-llama-4-scout (TruthfulQA)                  4    3  0.387  0.154  0.19  0.52   0.0%  0.45  0.19  +0.261
Groq-qwen3-32b (TruthfulQA)                      3    3  0.474  0.029  0.45  0.51   0.0%  0.47   nan    +nan
Cerebras-llama-3.1-8b (TriviaQA-200)           200  144  0.750  0.099  0.41  0.98   0.0%  0.77  0.69  +0.084
Qwen3.5-122B NIM (TruthfulQA 3cat)               8    7  0.694  0.097  0.50  0.82  12.5%  0.72  0.50  +0.221
Llama-4-Maverick NIM (TruthfulQA 3cat)           7    6  0.905  0.039  0.85  0.95   0.0%  0.90  0.92  -0.016

== Interpretation ==
  bc_σ ≈ 0.05 → narrow; bc is almost constant, info-theoretic limit to AUC
  0.5%  → fraction of logprob-fallback (0.5 placeholder); high % = extraction failure
  Δ     → mean bc gap between correct/incorrect; |Δ|<0.02 means bc useless pointwise

== Doubted premise ==
  If TruthfulQA bc σ is similar to TriviaQA, the chance-level AUC is NOT
  an extraction artifact — it is a semantic property of TruthfulQA.
