Multi-metric ablation — beyond AUC
N_boot=2000  threshold=0.5

Per-variant metric table (A4 vs A1 = full − no_pert)
Lower Brier / ECE = better; higher AUC / PR-AUC / F1 = better.

-- TriviaQA objective  n=200  pos=0.280 --
metric           A4       A1         Δ                  95% CI  better?
auc          0.7457   0.7517   -0.0061  [-0.0634, +0.0429]      tie
pr_auc       0.5693   0.5740   -0.0047  [-0.0850, +0.0686]      tie
brier        0.1997   0.1955   +0.0043  [-0.0202, +0.0292]      tie
ece          0.1508   0.1673   -0.0165  [-0.0575, +0.0370]      tie
f1           0.5385   0.5082   +0.0303  [-0.0650, +0.1224]      tie
DeLong z = -0.217  p = 0.828

-- NQ-Open objective  n=75  pos=0.547 --
metric           A4       A1         Δ                  95% CI  better?
auc          0.4240   0.4907   -0.0667  [-0.1835, +0.0502]      tie
pr_auc       0.4934   0.5404   -0.0470  [-0.1374, +0.0390]      tie
brier        0.3342   0.2961   +0.0381  [-0.0058, +0.0839]      tie
ece          0.3047   0.2382   +0.0665  [-0.0466, +0.1796]      tie
f1           0.6047   0.6076   -0.0029  [-0.1013, +0.1044]      tie
DeLong z = -1.103  p = 0.270

-- TruthfulQA single  n=200  pos=0.635 --
metric           A4       A1         Δ                  95% CI  better?
auc          0.6915   0.7183   -0.0268  [-0.0852, +0.0347]      tie
pr_auc       0.7997   0.8261   -0.0264  [-0.0737, +0.0213]      tie
brier        0.2279   0.2157   +0.0122  [-0.0122, +0.0359]      tie
ece          0.1426   0.1035   +0.0392  [-0.0232, +0.0819]      tie
f1           0.7126   0.7177   -0.0052  [-0.0649, +0.0519]      tie
DeLong z = -0.888  p = 0.374

-- TruthfulQA majority  n=200  pos=0.560 --
metric           A4       A1         Δ                  95% CI  better?
auc          0.6607   0.6417   +0.0190  [-0.0421, +0.0774]      tie
pr_auc       0.7387   0.7020   +0.0367  [-0.0209, +0.0902]      tie
brier        0.2440   0.2437   +0.0002  [-0.0226, +0.0250]      tie
ece          0.1427   0.1033   +0.0394  [-0.0406, +0.0845]      tie
f1           0.6355   0.6425   -0.0070  [-0.0763, +0.0601]      tie
DeLong z = +0.627  p = 0.531

-- TruthfulQA unanimous  n=140  pos=0.571 --
metric           A4       A1         Δ                  95% CI  better?
auc          0.6746   0.6965   -0.0219  [-0.0927, +0.0505]      tie
pr_auc       0.7401   0.7467   -0.0066  [-0.0723, +0.0555]      tie
brier        0.2453   0.2268   +0.0185  [-0.0117, +0.0485]      tie
ece          0.1589   0.1171   +0.0419  [-0.0382, +0.1146]      tie
f1           0.6309   0.6752   -0.0443  [-0.1269, +0.0384]      tie
DeLong z = -0.597  p = 0.551


== Interpretation ==
  AUC CI covering 0 is the headline, but if Brier / ECE CI excludes 0, the
  perturbation set does move *calibration* even when it doesn't move ranking.
  DeLong z-test supplements the bootstrap with a parametric comparison.
