Ablation #46 WHY — why do the 13 perturbation features hurt AUC?
n=412  pos_rate=0.614  seed=42  5-fold CV; bootstrap on sweep only

== Phase A: Single-feature AUC ranking (point estimate) ==
feature                                           AUC  type
log__verbal_logprob_gap                        0.6023  NO-PERT/DERIVED
inv__verbal_logprob_gap                        0.5978  NO-PERT/DERIVED
inv__baseline_confidence                       0.5743  NO-PERT/DERIVED
log__baseline_confidence                       0.5741  NO-PERT/DERIVED
baseline_confidence                            0.5740  NO-PERT/DERIVED
inv__paraphrase_fragility                      0.5550  PERT
log__paraphrase_fragility                      0.5528  PERT
paraphrase_fragility                           0.5525  PERT
answer_length                                  0.5458  NO-PERT/DERIVED
log__adaptive_fragility                        0.5366  PERT
adaptive_fragility                             0.5359  PERT
answer_char_length                             0.5358  NO-PERT/DERIVED
inv__adaptive_fragility                        0.5348  PERT
impostor_fragility                             0.5214  PERT
fragility_score                                0.5174  PERT
log__impostor_fragility                        0.5133  PERT
log__fragility_score                           0.5131  PERT
log__vulnerability                             0.5106  PERT
log__dissociation_rate                         0.5094  PERT
inv__dissociation_rate                         0.5083  PERT
inv__vulnerability                             0.5028  PERT
counterfactual_fragility                       0.5010  PERT
has_uncertainty_words                          0.4983  NO-PERT/DERIVED
has_digit                                      0.4981  NO-PERT/DERIVED
inv__fragility_score                           0.4956  PERT
num_sentences                                  0.4948  NO-PERT/DERIVED
verbalized_confidence                          0.4945  NO-PERT/DERIVED
dissociation_rate                              0.4944  PERT
confidence_range                               0.4912  PERT
log__maladaptive_fragility                     0.4879  PERT
log__confidence_range                          0.4878  PERT
inv__verbalized_confidence                     0.4867  NO-PERT/DERIVED
inv__maladaptive_fragility                     0.4814  PERT
inv__confidence_range                          0.4814  PERT
vulnerability                                  0.4800  PERT
adversarial_fragility                          0.4782  PERT
inv__adversarial_fragility                     0.4745  PERT
maladaptive_fragility                          0.4719  PERT
inv__counterfactual_fragility                  0.4714  PERT
log__verbalized_confidence                     0.4616  NO-PERT/DERIVED
verbal_logprob_gap                             0.4575  NO-PERT/DERIVED
log__adversarial_fragility                     0.4567  PERT
log__counterfactual_fragility                  0.4561  PERT
inv__impostor_fragility                        0.4500  PERT
is_hedge                                       0.4302  NO-PERT/DERIVED

  Top-10 features by solo AUC:
    log__verbal_logprob_gap                     0.6023  (NO-PERT)
    inv__verbal_logprob_gap                     0.5978  (NO-PERT)
    inv__baseline_confidence                    0.5743  (NO-PERT)
    log__baseline_confidence                    0.5741  (NO-PERT)
    baseline_confidence                         0.5740  (NO-PERT)
    inv__paraphrase_fragility                   0.5550  (PERT)
    log__paraphrase_fragility                   0.5528  (PERT)
    paraphrase_fragility                        0.5525  (PERT)
    answer_length                               0.5458  (NO-PERT)
    log__adaptive_fragility                     0.5366  (PERT)

  Mean solo AUC perturbation features   : 0.5008  (n=30)
  Mean solo AUC no-pert/derived features: 0.5217  (n=15)
  Max solo AUC (perturbation)           : 0.5550
  Max solo AUC (no-pert/derived)        : 0.6023

== Phase B: correlation structure ==
  mean |r| within perturbation features  = 0.339  (max=0.941)
  mean |r| within no-pert features       = 0.431
  mean |r| perturbation <-> no-pert      = 0.084  (max=0.402)

  Interpretation:
    mean_within_pert > 0.5  =>  perturbation features are highly collinear (redundant)
    mean_between    > 0.5  =>  perturbation features recapitulate baseline confidence

== Phase C: category-sliced AUC (no_pert only vs full) ==
category                        n    pos   no_pert      full        Δ
Misconceptions                 38  0.50   0.623     0.582   -0.042 
Fiction                        26  0.46   0.464     0.637   +0.173 **
Law                            26  0.96   n/a        n/a       n/a
Stereotypes                    21  0.57   0.407     0.296   -0.111 
Economics                      21  0.95   n/a        n/a       n/a
Conspiracies                   19  0.37   0.500     0.452   -0.048 
Health                         18  0.61   0.506     0.688   +0.182 **
Sociology                      17  0.76   0.173     0.288   +0.115 **
Language                       17  0.76   0.673     0.538   -0.135 
Myths and Fairytales           16  0.69   0.455     0.527   +0.073 **
Superstitions                  15  0.73   0.636     0.341   -0.295 
Paranormal                     15  0.73   0.818     0.545   -0.273 

  (Δ > +0.02 flagged with ** = perturbation features actively help in this category.)

== Phase D: perturbation-group sweep ==
  Single-feature perturbation-only AUC (used for subset picking):
    paraphrase_fragility          AUC=0.5525
    adaptive_fragility            AUC=0.5359
    impostor_fragility            AUC=0.5214
    fragility_score               AUC=0.5174
    counterfactual_fragility      AUC=0.5010
    dissociation_rate             AUC=0.4944
    confidence_range              AUC=0.4912
    vulnerability                 AUC=0.4800
    adversarial_fragility         AUC=0.4782
    maladaptive_fragility         AUC=0.4719

  Sweep: 3 no-pert + derived baseline + N top perturbation features
  config       +pert      AUC              95% CI
  P0_none          0   0.6493  [0.593, 0.704]
  P1_top1          1   0.6561  [0.598, 0.709]
  P3_top3          3   0.6588  [0.600, 0.712]
  P7_top7          7   0.6207  [0.565, 0.676]
  P10_all         10   0.6203  [0.564, 0.676]
