# Wonder: Knowledge Graph Properties and Exploitable Structure

## Query
What are the properties of the knowledge graph that we can exploit? It's non-Euclidean, is it a vector space? What kind of graph is it? What kind of properties does this graph have that we can exploit? What instruments do we have in place already vs what can we modify or add?

## Hypothesis
The agentmemory knowledge graph has exploitable mathematical structure beyond what the current Jaccard-based relationship detector uses. Specifically: (1) HRR's frequency-space representation enables asymmetric similarity (containment) naturally, (2) different regions of the graph may have different geometry requiring different metrics, and (3) the existing instruments are only using a fraction of the graph's potential.

## Methodology
- 4 parallel research subagents: graph math characterization, instrument inventory, graph theory literature, hypothesis generation
- 4 empirical tests: composite scoring (085b), metric geometry (085c), HRR correlation (085d), layer comparison
- Production database analysis (15,792 active beliefs, 24,484 semantic edges, 78,941 structural edges)

## Prior Beliefs (from agentmemory)
- HRR architecture: FTS5 (92% keyword) + HRR single-hop (8% vocabulary bridge) - validated Exp 24-34
- Graph has typed edges with Bayesian alpha/beta scoring on both nodes and edges
- 40 intention clusters exist but are too coarse (max=4,608 beliefs per cluster)
- graph_edges table is 86% orphaned (phantom scanner IDs)
- Relationship detector uses Jaccard only, skipped during bulk operations

## Mathematical Characterization

### The Vector Space
HRR operates in **R^2048** with circular convolution as the binding operation. This is a **commutative ring** over the reals:
- Binding = circular convolution via FFT: `ifft(fft(a) * fft(b))`
- Unbinding = circular correlation: `ifft(conj(fft(key)) * fft(composite))`
- Bundling = vector addition (superposition)
- Recovery = cosine similarity against cleanup memory (k-NN)

Capacity: dim // 9 = ~227 edges per partition before interference.

### The Graph
- **Type**: Directed, multi-typed, cyclic graph with Bayesian-weighted edges
- **Cycles**: Allowed (CONTRADICTS can create A->B->A). Path-finding uses visited sets.
- **Edge types**: 12+ types with priority weights (CONTRADICTS=2.0 down to WITHIN_SECTION=0.2)
- **Distance**: Multiplicative confidence decay: `path_conf = prod(belief_conf * edge_conf)` per hop

### The Non-Euclidean Structure
The graph is NOT a vector space in the traditional sense. It has:
- **Hyperbolic-like hierarchy**: SUPERSEDES chains create tree-like temporal structure (negative curvature)
- **Spherical-like clusters**: Same-topic beliefs form dense cliques (positive curvature)
- **Mixed curvature**: Different regions have different local geometry (product manifold)
- The Bayesian confidence creates a METRIC on paths that is non-Euclidean: multiplicative decay means triangle inequality fails (short path through high-confidence hub can beat direct low-confidence edge)

## New Findings

### Finding 1: Jaccard and FTS5 are the same axis measured twice
Adding cluster/HRR layers to candidate discovery doesn't help because Jaccard can't score what they find. All four strategies (FTS5, FTS5+clusters, FTS5+HRR, FTS5+all) produce identical edges. The layers that find beliefs with different vocabulary hand those candidates to a scorer that requires shared vocabulary.

### Finding 2: Different belief types have different geometry
| Type pair | Jaccard | Overlap | Gap |
|-----------|---------|---------|-----|
| factual+requirement | 0.103 | 0.302 | +0.199 |
| requirement (same) | 0.108 | 0.289 | +0.181 |
| factual (same) | 0.106 | 0.256 | +0.150 |
| correction (same) | 0.074 | 0.192 | +0.118 |

Requirements have the highest overlap-to-Jaccard gap -- they're long, detailed specs that short factual beliefs partially contain. Corrections are naturally isolated (unique corrective language).

### Finding 3: HRR correlation correctly identifies term presence
HRR's circular correlation (unbind) can detect which specific terms from belief A are present in belief B without explicit set operations:
- Shared terms recover at 0.22-0.49 cosine
- Non-shared terms recover at < 0.02
- Clean signal separation with no false positives in tested samples

This means the HRR vector space already encodes containment information -- we just aren't using it for edge detection.

### Finding 4: Composite scoring (cluster + low Jaccard) reaches 66-77% of orphans
- R2 (same cluster + Jaccard >= 0.10): 66% of orphans connected
- Union of all composite rules: 77% connected
- But cluster quality is low (40 clusters too coarse, max=4,608)

### Finding 5: Performance characteristics
| Operation | Time per 1000 comparisons |
|-----------|--------------------------|
| Jaccard (set ops) | 0.5ms |
| HRR symmetric (pre-encoded cosine) | 3.6ms |
| HRR asymmetric (per-term correlation) | 43ms |
| Cosine-TF | ~1ms |

HRR asymmetric is 86x slower but gives per-term attribution. Encoding is one-time cost (0.1ms/belief).

### Finding 6: graph_edges table is architectural dead weight
- 86.1% of entries reference phantom scanner IDs (not belief IDs)
- get_neighbors()/expand_graph() never queries graph_edges
- No Bayesian scoring columns (alpha/beta) on graph_edges
- Only used by get_all_edge_triples() for HRR encoding (including phantom nodes)

### Finding 7: Dimensional analysis of HRR for this task
| Dim | Signal | Noise | SNR |
|-----|--------|-------|-----|
| 256 | 0.056 | -0.005 | inf (noise negative) |
| 512 | 0.056 | 0.004 | 13.0 |
| 1024 | 0.054 | 0.003 | 20.2 |
| 2048 | 0.052 | 0.007 | 7.8 |
| 4096 | 0.054 | 0.001 | 89.3 |

Signal is stable across dimensions but noise floor drops with dimension. dim=1024 is the sweet spot for cost/benefit. dim=4096 has best SNR but 4x the memory.

## Gaps and Contradictions

1. **Cluster quality**: 40 clusters is provably too coarse, but we have no spectral or hierarchical alternative implemented
2. **HRR chicken-and-egg**: Orphan beliefs have zero HRR neighbors because HRR needs edges to encode. The correlation test (085d) suggests we can encode term-sets directly without graph edges, but this isn't how the production HRRGraph works (it encodes edge triples, not term sets)
3. **Two edge tables**: The split between `edges` (semantic, scored) and `graph_edges` (structural, unscored, mostly orphaned) means graph operations see different data depending on which table they query
4. **Bayesian scoring on edges is theoretically sound but empirically under-exercised**: Only 534/24,484 edges have been scored (alpha != 1.0 or beta != 1.0). The feedback loop exists but rarely fires because traversals are rare.
5. **Graph theory research agent did not return results** (still running at time of write-up)

## Proposed Experiments

### Exp 086: Type-aware metric selection
Replace the single Jaccard threshold with a metric lookup by (source_type, target_type):
- factual+requirement: use overlap coefficient >= 0.35
- factual+factual: use cosine-TF >= 0.20
- correction+anything: use Jaccard >= 0.25 (higher bar, naturally isolated)
Test: Run on all 15,792 beliefs, compare edge count and quality to current uniform Jaccard.

### Exp 087: HRR term-set encoding for orphan bootstrapping
Instead of encoding graph edge triples, encode each belief's term-set as an HRR superposition. Use asymmetric correlation to find containment relationships. This bypasses the chicken-and-egg (no graph edges needed).
Test: Pre-encode all 15,792 beliefs as term-set vectors, build a per-term recovery matrix, threshold at signal > 0.15 for RELATES_TO edges.

### Exp 088: Spectral clustering replacement
Replace the 40 intention clusters with spectral clustering on the existing semantic edge graph (Laplacian eigenvectors). The graph has enough edges (24K) for meaningful spectral decomposition.
Test: Compare cluster quality (intra-cluster edge density, cross-cluster edge sparsity) between current 40-cluster and spectral k for k in [100, 200, 400].

### Exp 089: Ricci curvature for edge prediction
Compute Ollivier-Ricci curvature on the existing edge graph. Negative curvature regions (tree-like, hierarchical) need SUPERSEDES edges. Positive curvature regions (dense, clique-like) need SUPPORTS edges. Curvature=0 regions (grid-like) need RELATES_TO edges.
Test: On a 1000-node subgraph, compute curvature and predict edge type from local topology. Compare to detect_relationships() output.

### Exp 090: Unified edge table with Bayesian scoring
Merge graph_edges into edges table with:
- Only keep entries where BOTH endpoints are real belief IDs (11,009 of 78,941)
- Add alpha/beta columns, initialize at (1.0, 1.0) (uninformative prior)
- Delete orphaned entries (67,932 rows referencing phantom IDs)
Test: After migration, re-run BFS reachability test. Verify expand_graph() now traverses structural edges. Measure: do CALLS/CITES/COMMIT_TOUCHES edges help retrieval?

## Agent Research Results

### Agent 1: Graph Mathematical Characterization

The graph is a **directed, multi-typed, cyclic graph** operating in a **circular convolution ring** (R^2048):
- HRR binding = circular convolution via FFT (commutative ring)
- Cycles are allowed (CONTRADICTS can create A->B->A)
- Distance = multiplicative confidence decay (NOT Euclidean -- probabilistic path cost)
- 4-dimensional uncertainty vector: Beta(a,b) per axis (feasibility, value, cost, dependency)
- 3-tier lock system: none/promoted/user with different evidence thresholds
- Valence propagation: signed multipliers per edge type, 0.5 decay/hop, max 3 hops

### Agent 2: Instrument Inventory (Key Blindnesses)

Critical blind spots in current architecture:
- `expand_graph()` ONLY queries `edges` table, never `graph_edges`
- No active edge pruning (pruned_at column exists, never written)
- `hrr_neighbors` table is precomputed but never queried in production
- `structural_boost` in score_belief is computed but DISABLED
- Multi-hop HRR disabled (Exp 26: single-hop sufficient)
- Edge confidence barely exercised: 534/24,484 edges ever scored
- TEMPORAL_NEXT excluded from consequence paths (temporal reasoning blocked)

### Agent 3: Graph Theory Literature

**Geometry:** Mixed curvature -- hyperbolic backbone (SUPERSEDES chains, document structure) with locally spherical pockets (topic clusters). Measurable via Gromov delta-hyperbolicity.

**Key techniques identified:**
1. **Ollivier-Ricci curvature** (per-edge) -- positive = within community, negative = bridge between communities. Python library `GraphRicciCurvature` is drop-in. Ricci flow gives parameter-free hierarchical community detection (no k to choose).
2. **PathSim** (Sun et al. VLDB 2011) -- meta-path similarity for heterogeneous graphs. Fully interpretable, no embeddings.
3. **Multi-layer spectral Laplacian** -- per-edge-type Laplacian with normalized combination.
4. **Belief Graphs with Reasoning Zones** (arxiv 2510.10042, Oct 2025) -- mathematically proven contractive confidence propagation, "reasoning zones" as internally-consistent subgraphs.

Sources with URLs:
- Ollivier-Ricci: https://github.com/saibalmars/GraphRicciCurvature
- PathSim: https://web.cs.ucla.edu/~yzsun/papers/vldb11_topKSim.pdf
- Belief Graphs: https://arxiv.org/abs/2510.10042
- Spectral clustering: https://arxiv.org/pdf/0711.0189

### Agent 4: Experiment Proposals (8 total, prioritized)

1. **EXP-102** (Edge credal gap): >80% of edges adjacent to orphans are at default prior. Graph can self-diagnose sparsity via Bayesian scoring state.
2. **EXP-101** (Sub-cluster structure): Top-5 clusters contain meaningful sub-structure at k=sqrt(n). Orphan connectivity jumps from 66% to >80%.
3. **EXP-105** (Degree distribution): Power law exponent 2.0-3.0. Orphans are disproportionately recent + agent-inferred. Anti-preferential-attachment edge creation.
4. **EXP-103** (Multi-metric composite): 2-of-5 voting rule connects >75% of orphans.
5. **EXP-104** (Temporal locality): Same-session beliefs have 2-4x higher base rate of relatedness. Dual-threshold (session-aware) is cheapest new signal.
6. **EXP-106** (Entity IDF filtering): Filter entities by IDF -- rare entities (IDF > median) = genuine co-reference.
7. **EXP-099** (Hyperbolicity): delta < 2 on structural, > 5 on semantic. Validates non-Euclidean embedding approach.
8. **EXP-100** (Ricci curvature): Bimodal distribution, positive = community, negative = bridge. Parameter-free community detection.

## Key Insight

The fundamental architectural gap is: **the graph has a rich algebraic structure (HRR ring + Bayesian manifold + typed heterogeneous edges) but the relationship detector uses none of it.** The detector operates in discrete set-land (Jaccard on term sets) while the graph lives in continuous vector-land (HRR in R^2048 with confidence-weighted paths).

The fix isn't "better Jaccard thresholds" -- it's using the space the graph already lives in for edge detection. Encode beliefs as HRR term-set vectors, use correlation for asymmetric matching, use the existing graph structure (where it exists) to propagate evidence to regions where it doesn't.

## Immediate Action Items

1. **Ollivier-Ricci curvature** -- drop-in library, replaces k=40 clustering with parameter-free communities, gives per-edge geometry labels
2. **Type-conditioned metric dispatch** -- formalize the experimental evidence into a lookup table
3. **Credal gap diagnostic** (EXP-102) -- cheapest test, validates self-diagnosis
4. **HRR term-set encoding** (EXP-087) -- bootstraps orphan connectivity in the graph's native space
