Metadata-Version: 2.4
Name: mimidai
Version: 0.5.0
Summary: Knowledge-graph substrate library (entities, relations, observations, pluggable backends, FTS5 search, rule-based inference, MCP server)
Project-URL: Homepage, https://github.com/dyslexai-cardnl/mimidAI
Project-URL: Issues, https://github.com/dyslexai-cardnl/mimidAI/issues
Author: dyslexai-cardnl
License: MIT
License-File: LICENSE
Requires-Python: >=3.11
Provides-Extra: conformance
Requires-Dist: pytest>=7.0; extra == 'conformance'
Description-Content-Type: text/markdown

# mimidAI

*A knowledge-graph substrate library.*

mimidAI is a Python library providing a pluggable graph-storage layer (entities, relations, observations) with a two-tier type system, FTS5 search, rule-based inference, and an MCP server surface. It was extracted from [corvidAI](https://github.com/dyslexai-cardnl/corvidAI) in Phase 32 (2026-04-23) so that three independent consumers — [corvidAI](https://github.com/dyslexai-cardnl/corvidAI) (research framework), [evilleAI](https://github.com/dyslexai-cardnl/evilleAI) (dev-agent monitoring), and [LaiR](https://github.com/dyslexai-cardnl/LaiR) (storytelling platform) — can share one substrate without depending on one another.

## The three-part substrate

mimidAI is posture-neutral by design (per [ADR-014](docs/architecture/decisions/ADR-014-substrate-ownership-and-data-governance-mobility.md) + [ADR-015](docs/architecture/decisions/ADR-015-three-way-substrate-repo-split.md), both Accepted 2026-05-14): the library encodes no privacy posture. Deployment posture is composed above the library by two sibling deployment-substrate repos.

| Repo | Role | Posture | Phase status |
|---|---|---|---|
| [`mimidAI`](https://github.com/dyslexai-cardnl/mimidAI) (this repo) | Core substrate library: ABCs, business-logic stores, type registry, rule miner, MCP surface, conformance plugin | Neutral | Active |
| [`mimidai-open-deployment`](https://github.com/dyslexai-cardnl/mimidai-open-deployment) | Plaintext-server deployment substrate | Operator can read plaintext | Phase-A bootstrap complete (2026-05-21); Phase B body pending |
| [`mimidai-blind-deployment`](https://github.com/dyslexai-cardnl/mimidai-blind-deployment) | Operator-blind deployment substrate | Operator never sees plaintext | Phase-A bootstrap complete (2026-05-21); Phase C body pending |

The Phase-A capstone landed 2026-05-21 as a cross-repo conformance loop: both deployment-substrates extend [`BaseStubEnvelopeSpec`](mimidai/conformance/stubs.py) and run the pytest-plugin conformance harness on every PR. See [`docs/architecture/decisions/ADR-019-envelope-spec-abc.md`](docs/architecture/decisions/ADR-019-envelope-spec-abc.md) for the ABC + the conformance contract.

Consumers ([corvidAI](https://github.com/dyslexai-cardnl/corvidAI) / [evilleAI](https://github.com/dyslexai-cardnl/evilleAI) / [LaiR](https://github.com/dyslexai-cardnl/LaiR)) depend on mimidAI core for the library surface and on whichever deployment-substrate matches their privacy posture. See [`docs/rfcs/RFC-001-consumer-contract.md`](docs/rfcs/RFC-001-consumer-contract.md) for the per-consumer schema commitments.

## Status

**Phase A complete (2026-05-21).** Three-way substrate split ratified; sibling deployment repos exist with cohort-stub envelope subclasses; cross-repo conformance loop runs green. [ADR-016](docs/architecture/decisions/ADR-016-cross-repo-ci-orchestration.md) governs cross-repo CI orchestration; [ADR-017](docs/architecture/decisions/ADR-017-postgres-age-version-envelope.md) coordinates Postgres+AGE version envelopes; [ADR-018](docs/architecture/decisions/ADR-018-operator-blind-substrate-vs-consumer-disambiguation.md) disambiguates substrate-side vs consumer-side operator-blindness; [ADR-019](docs/architecture/decisions/ADR-019-envelope-spec-abc.md) defines the EnvelopeSpec ABC.

**Library core mature (Phase 28-30 in corvidAI's history):** `GraphBackend` ABC with 14 CRUD methods, SQLite backend with FTS5 search, rule-based inference inspired by AnyBURL, 18-tool MCP server, two-tier type registry. Postgres + Apache AGE is a registered stub pending implementation; the substrate direction is committed in [ADR-012 §Server-Mode Substrate Direction](docs/architecture/decisions/ADR-012-backend-abstraction.md), and the binding promotion gate is carried by [evilleAI ADR-021](https://github.com/dyslexai-cardnl/evilleAI/blob/main/docs/architecture/decisions/ADR-021-mimidai-knowledge-graph-integration.md).

**Upcoming:** Phase B (`mimidai-open-deployment` body: SQLite migration + MCP server move + cross-repo CI fan-out); Phase C (`mimidai-blind-deployment` greenfield); Phase D (Postgres+AGE backend conformance per ADR-017).

See [docs/architecture/decisions/ADR-001-extraction-from-corvidai.md](docs/architecture/decisions/ADR-001-extraction-from-corvidai.md) for the extraction rationale and the design decisions frozen at the spin-out.

## Architecture

- `mimidai/backend_interface.py` — `GraphBackend` ABC (14 CRUD methods + traversal fallback)
- `mimidai/sqlite_backend.py` — SQLite implementation (default)
- `mimidai/backends.py` — registry and `resolve(name, **kwargs)` entry point; `sqlite` active, `postgres` stub
- `mimidai/search_index.py` + `sqlite_search.py` — `SearchIndex` ABC + FTS5 implementation
- `mimidai/entity_store.py` / `relation_store.py` / `observation_store.py` — business logic (validation, auto-ID, provisional types)
- `mimidai/type_registry.py` — two-tier type system (core + provisional) backed by `graph_types.toml`
- `mimidai/rule_miner.py` / `inference.py` — AnyBURL-inspired rule mining and application
- `mimidai/mcp_server.py` — MCP server exposing 18 tools (11 thin-core + 5 sugar + 2 inference)
- `mimidai/extraction/` — PDF + LLM-based extraction transform
- `mimidai/envelope_spec.py` — `EnvelopeSpec` ABC ([ADR-019](docs/architecture/decisions/ADR-019-envelope-spec-abc.md)): authenticated encryption + key non-custody + bounded metadata contract that deployment substrates implement
- `mimidai/conformance/` — pytest-plugin conformance harness + `BaseStubEnvelopeSpec` reference stub (Phase-A capstone). Registered as a `pytest11` entry point via `pyproject.toml`; see [conformance harness](#conformance-harness-for-deployment-substrate-authors) below.
- `mimidai/benchmarks/` — ingestion benchmark harness with SQLite baselines
- `mimidai/archive/kuzu/` — archived Kuzu backend (Phase 30 archival; reactivation criteria in `mimidai/archive/kuzu/ARCHIVE_README.md`). Kept inside the package so `python -m unittest mimidai.archive.kuzu.tests.<module>` works after `pip install`.

## Installation

Not on PyPI yet; install from git:

```bash
pip install git+https://github.com/dyslexai-cardnl/mimidAI.git
```

To include the PDF + Claude API extraction pipeline (`pdfminer.six` + `anthropic`):

```bash
pip install "mimidai[extraction] @ git+https://github.com/dyslexai-cardnl/mimidAI.git"
```

Downstream consumers should pin to a commit SHA or tag rather than tracking `main`, so their installs stay reproducible against future breaking changes in this repo.

## Conformance harness (for deployment-substrate authors)

If you're building a deployment substrate above mimidAI (the existing siblings, or a future cohort), the Phase-A capstone ships a pytest-plugin conformance harness that exercises your `EnvelopeSpec` implementation against the ABC's contract.

Install with the `[conformance]` extra so pytest comes along:

```bash
pip install "mimidai[conformance] @ git+https://github.com/dyslexai-cardnl/mimidAI.git"
```

Point it at your concrete (or stub) implementation:

```bash
python -m pytest --mimidai-conformance \
  --envelope-spec-impl=my_deployment._envelope.MyEnvelopeSpec
```

The plugin auto-collects the packaged conformance suite. Stub implementations that raise `NotImplementedError` on every abstract method (extending `mimidai.conformance.stubs.BaseStubEnvelopeSpec`) cause the suite to SKIP each test rather than FAIL; concrete implementations must satisfy C1-C6 (authenticated encryption, key non-custody, bounded metadata, spec_version semver-shape, plaintext-metadata negative tests). Override only `spec_version` to identify your cohort.

The existing sibling repos use this harness on every PR. See [`docs/architecture/decisions/ADR-019-envelope-spec-abc.md`](docs/architecture/decisions/ADR-019-envelope-spec-abc.md) for the ABC + conformance contract.

## Development

This repo follows shared conventions across the dyslexai-cardnl ecosystem:

- **Test-driven development** is mandatory. Red / green / refactor. Tests land with code.
- **Branch-protected `main`**: all changes via pull request. No direct pushes. Enforce-admins ON.
- **Conventional commits** (`type(scope): description`) with a co-author trailer.
- **Approved licenses**: Apache-2.0, MIT, BSD-2/3-Clause, ISC, CC0-1.0. GPL / AGPL / SSPL / CC-BY-NC blocked.

Runtime dependency: `mcp>=1.0.0`. Otherwise stdlib-only on the SQLite path.

## Ingestion CLI

Phase 1 (this repo) ships a continuous-ingestion CLI installed as `mimidai`:

```
mimidai ingest --source arxiv --query "<q>" --since "<YYYY-MM-DD>"
mimidai ingest --url "<pdf-url>" [--note "<text>"]
mimidai rollback <run_id>            # dry-run preview
mimidai rollback <run_id> --confirm  # hard delete + JSONL marker
mimidai smoke --quick | --full       # end-to-end smoke against real APIs
mimidai canary --reference-fixture <f> --reference-pdf-url <u>
```

Install with the `[ingestion]` extra so the LLM + PDF dependencies come along:

```bash
pip install "mimidai[ingestion] @ git+https://github.com/dyslexai-cardnl/mimidAI.git"
```

`ANTHROPIC_API_KEY` is required at CLI entry. Provenance defaults to `--provenance maximal`; dial down to `recommended` once the pipeline is stable. Roll-back granularity is run-level (D15). See [docs/architecture/decisions/ADR-013-ingestion-architecture.md](docs/architecture/decisions/ADR-013-ingestion-architecture.md) for the full design including the 18 D-decisions and 20 edge cases.

## Naming note

The "mimidAI" name belongs to this knowledge-graph substrate. It was briefly overloaded during a cross-layer rename (2026-04-22): the storytelling-platform project that had been called "mimidAI" was renamed to "LaiR" so that this library could keep the framework-layer bird-family name (Corvidae + Mimidae). ADR file titles inside this repo mostly read "mimidai"; a handful say "lair" because the rename happened mid-phase. No semantic difference.

## Key documents

| Document | Location |
|---|---|
| Consumer contract (per-consumer schema usage + coordination) | [docs/rfcs/RFC-001-consumer-contract.md](docs/rfcs/RFC-001-consumer-contract.md) |
| Extraction rationale | [docs/architecture/decisions/ADR-001-extraction-from-corvidai.md](docs/architecture/decisions/ADR-001-extraction-from-corvidai.md) |
| Original knowledge-graph design | [docs/architecture/decisions/ADR-010-mimidai-knowledge-graph.md](docs/architecture/decisions/ADR-010-mimidai-knowledge-graph.md) |
| Graph storage redesign + inference | [docs/architecture/decisions/ADR-011-mimidai-graph-storage-inference.md](docs/architecture/decisions/ADR-011-mimidai-graph-storage-inference.md) |
| Backend abstraction + server-mode substrate direction | [docs/architecture/decisions/ADR-012-backend-abstraction.md](docs/architecture/decisions/ADR-012-backend-abstraction.md) (see §Server-Mode Substrate Direction) |
| Continuous-ingestion architecture (Phase 1 V1) | [docs/architecture/decisions/ADR-013-ingestion-architecture.md](docs/architecture/decisions/ADR-013-ingestion-architecture.md) |
| Substrate ownership + data-governance mobility stance (**Accepted 2026-05-14**) | [docs/architecture/decisions/ADR-014-substrate-ownership-and-data-governance-mobility.md](docs/architecture/decisions/ADR-014-substrate-ownership-and-data-governance-mobility.md) |
| Three-way substrate repo split (**Accepted 2026-05-14**, phase-3-amended) | [docs/architecture/decisions/ADR-015-three-way-substrate-repo-split.md](docs/architecture/decisions/ADR-015-three-way-substrate-repo-split.md) |
| Cross-repo CI orchestration (**Accepted 2026-05-20**) | [docs/architecture/decisions/ADR-016-cross-repo-ci-orchestration.md](docs/architecture/decisions/ADR-016-cross-repo-ci-orchestration.md) |
| Postgres+AGE version envelope coordination (**Accepted 2026-05-20**) | [docs/architecture/decisions/ADR-017-postgres-age-version-envelope.md](docs/architecture/decisions/ADR-017-postgres-age-version-envelope.md) |
| Operator-blind substrate vs consumer disambiguation (**Accepted 2026-05-20**) | [docs/architecture/decisions/ADR-018-operator-blind-substrate-vs-consumer-disambiguation.md](docs/architecture/decisions/ADR-018-operator-blind-substrate-vs-consumer-disambiguation.md) |
| EnvelopeSpec ABC (**Accepted 2026-05-20**) | [docs/architecture/decisions/ADR-019-envelope-spec-abc.md](docs/architecture/decisions/ADR-019-envelope-spec-abc.md) |
| Phase-A capstone conformance plugin (2026-05-21) | [mimidai/conformance/plugin.py](mimidai/conformance/plugin.py) + [mimidai/conformance/stubs.py](mimidai/conformance/stubs.py) |
| Cross-repo review rounds for ADR-014 + ADR-015 (phase-2 + phase-3) | [docs/architecture/reviews/2026-05-11-substrate-governance-round/](docs/architecture/reviews/2026-05-11-substrate-governance-round/) + [docs/architecture/reviews/2026-05-13-phase-3-destination-round/](docs/architecture/reviews/2026-05-13-phase-3-destination-round/) |
| Downstream integration (external) | evilleAI [`ADR-021-mimidai-knowledge-graph-integration.md`](https://github.com/dyslexai-cardnl/evilleAI/blob/main/docs/architecture/decisions/ADR-021-mimidai-knowledge-graph-integration.md) (carries the binding consumer-review gate) + [`ADR-023-agent-event-provenance-graph.md`](https://github.com/dyslexai-cardnl/evilleAI/blob/main/docs/architecture/decisions/ADR-023-agent-event-provenance-graph.md) (event provenance graph + `eville_prov` subgraph) |

## License

MIT. See [LICENSE](LICENSE).
