# Architecture — `mgf-common` for new library devs

> **Audience:** new contributors to `mgf-common` itself (not
> consumers). Read [`docs/public/library_common.md`](../public/library_common.md)
> first if you're not sure which.

This doc answers "how is this library structured, and why?" so
new devs get oriented in 30 minutes instead of 3 days.

## The two-line summary

`mgf-common` is **one closed-box public surface** under
`mgf.common.*` exposing **one transactional bootstrap** that
wires identity + structured logging + OTel + crash reporter +
typed-error translator + (optionally) framework adapters.

Everything else is in service of those two ideas.

## The closed-box rule (DP-13)

The library's competitive advantage is that consumers get full
power without bypassing to underlying layers.

**Rule:** every public capability lives at `mgf.common.<x>`,
never at `mgf.common.<x>._<y>` or by reaching into a transitive
dep (`pydantic`, `httpx`, `opentelemetry`, `fastapi`, `django`,
`alembic`, `sqlalchemy`).

**Enforcement:**
- Underscore-prefixed modules (`_lifecycle`, `_apps`, `_engine`)
  are private. Consumers MUST NOT import from them.
- The pin test
  [`tests/unit/test_public_api_doc.py`](../../tests/unit/test_public_api_doc.py)
  asserts every name in `__all__` has a row in `PUBLIC_API.md`.
- The pin test
  [`tests/unit/test_public_api_json_pin.py`](../../tests/unit/test_public_api_json_pin.py)
  asserts the JSON contract matches the live `__all__`.
- Import-linter (`.importlinter`) ensures `mgf.common` doesn't
  reach into consumer or sibling projects.

When you add a new public name: add it to `__all__`, add a row
to `PUBLIC_API.md`, run `python tools/generate_public_api_json.py`.
The two pin tests fail loudly when you forget.

## Package shape

```
mgf.common/ — namespace package (PEP 420; no __init__.py at mgf/)
├── __init__.py — top-level: bootstrap, AppContext, current_context, …
├── _lifecycle.py — bootstrap() implementation (private)
├── _identity.py — UnconfiguredWarning + identity helpers
├── _request_id.py — shared request-id contextvar (used by .fastapi + .django)
├── _errors.py — internal helpers: PEP 678 friendly notes, ValidationError wrap
│
├── exceptions.py — typed hierarchy (1 base + 7 categories + 7 generics)
├── config/ — BaseAppSettings + atomic load/save + path helpers
├── settings.py — MgfSettings (the library's own typed config)
├── observability/ — logging + OTel + Redactor + JsonFormatter + crash reporter
├── typing.py — runtime_checkable Protocols
├── progress/ — CancellationToken + ProgressReporter
├── vault/ — credential storage (env / keyring / encrypted file) + registry
├── cli/ — exit codes + translator + subcommand registry + diagnostic CLI
├── providers/ — ProviderABC + @translate_at_seam ()
├── pytest_plugin.py — auto-registered fixtures (entry-point pytest11)
├── registry.py — generic Registry[F]
│
├── http/ — [http] extra — AsyncHttpClient + RetryPolicy
├── fastapi/ — [fastapi] extra — lifespan + middleware + Depends
├── db/ — [db] extra — async SQLAlchemy + tenant_session
├── alembic/ — [alembic] extra — env.py one-liner
└── django/ — [django] extra — AppConfig + middleware + JsonFormatter
```

PEP 420 namespace package: `src/mgf/` has NO `__init__.py`, so
sibling packages (`mgf.apiprobe`, future `mgf.cli` etc.) can
contribute their own subpackages under the `mgf.*` umbrella
without touching this leaf.

## The transactional bootstrap

`bootstrap()` wires five things, in order, with rollback on
failure:

1. **Identity** — `app_name` + `app_version` (auto-derived from
   package metadata or passed explicitly).
2. **Logging** — JSON formatter + redaction + handler stack
   (stderr, optional file, optional journald, optional HTTP sink).
3. **OTel** — span processor + exporter (when `[observability]`
   is installed and the URL is configured).
4. **Crash reporter** — sys.excepthook + threading.excepthook +
   sink dispatch.
5. **AppContext registration** — sets the process-global
   `_GLOBAL_CONTEXT` and the per-task contextvar
   `_CONTEXT_VAR`.

If step N fails, steps 1..N-1 roll back in reverse order. The
process is left in the state it was before bootstrap was called.
This is the "one transactional bootstrap" promise.

The implementation is in [`src/mgf/common/_lifecycle.py`](../../src/mgf/common/_lifecycle.py).
Private — consumers call `mgf.common.bootstrap`.

## Settings architecture

`MgfSettings` is the library's typed config root (subclass of
`BaseAppSettings`, a `pydantic_settings.BaseSettings` subclass
with the project's load/save/atomicity contract baked in).

Five sources, highest precedence first:

1. Explicit constructor kwargs (`MgfSettings(logging=...)`).
2. `MGF_*` env vars.
3. `[tool.mgf]` in `pyproject.toml` (walks up from CWD).
4. `.env` file (if configured).
5. Field defaults.

`MgfSettings.preset` (`"production"` / `"testing"` /
`"development"` / `"explicit"`) layers on top as a final pass —
preset values fill fields no source supplied.

Consumers compose by carrying an `mgf: MgfSettings` field on
their own `BaseAppSettings` subclass; mgf-common never owns the
top-level settings node.

## Federation pattern

`mgf-common` is a **federation leaf** — it never depends on
consumer or sibling projects. The closed-box invariant is enforced
by import-linter:

```ini
# .importlinter (project root)
[importlinter]
root_packages = mgf

[importlinter:contract:mgf-common-is-the-federation-leaf]
name = mgf.common does not import any consumer or sibling project
type = forbidden
source_modules = mgf.common
forbidden_modules = vmanager, plasmamapper, inthewords, mgf.apiprobe
```

Sibling packages (e.g., `mgf-apiprobe`) live in their own repo,
with their own release cadence, and depend on `mgf-common` as a
runtime dep. Cross-sibling imports are forbidden — every dep
chain terminates at `mgf-common`.

## Testing strategy

**Unit tests** live in `tests/unit/<module>/` mirroring `src/mgf/common/<module>/`.
Each public surface has positive + negative cases.

**Four green gates** before any commit (per `CONTRIBUTING.md`):

1. `coverage run -m pytest tests/` (≥80%)
2. `mypy --strict src/`
3. `ruff check src tests`
4. `lint-imports`

The gates are non-negotiable. A red gate is a commit blocker.

**Test isolation:** the pytest plugin (`mgf.common.pytest_plugin`)
ships fixtures that bootstrap test state and tear it down between
cases. Consumer projects inherit the same fixtures by installing
mgf-common — no duplication.

A common gotcha: tests that call `bootstrap()` MUST clean up the
process-global context, or sibling tests inherit stale identity.
The pytest plugin handles this for fixture-driven tests; manual
`bootstrap()` calls in test functions need explicit teardown
(see `tests/unit/cli/test_bench.py` for an autouse-fixture
example).

## Adding a new public surface

The mechanical 8-step recipe lives in
[`docs/internal/LIFT_CHECKLIST.md`](LIFT_CHECKLIST.md). Don't reinvent it.

The summary:
1. Extract a `Protocol` first if applicable.
2. Implement under `mgf.common.<x>/_impl.py` (private).
3. Re-export from `mgf.common.<x>/__init__.py` (`__all__`).
4. Add a `PUBLIC_API.md` row.
5. Update `tools/generate_public_api_json.py`'s `PUBLIC_MODULES`
   if you added a new subpackage.
6. Add `[<x>]` and `[<x>-test]` extras to `pyproject.toml` if
   the surface needs an optional dep.
7. Tests under `tests/unit/<x>/`.
8. Recipe at `docs/public/recipes/<x>.md`.

CHANGELOG entry happens at the release commit, not per-feature
commit.

## Versioning + release cadence

Follows SemVer in spirit during 0.x:

- **MAJOR** = breaking change to public surface (deferred to v1.0).
- **MINOR** = new feature; additive only.
- **PATCH** = bugfix or doc-only refresh.

Each release commit:
1. Bumps `pyproject.toml::project.version` + `src/mgf/common/__init__.py::__version__`.
2. Moves CHANGELOG `[Unreleased]` to a dated `[X.Y.Z]` section.
3. Regenerates `PUBLIC_API.json`.
4. All four gates green.
5. Tags `vX.Y.Z`.
6. Manual PyPI upload (Codeberg has no OIDC; uploads are
   maintainer-driven).

The strategic plan is `my_stuff/IMPLEMENTATION_PLAN_v0.5_to_v1.0.md`
(internal, in `my_stuff/` so it's gitignored — the file is the
maintainer's planning notes, not a contract).

## Performance contract

`mgf-common bench` prints microbench numbers for the load-bearing
primitives. Numbers are tracked in
[`docs/internal/BENCHMARKS.md`](BENCHMARKS.md); regressions
> 25% release-over-release call for an explicit justification or
a fix.

Hot-path constraints:
- `current_context()` MUST stay below 1µs (currently ~160ns).
- `operation_span()` open+close MUST stay below 10µs.
- log call (async) MUST stay below 10µs (currently ~6µs).

## Documentation map

```
docs/
├── public/ — consumer-facing
│ ├── library_common.md — overview (read this first)
│ ├── 01_lifecycle.md — bootstrap deep dive
│ ├── 02_exceptions.md — typed hierarchy
│ ├── 03_config.md — BaseAppSettings
│ ├── 04_settings.md — MgfSettings
│ ├── 05_observability.md — logging + OTel + Redactor
│ ├── 06_crash_reporting.md — crash sinks
│ ├── 07_vault.md — credentials
│ ├── 08_progress.md — cancellation/reporter
│ ├── 09_cli.md — diagnostic CLI
│ ├── 10_full_app.md — integration shape (CLI + service)
│ └── recipes/ — task-shaped how-to (one per integration)
│
├── internal/ — maintainer-facing (this directory)
│ ├── ARCHITECTURE.md — this file
│ ├── BENCHMARKS.md — perf tracking
│ ├── CONSUMER_AUDIT_v0.5.md — audit-script output
│ └── v0.5_PHASE_A_DECISIONS.md — phase-decision archive
│
├── standards/ — cross-project engineering standards
├── claude/ — Claude session rules (not user-facing)
└── internal/
    ├── ARCHITECTURE.md — this file
    ├── LIFT_CHECKLIST.md — when/how to lift code into the library
    ├── EXTRACTION_HISTORY.md — original extraction-plan archive (Round 5)
    └── STRATEGIC_REVIEW.md — three-priority working backlog
```

## Where to ask questions

- **API design / surface decisions:**
  [`docs/standards/API_DESIGN.md`](../standards/API_DESIGN.md)
  has the AP-* rules.
- **Error handling:**
  [`docs/standards/ERROR_HANDLING.md`](../standards/ERROR_HANDLING.md).
- **Logging:**
  [`docs/standards/LOGGING.md`](../standards/LOGGING.md).
- **Configuration:**
  [`docs/standards/CONFIGURATION.md`](../standards/CONFIGURATION.md).
- **Security:**
  [`docs/standards/SECURITY_STANDARD.md`](../standards/SECURITY_STANDARD.md).
- **Scope (what's in / out / adjacent):**
  [`docs/standards/SCOPE.md`](../standards/SCOPE.md).

## TL;DR for the first PR

1. Read `library_common.md`.
2. Read `LIFT_CHECKLIST.md` if adding new public surface.
3. Run the four gates locally before pushing.
4. Add a PUBLIC_API row.
5. Add a recipe doc (or update one) if the change is consumer-visible.
