# `mgf-common` — library overview

The shared ground every component doc in this directory builds
on. Read this once; the per-component docs assume it.

`mgf-common` is the cornerstone library for the MGF project
family. **One typed, opt-in observability + configuration stack**
— typed exceptions, central configuration, structured logging,
OpenTelemetry tracing, credential vault, CLI helpers — so every
project in the family inherits the same shape instead of
re-inventing it.

The library's competitive advantage is the **closed-box
interface**: consumers get full power without bypassing to
underlying layers (`pydantic`, `opentelemetry`, stdlib
`logging`). Every capability is exposed through `mgf.common.*`;
consumers should almost never need `import pydantic` or
`import opentelemetry` for normal use.

---

## Install

```bash
# Core — slim install, no optional sinks (pulls only pydantic +
# pydantic-settings + pyyaml — three deps total).
pip install mgf-common

# Tier C observability — OpenTelemetry SDK + Sentry SDK +
# httpx instrumentation.
pip install 'mgf-common[observability]'

# Vault backends (system keyring + encrypted file).
pip install 'mgf-common[vault]'

# Service-shape adapters — pick the ones your stack uses.
pip install 'mgf-common[http]' # typed async httpx wrapper
pip install 'mgf-common[fastapi]' # lifespan + middleware + Depends helpers
pip install 'mgf-common[db]' # async SQLAlchemy 2.x
pip install 'mgf-common[alembic]' # alembic env.py one-liner
pip install 'mgf-common[django]' # AppConfig + middleware + JsonFormatter

# Combine as needed:
pip install 'mgf-common[observability,fastapi,db,alembic]'
```

Requires **Python 3.11+**. No system-package dependencies —
installs cleanly into any virtualenv.

Every adapter is **opt-in via extra**. Bare `pip install mgf-common`
adds three transitive deps. The library never reaches past its public
surface — `import pydantic` / `import opentelemetry` / `import
fastapi` from consumer code is the rare exception, not the rule.

---

## Package layout

```
mgf.common
│
├── Core (no extras required)
│ ├── (top-level) bootstrap, AppContext, current_context,
│ │ app_name, app_version, bootstrap_for_test
│ ├── exceptions typed hierarchy: AppError + 7 categories + 7 generics
│ ├── config BaseAppSettings + atomic load/save + path helpers
│ ├── settings MgfSettings: the library's own typed config
│ ├── observability logging + OTel + Redactor + JsonFormatter +
│ │ AsyncJsonHandler + crash reporter + excepthooks
│ ├── typing 4 runtime_checkable Protocols
│ ├── progress CancellationToken + ProgressReporter (+ asyncio sibling)
│ ├── vault EnvVault / KeyringVault / EncryptedFileVault + registry
│ ├── cli exit codes + translator + subcommand registry +
│ │ diagnostic CLI (explain / validate / doctor /
│ │ migrate / bench)
│ ├── providers ProviderABC + @translate_at_seam ()
│ ├── pytest_plugin auto-registered fixtures (mgf_settings, mgf_context,
│ │ redaction_recorder, …)
│ └── registry generic Registry[F] primitive (extension point)
│
└── Service-shape adapters (opt-in extras)
    ├── http              [http]    AsyncHttpClient + RetryPolicy ()
    ├── fastapi           [fastapi] lifespan + middleware + Depends ()
    ├── db                [db]      async SQLAlchemy + tenant_session ()
    ├── alembic           [alembic] env.py one-liner ()
    └── django            [django]  AppConfig + middleware + JsonFormatter ()
```

Every adapter under "Service-shape" raises `ImportError` at
import time when its extra isn't installed — the message names
the missing extra so the fix is obvious. The closed-box rule
applies uniformly: consumers MUST import from
`mgf.common.fastapi.X`, never reach into `mgf.common.fastapi._foo`.

The library is a [PEP 420 namespace
package](https://peps.python.org/pep-0420/): no `__init__.py` at
`src/mgf/`, so federation siblings (e.g., `mgf.apiprobe`) can
contribute their own subpackages under the same `mgf.*` umbrella
without touching this leaf.

---

## Three integration shapes — pick yours

The most common decision when adopting the library: how do I
integrate? Three shapes, distinct postures.

```
Are you building a top-level application
(an executable or a service)?
│
├─ Yes ─► §"I'm an app" — bootstrap()
│
└─ No
   │
   Are you a library that other apps depend on?
   │
   ├─ Yes ─► §"I'm a library"  — get_logger / operation_span; don't bootstrap
   │
   └─ No (you're writing tests)
      │
      └─►  §"I'm a test"       — bootstrap_for_test()
```

### §"I'm an app"

Top-level executable / service. You own the process lifecycle.

```python
import mgf.common

def main() -> int:
    with mgf.common.bootstrap() as ctx:
        # ... your app's logic
        return 0

if __name__ == "__main__":
    raise SystemExit(main())
```

`bootstrap()` with no args auto-derives identity from the
launching package's distribution metadata (works for `python -m
mypackage` and console-script entries). Pass identity explicitly
for ad-hoc scripts:

```python
with mgf.common.bootstrap(app_name="myapp", app_version="1.0") as ctx:
    ...
```

For more shape, see [`01_lifecycle.md`](01_lifecycle.md).

### §"I'm a library"

Your package is imported INTO an app process; you don't own the
process. **Don't** call `bootstrap()` — the host owns identity /
logging / OTel wiring.

```python
from mgf.common.observability import get_logger, operation_span

LOG = get_logger(__name__)

def fetch_user(user_id: str) -> dict:
    with operation_span("my_library.fetch_user", user_id=user_id):
        LOG.info("fetching user %s", user_id)
        # ... real work ...
```

`operation_span` is no-op-safe when no host has bootstrapped;
your library doesn't crash and doesn't reach past the public
surface. See [`05_observability.md`](05_observability.md).

### §"I'm a test"

```python
import pytest
from mgf.common import bootstrap_for_test, AppContext

@pytest.fixture
def app_ctx() -> AppContext:
    with bootstrap_for_test() as ctx:
        yield ctx

def test_my_thing(app_ctx: AppContext) -> None:
    # ... ctx.settings is MgfSettings.testing() ...
    ...
```

`bootstrap_for_test()` uses test-friendly defaults (WARNING-level
logging, OTel off, deterministic preset). Pass
`capture_logs=True` / `capture_spans=True` to assert on emitted
records. See [`01_lifecycle.md`](01_lifecycle.md) §"Test bootstrap".

---

## Configuration sources (precedence)

`MgfSettings` reads from five sources, highest precedence first:

1. **Explicit constructor kwargs** — `MgfSettings(logging=...)`
2. **`MGF_*` environment variables** — both
   `MGF_LOGGING_LEVEL=DEBUG` (single underscore, +) and
   `MGF_LOGGING__LEVEL=DEBUG` (double, legacy) work
3. **`[tool.mgf]` in `pyproject.toml`** — walks up from CWD; kill
   switch `MGF_DISABLE_PYPROJECT=1`
4. **`.env` file** — if explicitly configured
5. **Field defaults** — declared on the sub-models

The `preset` field (`MgfSettings.preset`) layers on top of all
these as a final pass — preset values fill in fields no source
supplied. See [`04_settings.md`](04_settings.md).

---

## Best practices

- **Bootstrap once per entry point.** App-level code calls
  `bootstrap()` exactly once. Library-level code never bootstraps.
- **Compose, don't subclass `MgfSettings`.** Your app's settings
  carry an `mgf: MgfSettings` field. Pattern in
  [`04_settings.md`](04_settings.md) §"Composing with your app".
- **Subclass typed exceptions**, don't invent ad-hoc strings.
  Pattern in [`02_exceptions.md`](02_exceptions.md).
- **Use `operation_span` even when OTel is disabled.** It's
  no-op-safe; the spans get exported when OTel is enabled later
  without code changes.
- **Tests MUST be deterministic.** `bootstrap_for_test()` +
  `MGF_DISABLE_PYPROJECT=1` autouse fixture keeps host-shell env
  out of test runs.
- **Don't reach into `mgf.common._*` modules.** Underscore-prefix
  is private; the closed-box interface is the contract. The
  diagnostic CLI's `mgf-common doctor` prints a deprecation hint
  if any -shape API leaks back in.

---

## Where to look next

- **Component-level how-to:** [`01_lifecycle.md`](01_lifecycle.md)
  through [`10_full_app.md`](10_full_app.md) in this directory.
- **Recipes directory** (task-shaped how-to):
  [`recipes/`](recipes/) — one file per integration shape:
  - [`recipes/cli.md`](recipes/cli.md) — building CLIs.
  - [`recipes/script.md`](recipes/script.md) — one-off scripts.
  - [`recipes/testing.md`](recipes/testing.md) — pytest fixtures.
  - [`recipes/http.md`](recipes/http.md) — `AsyncHttpClient` + migration recipe.
  - [`recipes/fastapi.md`](recipes/fastapi.md) — FastAPI services.
  - [`recipes/db.md`](recipes/db.md) — async SQLAlchemy + RLS.
  - [`recipes/alembic.md`](recipes/alembic.md) — env.py one-liner.
  - [`recipes/django.md`](recipes/django.md) — Django integration.
  - [`recipes/providers.md`](recipes/providers.md) — provider seams.
  - [`recipes/multi_tenant_settings.md`](recipes/multi_tenant_settings.md) — per-tenant config (recipe-as-primitive; promotion criteria in [`docs/internal/LIFT_CHECKLIST.md`](../internal/LIFT_CHECKLIST.md)).
- **Public-API contract:** [`PUBLIC_API.md`](../../PUBLIC_API.md)
  — every public name with stability tier.
- **Engineering standards** (cross-project):
  [`docs/standards/`](../standards/) — what every consuming
  project commits to.
- **Performance:**
  [`docs/internal/BENCHMARKS.md`](../internal/BENCHMARKS.md) —
  microbench numbers tracked across releases. Reproduce with
  `mgf-common bench`.
- **Migrating from ?** +
  `mgf-common migrate` codemod.
- **Reference consumer:**
  [`example_app/`](../../example_app/) — a real packaged
  consumer demonstrating composition + custom CLI subcommand +
  `bootstrap_for_test`.
- **Layered runnable examples:**
  [`examples/`](../../examples/) — six topics × three layers
  (simple → practical → advanced); the smoke test at
  `tests/unit/test_examples.py` keeps them from drifting.

---

## Appendix — cross-reference graph

The reference graph between docs in this directory plus the
cross-cutting docs they reach out to. Single source of truth;
update this table when adding a link rather than per-doc
appendix tables (which rot).

| Doc | References (out) | Referenced by (in) |
|---|---|---|
| **library_common.md** (this) | every per-component doc; PUBLIC_API.md; standards/; MIGRATION_0.1_TO_0.3.md; example_app/; examples/ | every per-component doc (as the "common ground") |
| **01_lifecycle.md** | library_common; 04_settings; 05_observability; 10_full_app | 02–10 (assumes lifecycle for any code that bootstraps) |
| **02_exceptions.md** | library_common; PUBLIC_API.md; standards/ERROR_HANDLING.md | 01_lifecycle (`BootstrapError`); 06_crash_reporting; 09_cli (exit-code map) |
| **03_config.md** | library_common; 04_settings; standards/CONFIGURATION.md | 04_settings (BaseAppSettings is the base) |
| **04_settings.md** | library_common; 03_config; 01_lifecycle; standards/CONFIGURATION.md; examples/02_config | 01_lifecycle (`bootstrap(settings=...)`); 05_observability (LoggingSettings); 07_vault (VaultSettings); 09_cli (`mgf-common explain`) |
| **05_observability.md** | library_common; 04_settings; 02_exceptions; standards/LOGGING.md; standards/DESIGN_PRINCIPLES.md (DP-12); examples/03_logging; examples/04_observability | 06_crash_reporting; 10_full_app |
| **06_crash_reporting.md** | library_common; 02_exceptions; 05_observability; 09_cli (registry pattern); examples/05_crash_reporting | 01_lifecycle (excepthooks); 10_full_app |
| **07_vault.md** | library_common; 04_settings; standards/SECURITY_STANDARD.md (SC-01, SC-13, SC-17) | 04_settings (`VaultSettings`) |
| **08_progress.md** | library_common; 02_exceptions (`CancellationError`); standards/DESIGN_PRINCIPLES.md (DP-09, DP-11) | 09_cli (translator handles `CancellationError`) |
| **09_cli.md** | library_common; 02_exceptions; 04_settings (`mgf-common explain` reads MgfSettings); examples (none yet) | 10_full_app (custom subcommands) |
| **10_full_app.md** | every other doc + example_app/ + examples/06_full_app | — (terminal node; the integration) |

External (not in this dir):
- [`PUBLIC_API.md`](../../PUBLIC_API.md) — API contract; every component doc links to relevant rows here.
- [`docs/standards/`](../standards/) — engineering bar; component docs reference the relevant topic doc.
- [`docs/claude/CLAUDE.md`](../claude/CLAUDE.md) — Claude session rules; not user-facing.
- — version-upgrade recipe.
- [`docs/internal/LIFT_CHECKLIST.md`](../internal/LIFT_CHECKLIST.md) — contributor recipe for adding components.
