# mgf-common

[![PyPI version](https://img.shields.io/pypi/v/mgf-common.svg)](https://pypi.org/project/mgf-common/)
[![Python versions](https://img.shields.io/pypi/pyversions/mgf-common.svg)](https://pypi.org/project/mgf-common/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Typed: PEP 561](https://img.shields.io/badge/typed-PEP%20561-brightgreen.svg)](https://peps.python.org/pep-0561/)

The cornerstone library for the **MGF** project family. One typed, tested,
opt-in observability stack — **typed exceptions, central configuration,
structured logging, OpenTelemetry tracing** — so every project in the
family inherits the same shape instead of re-inventing it.

> **Status:** pre-1.0. The API is "lifted-from-VManager-shaped" and
> may still evolve — see [`FEEDBACK.md`](FEEDBACK.md) for the
> early-consumer feedback pipeline. Reference consumer:
> [VManager](https://codeberg.org/magogi-admin/VManager).
>
> **New here?** → [`STARTHERE.md`](STARTHERE.md) — one-page guide
> for starting a new project or catching up an existing one.

---

## Two things in one repository

This repository serves a dual role; both halves move together.

1. **The library** (`src/mgf/common/`) — battle-tested infrastructure,
   lifted from VManager and packaged for reuse. Ships with a PEP 561
   `py.typed` marker so consumers' `mypy --strict` accepts inline
   annotations. Subpackages, all under one closed-box interface:
   - **Core (no extras required):**
   - `mgf.common.exceptions` — typed exception hierarchy
     (1 base + 7 categories + 7 generic concretes)
   - `mgf.common.config` — `BaseAppSettings` (pydantic-settings) +
     `settings_config` + atomic loader/saver + cross-OS path helpers
   - `mgf.common.observability` — structured logging,
     `operation_span`, `Redactor`, `JsonFormatter`, `AsyncJsonHandler`,
     crash reporter, sys/threading excepthooks (OpenTelemetry deps live
     behind the `[observability]` extra)
   - `mgf.common.settings` — `MgfSettings`: the library's typed config root
   - `mgf.common.typing` — runtime-checkable Protocols
   - `mgf.common.progress` — `CancellationToken` + `ProgressReporter` (+ asyncio sibling)
   - `mgf.common.vault` — credential storage backends (env / keyring / encrypted file)
   - `mgf.common.cli` — exit codes + translator + subcommand registry + diagnostic CLI (`mgf-common`)
   - `mgf.common.providers` — `ProviderABC` + `@translate_at_seam` for SDK seams
   - `mgf.common.registry` — generic `Registry[F]` extension primitive
   - **Service-shape adapters (opt-in extras, all /):**
   - `mgf.common.http` (`[http]`) — typed async HTTP client wrapping `httpx`
   - `mgf.common.fastapi` (`[fastapi]`) — lifespan + middleware + Depends helpers
   - `mgf.common.db` (`[db]`) — async SQLAlchemy + Postgres-RLS multi-tenancy
   - `mgf.common.alembic` (`[alembic]`) — `env.py` boilerplate replaced with one call
   - `mgf.common.django` (`[django]`) — `AppConfig` + middleware + `JsonFormatter` for `LOGGING`

2. **The standards** ([`docs/standards/`](docs/standards/)) — the
   **single source of truth** for the cross-project engineering bar
   every MGF project commits to. Five topic docs covering design
   principles, error handling, logging, configuration, and the
   common-component catalogue, with conformance levels (L0 / L1 / L2)
   every project declares against. Grounded in code that ships in this
   repository, illustrated with real VManager call sites.

---

## Install

```bash
# Default — slim install, no optional sinks pulled in
pip install mgf-common

# Tier C observability — adds OpenTelemetry SDK + Sentry SDK + httpx instrumentation
pip install 'mgf-common[observability]'

# Vault backends (system keyring + encrypted file)
pip install 'mgf-common[vault]'

# Service-shape extras — pick the ones your stack uses
pip install 'mgf-common[http]' # typed async HTTP client
pip install 'mgf-common[fastapi]' # FastAPI integration
pip install 'mgf-common[db]' # async SQLAlchemy
pip install 'mgf-common[alembic]' # alembic env.py helper
pip install 'mgf-common[django]' # Django integration

# Combine as needed:
pip install 'mgf-common[observability,fastapi,db,alembic]'
```

Requires Python **3.11+**. No system-package dependencies — installs
cleanly into any virtualenv. Bare `pip install mgf-common` pulls only
three deps (pydantic, pydantic-settings, pyyaml); every framework
adapter is opt-in.

## Quick start

```python
# my_app/__main__.py
import mgf.common

def main() -> int:
    # bootstrap() is the one-call entry: identity + logging + OTel +
    # crash reporter + excepthooks, transactional with rollback. No
    # args needed — identity auto-derives from your distribution
    # metadata when run via `python -m my_app` or via a console-
    # script entry. Pass app_name/app_version explicitly for ad-hoc
    # scripts and REPLs.
    with mgf.common.bootstrap() as ctx:
        # ... your app's logic
        return 0

if __name__ == "__main__":
    raise SystemExit(main())
```

That's the entire integration for a typical app. To tune
observability, pass an `MgfSettings` (intent-driven factories +
typed sub-models):

```python
from mgf.common import bootstrap
from mgf.common.settings import MgfSettings, OtelSettings

with bootstrap(
    settings=MgfSettings.production(
        otel=OtelSettings(
            enabled=True,
            endpoint="http://otel-collector:4318",
            sample_rate=0.05,
        ),
    ),
) as ctx:
    ...
```

Three factories: `MgfSettings.dev()` / `.production()` / `.testing()`.
For per-deployment config, drop a `[tool.mgf]` section into your
`pyproject.toml` — `MgfSettings` auto-reads it (precedence: explicit
kwargs > `MGF_*` env vars > pyproject > defaults).

**Typed exceptions, typed configuration, typed observability** —
the rest of the library composes from this single entry point.
See [`docs/public/library_common.md`](docs/public/library_common.md) for the
three integration shapes (app / library / test) and
[`docs/public/04_settings.md`](docs/public/04_settings.md) for the
canonical pattern when your app has its own typed settings that
should compose with `MgfSettings`.

## Examples

Eighteen runnable examples under [`examples/`](examples/) — six
topics, three layers each (simple → practical → advanced):

```
examples/
├── 01_exceptions/ typed hierarchy + subclassing + translation seam
├── 02_config/ BaseAppSettings + make_settings_config + load/save
├── 03_logging/ setup_logging + Redactor + structured fields
├── 04_observability/ setup_otel + operation_span + trace correlation
├── 05_crash_reporting/ report_now + sys/threading excepthooks
└── 06_full_app/ a complete tiny CLI tying everything together
```

Every script is self-contained (uses `tempfile.TemporaryDirectory`
for disk side-effects), runs in under a second, and is
smoke-tested in CI so the examples never drift from the API. Run
any of them with `python examples/<topic>/<layer>.py` or browse
[`examples/README.md`](examples/README.md) for the index.

---

## Standards

The [`docs/standards/`](docs/standards/) folder is the cross-project
engineering bar. Every MGF project declares its conformance level
against it.

| Doc | What it covers |
|---|---|
| [`README.md`](docs/standards/README.md) | Master index, [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119) keywords, L0/L1/L2 levels, sortable index of every MUST |
| [`DESIGN_PRINCIPLES.md`](docs/standards/DESIGN_PRINCIPLES.md) | Layering (DP-01..13), provider ABC pattern, frozen domain types, GUI threading, async + time + resource discipline, mechanical enforcement matrix |
| [`ERROR_HANDLING.md`](docs/standards/ERROR_HANDLING.md) | Exception hierarchy (EH-01..10), translation seams, top-level handlers, crash reporter, async error handling, error-message style guide |
| [`LOGGING.md`](docs/standards/LOGGING.md) | Stdlib + JSON, OTel correlation, redaction, sinks (LG-01..14), volume control (rate-limit / sample / dedup), contextvars correlation |
| [`CONFIGURATION.md`](docs/standards/CONFIGURATION.md) | pydantic-settings, layered precedence, schema versioning, atomic writes (CF-01..12), discriminated unions, env-tier layering, alias renames |
| [`SECURITY.md`](docs/standards/SECURITY_STANDARD.md) | Cross-project security standard (SC-01..18): secrets, subprocess, deserialization, crypto, TLS, input validation, dependency scanning, vulnerability response, threat model template |
| [`TESTING.md`](docs/standards/TESTING.md) | Cross-project testing standard (TS-01..18): layout, fixtures, mock provider, property tests, GUI / async testing, performance benchmarks, security pins |
| [`API_DESIGN.md`](docs/standards/API_DESIGN.md) | Cross-project API-design standard (AP-01..15): `__all__` discipline, semver rigor, deprecation cycle, type stability, stability tiers, `PUBLIC_API.md` contract, CHANGELOG migration recipes |
| [`COMMON_COMPONENTS.md`](docs/standards/COMMON_COMPONENTS.md) | Catalogue of every common component + lift instructions + cornerstone-rhythm guidance |

**Standards version:** v2.0 (released 2026-04-25 — introduced
[`SECURITY.md`](docs/standards/SECURITY_STANDARD.md),
[`TESTING.md`](docs/standards/TESTING.md), and
[`API_DESIGN.md`](docs/standards/API_DESIGN.md), plus new MUSTs
across the existing docs; see
[What's new in v2.0](docs/standards/README.md#whats-new-in-v20)).
**Versioning policy:** MAJOR for a MUST change, MINOR for
SHOULD/template change, PATCH for wording. Independent of the
package version.

## Conformance levels

Every project that adopts these standards declares its target level in
its top-level README:

| Level | What you commit to | Typical use |
|---|---|---|
| **L0 — Bootstrap** | Hierarchical logging, base exception class, env-driven config (no schema). MUST NOT log secrets. MUST NOT swallow exceptions silently. | Spike, prototype, throwaway tool |
| **L1 — Production-Ready** | Full typed exception hierarchy, JSON-capable structured logging with correlation IDs, `pydantic-settings` config with schema versioning, top-level error handlers wired (CLI + GUI/threading/asyncio as applicable), local crash reporter. | Any tool a real user will run |
| **L2 — Observability-Native** | Everything in L1 + OpenTelemetry traces/metrics/logs (OTLP exporter, off by default but flip-able via env), optional Sentry/GlitchTip handler, log redaction on by default, GUI log viewer dialog, remote log shipping handler available. | Any tool that runs unattended, in production, on a user's machine, or as a service |

`mgf-common` ships at **L2**. Consumers that depend on it are L1 by
default; they reach L2 by installing the `[observability]` extra and
calling `mgf.common.observability.setup_otel()`.

---

## Quality gates

| Gate | Today | Tooling |
|---|---|---|
| Tests | **176 passing** in <1.5s on a warm machine; no external services, no network. 19 of those are example-smoke tests verifying every script in `examples/` runs cleanly | `pytest` (see `[tool.pytest.ini_options]`) |
| Type checking | `mypy --strict` clean on 18 source files | `mypy` with the `[dev,observability]` install |
| Lint | clean | `ruff check src tests` |
| Layering | `mgf-common-is-a-leaf` contract enforced (forbids any upward import on consumer projects) | `import-linter` |
| Coverage | ~85% (gate at 80%) — lazy-imported sink code inflates the unreachable-without-real-backends surface | `pytest --cov` |

CI is in [`.woodpecker.yml`](.woodpecker.yml) and runs on every
push / PR / tag once active. `ci.codeberg.org` access is allow-list
gated; apply at <https://codeberg.org/Codeberg-CI/request-access>.

---

## Layout

```
mgf_common/
├── pyproject.toml hatchling, packages = ["src/mgf"]
├── README.md (this file)
├── PUBLIC_API.md public-API contract per AP-10 (every public name + tier)
├── CHANGELOG.md per-release notes (Keep a Changelog), backfilled to 0.13.0
├── FEEDBACK.md early-consumer feedback + roadmap input
├── CONTRIBUTING.md dev setup, four green gates, PR process, communication channels
├── STARTHERE.md two-flow nav hub for adopting mgf-common (new project / existing project)
├── SECURITY.md vulnerability reporting policy (per SC-12)
├── LICENSE MIT
├── .importlinter single contract: mgf-common-is-a-leaf
├── .woodpecker.yml lint + typecheck + test matrix + build (CI pending)
├── examples/ runnable layered examples (6 topics × 3 layers, smoke-tested in CI)
├── docs/
│ ├── claude/ AI-agent reference (CLAUDE.md + cross-project rules)
│ ├── internal/ contributor-side architecture, lift checklist, extraction history
│ ├── public/ consumer-facing tutorial + recipes + per-component guides
│ └── standards/ cross-project engineering standards (10 topic docs + master README)
└── src/mgf/ PEP 420 namespace package (no __init__.py at src/mgf/)
    └── common/
        ├── __init__.py            re-exports configure / app_name / app_version + __version__
        ├── _identity.py           process-wide identity cache + configure()
        ├── py.typed               PEP 561 marker
        ├── exceptions/            AppError + 7 categories + 7 generic concretes
        ├── config/                BaseAppSettings + make_settings_config + load/save + paths
        └── observability/         setup_logging + setup_otel + crash reporter + redactor + excepthooks
```

---

## Documentation

| File | Purpose |
|---|---|
| [`README.md`](README.md) | This file — install, quick-start, standards links, conformance levels, project layout |
| [`PUBLIC_API.md`](PUBLIC_API.md) | The public-API contract list — every public name with its stability tier, per AP-10 |
| [`CHANGELOG.md`](CHANGELOG.md) | Per-release notes (Keep a Changelog), backfilled from `v0.13.0`, with migration recipes per AP-12 |
| [`FEEDBACK.md`](FEEDBACK.md) | Living document for early-consumer feedback + roadmap input |
| [`CONTRIBUTING.md`](CONTRIBUTING.md) | How to set up dev, the four green gates, the PR process, the design-discussion channels |
| [`STARTHERE.md`](STARTHERE.md) | Two-flow navigation hub: starting a new project vs adopting in an existing one |
| [`SECURITY.md`](SECURITY.md) | Vulnerability reporting policy (per SC-12) |
| [`examples/`](examples/) | Eighteen runnable examples (6 topics × 3 layers); smoke-tested in CI |
| [`docs/standards/`](docs/standards/) | Cross-project engineering standards (10 topic docs + master README, including [`RELEASING.md`](docs/standards/RELEASING.md)) |
| [`docs/public/`](docs/public/) | Consumer-facing tutorial + per-component guides + recipes |
| [`docs/internal/`](docs/internal/) | Contributor-side: [`ARCHITECTURE.md`](docs/internal/ARCHITECTURE.md), [`CI_STARTHERE.md`](docs/internal/CI_STARTHERE.md) (release + diagnose runbook), [`LIFT_CHECKLIST.md`](docs/internal/LIFT_CHECKLIST.md), [`EXTRACTION_HISTORY.md`](docs/internal/EXTRACTION_HISTORY.md), [`STRATEGIC_REVIEW.md`](docs/internal/STRATEGIC_REVIEW.md) |
| [`docs/claude/CLAUDE.md`](docs/claude/CLAUDE.md) | Dense lookup for AI agents and maintainers (gotchas, invariants, phase-by-phase critical facts) |

---

## Reference consumer

[**VManager**](https://codeberg.org/magogi-admin/VManager) — VM-management
desktop application built on top of `mgf-common`. The standards docs
are grounded in real VManager code throughout; "AS-IS in VManager"
sections show the canonical real-world example for each pattern.

The two repos act as a federation:

- This repo owns the library implementation, the cross-project
  standards, and the consumer-feedback pipeline.
- VManager owns its VM-management domain code and its
  (the rolling audit document — see
  [`docs/standards/COMMON_COMPONENTS.md`](docs/standards/COMMON_COMPONENTS.md)
  §"Maintaining compliance over time").

---

## Feedback from consumers

[`FEEDBACK.md`](FEEDBACK.md) is the living interface for opinionated,
brutally-honest feedback from real consumers BEFORE the API locks at
v1.0. The stakes are asymmetric: a 🔴 Blocker concern raised now is
a minor commit; the same concern after v1.0 ships with N consumers
becomes a breaking-change coordination exercise across the ecosystem.

If your project is **about to adopt** `mgf-common`, read
[`FEEDBACK.md`](FEEDBACK.md) first — an early consumer may already
have surfaced a concern that shapes your integration.

If your project is **already using** `mgf-common`, add your own
feedback section. The submission process is documented in
[`FEEDBACK.md`](FEEDBACK.md) §10.

---

## Development

```bash
git clone ssh://git@codeberg.org/magogi-admin/mgf_common.git
cd mgf_common
uv venv
uv sync --extra dev --extra observability

# Quality gates — same set the future CI runs
uv run pytest --tb=short -q
uv run ruff check src tests
uv run mypy src
uv run lint-imports
```

For local development against a consumer, depend on a path source
instead of the PyPI pin:

```toml
# In a consumer project's pyproject.toml:
dependencies = ["mgf-common"]

[tool.uv.sources]
mgf-common = { path = "../mgf_common", editable = true }
```

The release workflow (manual `twine upload` until PyPI adds Codeberg
as an OIDC issuer) is documented in
[`docs/claude/CLAUDE.md`](docs/claude/CLAUDE.md) §"Build and release".

---

## Versioning

The package follows [Semantic Versioning](https://semver.org/). The
**0.x series** exists to make API churn possible while VManager and
other early consumers shape the surface — see
[`docs/internal/EXTRACTION_HISTORY.md`](docs/internal/EXTRACTION_HISTORY.md) §8.5 for the version-bump policy. The release-engineering discipline lives in [`docs/standards/RELEASING.md`](docs/standards/RELEASING.md); per-release notes are in [`CHANGELOG.md`](CHANGELOG.md).

Standards docs (`docs/standards/`) version **independently** of the
package — see [`docs/standards/README.md`](docs/standards/README.md)
§Versioning.

---

## License

MIT — see [`LICENSE`](LICENSE).

---

## Links

- **PyPI:** <https://pypi.org/project/mgf-common/>
- **Codeberg repo:** <https://codeberg.org/magogi-admin/mgf_common>
- **Issues:** <https://codeberg.org/magogi-admin/mgf_common/issues>
- **Reference consumer (VManager):** <https://codeberg.org/magogi-admin/VManager>
