# The `mgf.*` federation — sibling packages, one namespace

> **Audience:** anyone evaluating `mgf-common` for a multi-
> package architecture, building a sibling package, or curious
> why this collection looks the way it does.

This document explains the **federation pattern** — the design
choice to ship `mgf-common` plus N sibling packages under one
shared `mgf.*` namespace rather than absorbing every shipped
piece into a single monolith.

It commits the project to a specific shape so consumers can
reason about the architecture instead of inferring it.

## TL;DR

- `mgf-common` is the **federation leaf**: every sibling depends
  on it; nothing depends BACK on a sibling.
- Siblings live under `mgf.*` (PEP 420 namespace). E.g.
  `mgf.common`, `mgf.apiprobe`, `mgf-qt` (future), `mgf.cli`
  (future).
- Each sibling is a **separately distributable** PyPI package
  that consumers install only when they want it. `pip install
  mgf-common` doesn't pull `mgf-apiprobe`.
- The closed-box rule applies federation-wide: no consumer
  reaches past a sibling's documented public surface.
- Cross-sibling imports are **forbidden by import-linter**.
  `mgf-apiprobe` MAY import from `mgf.common`. It MUST NOT
  import from a hypothetical `mgf.qt`.

## The shape

```
                       ┌────────────────────────┐
                       │      Consumer App      │
                       │       (VManager,       │
                       │       my-service, ...) │
                       └────────────┬───────────┘
                                    │ imports
                  ┌─────────────────┼─────────────────┐
                  ▼                 ▼                 ▼
        ┌─────────────────┐ ┌──────────────┐ ┌────────────────┐
        │   mgf.common    │ │ mgf.apiprobe │ │  mgf.qt (...)  │
        │  (this repo —   │ │  (sibling)   │ │   (sibling)    │
        │   the leaf)     │ └──────┬───────┘ └────────┬───────┘
        └─────────────────┘        │                  │
                  ▲                │ depends on       │ depends on
                  │                ▼                  ▼
                  └──────────────  same  ──────────────┘
                                  mgf.common
```

`mgf.common` sits at the bottom. Every sibling depends on it
(one-way). Consumers depend on whichever siblings they need —
typically `mgf.common` plus zero or more siblings.

## Why federation, not monolith

The first version of this work was a single package with every
capability inside. After a principal-engineer review against
the closed-box invariant, that was reversed. The five reasons:

### 1. Surface explosion

Every shipped capability adds public symbols to one
`__all__`. After two or three sibling-shaped additions
(API verification, GUI helpers, CLI orchestration), the public
surface grows beyond what consumers can hold in their head.

The federation lets each capability ship its own surface,
documented in its own `PUBLIC_API.md`.

### 2. The closed-box invariant

`mgf-common`'s load-bearing principle is that consumers don't
reach past it — every interaction with `pydantic`,
`opentelemetry`, `httpx`, etc., goes through a `mgf.common.*`
public name. That property is fragile when the package grows
fast.

Each new capability is a chance to break the invariant. By
isolating capabilities into siblings, each sibling owns its own
closed-box compliance. `mgf.common`'s invariant stays intact;
new siblings adopt it independently.

### 3. Versioning independence

`mgf.common` is the slow-moving leaf — its surface stabilises
toward 1.0. Sibling packages move at their own cadence: a
sibling at `v0.3` doesn't drag `mgf-common`'s minor bumps.

Consumers pin per-package: `mgf-common >= 0.12, < 0.13` AND
`mgf-apiprobe >= 0.2, < 0.3`. Each sibling's release notes
matter to its own users; consumers of one sibling don't pay
for another sibling's churn.

### 4. Standards conformance ramp

`mgf-common` aims at L2 conformance against the engineering
standards (mypy --strict, ≥85% coverage, `PUBLIC_API.md` pin
test, AST-pinned public surface). New capabilities can't ship
at L2 day-one — that's multi-quarter work. Siblings ship at L1
and graduate to L2 on their own schedule without holding back
`mgf-common`.

### 5. Scope discipline

Each sibling has a clear scope (declared in its own
`docs/standards/SCOPE.md`). The federation makes "is this in
scope?" a per-sibling question, not a single
"is-this-in-mgf-common's-scope?" debate.

## Naming + import shape

### Distribution names: `mgf-<sibling>`

PyPI distribution names use hyphens:

- `mgf-common` ← this repo
- `mgf-apiprobe` ← API verification sibling
- `mgf-qt` ← Qt-shaped GUI helpers (hypothetical future)
- `mgf-cli` ← CLI orchestration sibling (hypothetical future)

### Import paths: `mgf.<sibling>`

Underscores inside the sibling part are converted to dots:

- `pip install mgf-common` → `import mgf.common`
- `pip install mgf-apiprobe` → `import mgf.apiprobe`

The `mgf` prefix is a [PEP 420 namespace
package](https://peps.python.org/pep-0420/) — there is NO
`src/mgf/__init__.py` anywhere in the federation. Each sibling's
wheel ships `src/mgf/<sibling>/` content; pip merges the
`mgf/` namespace at install time.

This is what lets `mgf.common` and `mgf.apiprobe` co-exist as
separately-installable packages without either owning `mgf/`.

### Verifying the namespace shape

A consumer can sanity-check the namespace at runtime:

```python
import mgf

# Namespace package — no __file__ at the namespace root.
assert mgf.__file__ is None

# Each installed sibling is a regular subpackage.
import mgf.common
assert mgf.common.__file__ is not None
```

## The federation contracts

Every sibling commits to the rules below. They're enforced by
import-linter + the public-surface tests.

### Contract 1: one-way dependency

```
                    sibling A ──depends on──► mgf.common
                    sibling B ──depends on──► mgf.common
                    sibling A ✗ depends on ✗ sibling B
```

Each sibling MAY depend on `mgf-common`. Siblings MUST NOT
depend on each other.

The contract is enforced via `import-linter`:

```ini
# In each sibling's .importlinter:
[importlinter:contract:no-cross-sibling-imports]
name = sibling-foo MUST NOT import from any other mgf.* sibling
type = forbidden
source_modules = mgf.foo
forbidden_modules = mgf.qt mgf.cli   # whatever siblings exist
```

A consumer importing both `mgf.apiprobe` and `mgf.qt` is fine.
The rule is about **what each sibling itself imports**.

### Contract 2: closed-box invariant

Every sibling exposes a `PUBLIC_API.md` listing its public
surface. Consumers MUST NOT import from underscore-prefixed
modules / names within a sibling.

The discipline is enforced by:

- An AST-walking pin test in each sibling's
  `tests/public_surface/test_no_private_imports.py`
  (mirrored from `mgf-common`'s).
- An `__all__` discipline at every public package boundary.
- AP-10 sync test — every name in `__all__` MUST appear in
  `PUBLIC_API.md`.

Consumers MAY enforce the rule on their own side too, via
their own `import-linter` contract — see
[`recipes/observability_shim.md`](recipes/observability_shim.md)
"Enforce the closed-box with `import-linter`" for the shape.

### Contract 3: standards conformance

Every sibling targets at least L1 against the standards docs
in `mgf-common/docs/standards/`. L1 means:

- `BaseAppSettings` subclass for the sibling's settings root.
- `AppError` hierarchy for the sibling's exceptions (rule EH-01).
- Structured logging via `mgf.common.observability`.
- `__all__` at every public boundary; `py.typed` marker.
- Semver pre-1.0 with `experimental` stability tier.
- Mirrored `tests/{unit,public_surface}/` test layout.
- `filterwarnings=["error", ...]` in pytest config.

L2 graduation (ship-time goal) adds:

- ≥85% coverage.
- Property tests on load-bearing primitives (matchers,
  redactors, etc.).
- `PUBLIC_API.md` + AP-10 sync test.
- AST-pinned public surface (`tests/public_surface/`).
- OTel correlation on every primitive's hot path.

Each sibling tracks its conformance posture in a
`LIFT_CHECKLIST.md` at the repo root. See
[`mgf-apiprobe`'s LIFT_CHECKLIST.md](https://codeberg.org/magogi-admin/mgf_apiprobe/src/branch/master/LIFT_CHECKLIST.md)
for the canonical shape.

### Contract 4: federation hub vs federation member

`mgf.common` is the **hub** — it's the sole leaf every sibling
imports. It MUST NOT import from any sibling (enforced via
`import-linter` contract `mgf-common-is-the-federation-leaf`).

Every other `mgf.*` package is a **member** — it imports the
hub but not other members.

Consumer applications are NOT siblings; they're applications.
A consumer app MAY import from any combination of siblings.

## Sibling vs optional extra: when to choose which

Some integrations live INSIDE `mgf.common` as optional extras
(e.g., `mgf.common.fastapi`, `mgf.common.db`). Others live as
siblings (e.g., `mgf-apiprobe`).

The decision tree:

```
┌──────────────────────────────────────────────────────────┐
│ Is the integration a thin wrapper that REUSES            │
│ mgf.common's primitives directly (logging, identity,     │
│ exceptions) without owning deep state?                   │
│                                                          │
│   Yes → optional extra under mgf.common.<framework>      │
│         (e.g., mgf.common.fastapi)                       │
│   No  → continue                                         │
├──────────────────────────────────────────────────────────┤
│ Does the integration ship its own primitives (a CLI,     │
│ a runtime, a public-contract catalogue) that consumers   │
│ depend on directly — beyond mgf.common's surface?        │
│                                                          │
│   Yes → SIBLING (mgf-<name>, separate PyPI package)      │
│   No  → optional extra (you're probably overthinking it) │
└──────────────────────────────────────────────────────────┘
```

`mgf-apiprobe` is a sibling because it ships:
- Its own primitives (Probe / Check / Suite / AuthFlow / ...).
- Its own CLI (`apiprobe`).
- A public check-code catalogue with semver-stable codes.
- A plug-in architecture (entry-point groups namespaced
  `mgf_apiprobe.*`).

`mgf.common.fastapi` is an extra because it's:
- Stateless (middlewares + Depends helpers).
- Reuses `mgf.common.observability` directly.
- No CLI, no public check codes, no plug-in groups.

The recommendation: **start as an extra**. Promote to a
sibling only when one of the sibling-only requirements
materialises. The reverse direction — sibling → extra — is
breaking and slow. Extra → sibling is additive.

See [`docs/internal/LIFT_CHECKLIST.md`](../internal/LIFT_CHECKLIST.md)
for the full lift recipe + framework-adapter decision matrix.

## Adding a new sibling

If you're starting an `mgf-<name>` sibling, the checklist:

### Repo setup

- [ ] Distribution name `mgf-<name>` (hyphenated, lowercase).
- [ ] Wheel layout `src/mgf/<name>/` (no `src/mgf/__init__.py`).
- [ ] `[project.dependencies]` declares `mgf-common >= X, < Y`.
- [ ] `[tool.hatch.build.targets.wheel] packages = ["src/mgf"]`
      so the wheel ships the namespace prefix.

### Federation contracts

- [ ] `LIFT_CHECKLIST.md` at the repo root, mirroring
      `mgf-apiprobe`'s shape — declare the conformance posture
      and the federation rules the sibling commits to.
- [ ] `.importlinter` ships
      `<sibling>-only-imports-from-mgf-common` — forbidden
      imports list every other federation member.
- [ ] `pytest -W error` + `filterwarnings=["error", ...]` in
      pyproject.

### Public-surface discipline

- [ ] `PUBLIC_API.md` documenting every name in every `__all__`.
- [ ] `tests/unit/test_public_api_doc.py` AP-10 sync test
      (parametrise `_public_names()` over the sibling's modules).
- [ ] `tests/public_surface/` user-perspective tests + the
      `test_no_private_imports.py` AST pin.

### Pre-publish review

- [ ] All four gates green.
- [ ] `mgf-common doctor --standards` passes against the repo.
- [ ] `mgf-common catch-up` reports no drift.
- [ ] Wheel + sdist build cleanly via `uv build`.
- [ ] Smoke install: `pip install dist/mgf_<name>-*.whl` in a
      fresh venv works without `mgf-common` already installed
      (it will be pulled in as a transitive dep).

The reference second-consumer is `mgf-apiprobe`. Read its repo
end-to-end before starting a new sibling — most of the
patterns are already there.

## Federation diagnostics (future)

`mgf-common` is the natural home for federation-wide
diagnostics:

```bash
mgf-common doctor --federation       # future add — list installed
                                       # mgf.* siblings + their pins +
                                       # their conformance posture.
```

Until that ships, you can introspect manually:

```python
import importlib.metadata as md

for dist in md.distributions():
    name = dist.metadata["Name"]
    if name and name.startswith("mgf-"):
        print(f"{name:25s} {dist.version}")
```

## See also

- [`PUBLIC_API.md`](../../PUBLIC_API.md) — `mgf.common`'s
  public surface (the federation leaf's contract).
- [`docs/internal/LIFT_CHECKLIST.md`](../internal/LIFT_CHECKLIST.md) —
  when to lift a component into `mgf.common` vs ship as a sibling
  vs leave in a consumer.
- [`docs/standards/SCOPE.md`](../standards/SCOPE.md) — the
  `mgf.common`-side scope policy. Each sibling carries its
  own scope policy in its own `docs/standards/SCOPE.md`.
- [`recipes/observability_shim.md`](recipes/observability_shim.md)
  — the consumer-side shim pattern; closed-box compliance
  applies inside the shim too.
- [`mgf-apiprobe`'s repo](https://codeberg.org/magogi-admin/mgf_apiprobe)
  — the reference sibling.
