# v0.21.0 — Cutover guide

> **Audience:** consumer agents (and humans relaying for them)
> upgrading to `mgf-common` 0.21.0.
> **What this is:** the canonical migration document for this
> release. Per [`RELEASING.md`](../standards/RELEASING.md) **REL-09**
> + the cutover-guide pattern, individual per-entry notification
> paragraphs are pointers into this guide.

This is a **feature-addition release** — no closures from the
existing `FEEDBACK.md` queue. New top-level surface
`mgf.common.process` for unified subprocess + service-lifecycle
orchestration in tools, scripts, and tests.

The motivation: every consumer reinvents subprocess-with-timeout +
service-with-readiness as needed. The audit before this release
counted 12 raw `subprocess.run` call-sites across VManager +
inthewords + PlasmaMapper, plus `inthewords/tools/run_stack.sh`
which orchestrates Postgres + Redis + Mailpit + Celery + Uvicorn
in bash. The shape that came out of that audit is two top-level
primitives: `run()` for one-shot + `Service` for long-lived.

---

## At-a-glance

| What landed | Severity | What it covers |
|---|---|---|
| **`mgf.common.process.run()`** | feature-addition | Structured replacement for `subprocess.run`. Timeout, redaction, log streaming, `command_id` correlation, process-group cascade. |
| **`mgf.common.process.Service`** | feature-addition | Long-lived process with readiness probe + clean shutdown. Context manager. Out-of-scope: restart-on-crash supervision. |
| **Readiness predicates** | feature-addition | `PortReady`, `HttpReady`, `LogLineReady`, `CallableReady`, `AllOf`, `AnyOf`. Compose for realistic ready signals. |
| **`ProcessError` taxonomy** | feature-addition | 5 new names in `mgf.common.exceptions` — `ProcessError` base + `ProcessTimeout` / `ProcessNonZeroExit` / `ProcessSignaled` / `ServiceNotReady`. All inherit `OperationError`. |
| **Recipe** | docs | `docs/public/recipes/process-management.md` (~370 lines). |

**Pin bump:** `mgf-common = ">=0.21,<0.22"` per the 0.x window
discipline (one minor at a time; AP-03).

**Public-Python-surface change:** +15 names total —
+10 in `mgf.common.process`, +5 in `mgf.common.exceptions`.
Additive only; nothing renamed or removed.

**Behaviour change:** none for existing surface.

**No deprecation cycles introduced.**

---

## What `run()` does

```python
from mgf.common.process import run

result = run(
    ["alembic", "upgrade", "head"],
    cwd="/path/to/project",
    timeout=60.0,
    env_extra={"DB_DSN": dsn},
)
print(result.returncode, result.stdout, result.command_id)
```

Compared to raw `subprocess.run`:

| Concern | `subprocess.run` | `mgf.common.process.run` |
|---|---|---|
| Timeout | `timeout=` raises `TimeoutExpired`; child not necessarily reaped | `timeout=` raises `ProcessTimeout`; child terminated with SIGTERM→SIGKILL escalation; process group cascades |
| Output capture | `capture_output=True` returns `bytes` (or text with `text=True`) — no live streaming | Always streamed live to `logging.getLogger("mgf.process.<name>")` line-by-line; captured as text on result |
| Secret redaction | None | `Redactor` (when bootstrap is active) scrubs each line BEFORE it lands in the log |
| Error correlation | Caller writes their own ID-stamping logic | Every record carries `extra={"child_pid", "stream", "command_id"}` automatically |
| Command-injection | Accepts shell strings via `shell=True` | List-only by construction; rejects strings at the API surface |
| Process group | Caller wires `start_new_session` + `killpg` | Default `process_group=True`; SIGTERM cascades to children |
| Returncode signal kill | Negative returncode; caller checks | `check=True` raises typed `ProcessSignaled` |

When to use raw `subprocess.run` instead: Windows-targeted code
needing CREATE_NEW_PROCESS_GROUP semantics, or any place where
you want strict stdlib-only with no transitive imports.

---

## What `Service` does

```python
from mgf.common.process import Service, AllOf, PortReady, LogLineReady

with Service(
    ["postgres", "-D", str(db_dir), "-p", str(port), "-h", "127.0.0.1"],
    name="pg",
    ready=AllOf([
        PortReady("127.0.0.1", port),
        LogLineReady(r"database system is ready to accept connections"),
    ]),
    timeout_s=20.0,
) as svc:
    run_my_integration_tests(port=port)
# Service is stopped on context-exit; SIGTERM → 5s grace → SIGKILL.
# Process group cascades — no orphaned children.
```

Compared to ad-hoc bash:

| Concern | `tools/run_stack.sh`-style bash | `mgf.common.process.Service` |
|---|---|---|
| Readiness | `sleep 5` or `wait-for-it.sh` | Typed predicates; compose `PortReady` + `LogLineReady` for canonical signals |
| Cleanup on test failure | Trap `EXIT` / `INT`; manual cascade | Context manager; LIFO unwind via `ExitStack` for multi-service |
| Captured output | Tail of a logfile if you remembered to redirect | Logger records with `command_id` correlation; live + persisted |
| Cross-platform | bash-only | POSIX (full) + Windows (best-effort) |
| Composability | `&` + `wait` + manual ordering | Nest `Service` context managers in `ExitStack` for dependency order |

---

## Failure modes

All four `Process*` errors inherit `OperationError`. Existing CLI
top-level handlers that map `OperationError` → exit code (rule
EH-04) catch them unchanged:

```python
from mgf.common.exceptions import (
    ProcessNonZeroExit,
    ProcessSignaled,
    ProcessTimeout,
    ServiceNotReady,
)

try:
    run(["sleep", "30"], timeout=2.0)
except ProcessTimeout as exc:
    # exc.command, exc.command_id, exc.timeout
    log.error("hung", extra={"command_id": exc.command_id})

try:
    run(["false"])
except ProcessNonZeroExit as exc:
    # exc.returncode, exc.stderr_tail (last 10 stderr lines)
    log.error(f"exit {exc.returncode}: {exc.stderr_tail}")

try:
    Service([...], ready=PortReady("127.0.0.1", 5432), timeout_s=5.0).start()
except ServiceNotReady as exc:
    # exc.probe_description, exc.timeout, exc.last_error
    # The service is already torn down — no cleanup needed.
    log.error(f"service didn't come up: {exc.probe_description}")
```

---

## Per-consumer migration notes

### VManager (Qt desktop)

**Suggested adoption:** moderate — direct fit for some code, no rush for others.

- **`smoke-real-libvirt-interactive.py`** has multiple raw
  `subprocess.run([...vmanager_cli...])` invocations. Replace
  with `mgf.common.process.run([..vmanager_cli..], timeout=...,
  capture_output=True)`. You get `command_id` correlation in
  the smoke run's structured log + automatic SIGTERM-on-timeout
  + redaction.
- **`forensics/live.py`** has its own `_PopenLike` Protocol and
  `popen_fn` injection seam. This is a different shape — you
  hand-rolled it because libvirt/qemu forensics have specific
  needs (long-running stream capture with explicit cancellation
  semantics). **No action needed.** The new `Service` shape
  could in principle replace it, but the `_PopenLike` injection
  for testability is a different abstraction; don't churn.
- For **smaller `subprocess.run` call-sites in scripts**: opt
  in opportunistically when you touch them. No bulk migration
  required.

Pin: `mgf-common = ">=0.21,<0.22"`.

### inthewords (Django web)

**Suggested adoption:** direct fit for `tools/`. Significant uplift available.

- **`tools/run_stack.sh`** is the canonical use case for
  `Service` (your bash today: Postgres + Redis + Mailpit +
  Celery + Uvicorn with manual cleanup traps). A Python
  rewrite as `tools/run_stack.py` using `Service` per docker
  service + `ExitStack` for ordering would replace the
  shell-trap dance with deterministic LIFO teardown +
  structured logging. **Not urgent**, but worth filing as a
  follow-up if you'd like the pattern in a recipe.
- **`tools/check_health.py::_docker_inspect`** uses
  `subprocess.run` for `docker inspect ...`. Replace with
  `mgf.common.process.run` — you get `command_id` in your
  structured logs alongside the rest of the health-check
  output (already structured per Phase 8D logging stack).
- **`tools/test_history/cli.py` and `db.py`** — same pattern.
  Replace `subprocess.run` calls.

Pin: `mgf-common = ">=0.21,<0.22"`.

### PlasmaMapper (FastAPI scientific platform)

**Suggested adoption:** direct fit for tooling + tests.

- **`tools/check_health.py`** uses `subprocess.run` for docker
  / system probes. Replace with `mgf.common.process.run` for
  consistency with mgf-common's logging stack.
- **Pytest fixtures** that spin up Postgres-for-integration are
  a strong fit for `Service` + `AllOf([PortReady,
  LogLineReady])`. The recipe doc covers this case explicitly.
- The **uvicorn-style start in `src/plasmamapper/api/__main__.py`**
  is `uvicorn.run(...)` (in-process). `Service` is for *out-of-
  process* launches; don't touch the in-process startup path.

Pin: `mgf-common = ">=0.21,<0.22"`.

---

## Stability promise

Every name in `mgf.common.process` ships at the **`experimental`**
tier per AP-11. Practical promotion bar:

1. One full MINOR cycle without a 🔴 Blocker filed in
   [`FEEDBACK.md`](../../FEEDBACK.md).
2. At least two consumers exercise the API in production-shape
   workloads (a smoke test + a CI integration test counts).

If you hit friction during adoption, file a paper. The
shipping shape is informed by the pre-release audit but real
adoption is what stress-tests it. Same `Approach A / Approach B`
template as every other `FEEDBACK.md` entry.

---

## Engineering note

v0.21.0 is the first **feature-addition release** since v0.18.0
(HTTP-01 architectural batch). Five releases in between were
either bug-fix consolidations (v0.18.0 ↔ v0.19.0) or
documentation polish (v0.20.0). The cadence is reasonable:
features land when consumer-side adoption signals demand them,
not when the maintainer is bored.

The audit-first pattern (Q8 in the pre-release Q&A) is worth
naming — before committing to a feature shape, scan all live
consumer projects for the existing pattern. If their code already
solves the problem in shape X, the new API converges on X (or
makes the case for shape Y explicit). The Q8 audit found 12 raw
`subprocess.run` sites across three projects + one bash
orchestrator; the API surface that fell out covers all of them
without forcing any consumer to refactor on day one.
