# Recipe — Process management (`mgf.common.process`)

> **Audience:** any consumer writing tools / scripts / tests that
> launch external processes — CLI helpers, integration test
> fixtures, demo orchestrators, smoke-test runners, deploy
> tooling.

`mgf.common.process` (v0.21.0) is the structured replacement for
raw `subprocess.run` and ad-hoc "launch this dev server, wait for
the port, run my smoke test, kill it" bash. Two top-level shapes:

- `mgf.common.process.run()` — **one-shot** subprocess wrapper
  with structured `ProcessResult`, timeout-with-grace, redaction-
  aware log streaming, and OTel-correlated `command_id`.
- `mgf.common.process.Service` — **long-lived** process with a
  readiness predicate + clean shutdown. Use as a context manager.

POSIX-first; Windows is best-effort (signal cascade differs;
`process_group=True` silently downgraded). No external
dependencies — stdlib `subprocess` + `socket` + `urllib`.

---

## When to use this recipe

Reach for `mgf.common.process` when:

- You're tempted to `import subprocess` for ad-hoc use (run
  `pip-compile`, run `alembic upgrade`, run `docker ps`, run
  `pytest`, etc.). The wrapper gives you timeout + redaction +
  structured result + log correlation for free.
- You're writing a test fixture that needs to spin up Postgres /
  Redis / your own dev server, wait until it's ready, then
  exercise the system under test.
- You're building a demo runner that orchestrates "start
  docker-compose, run my showcase script, tear down."
- You're writing a CLI helper that pipelines several external
  tools and you want each child's output threaded into your
  app's structured logs (with secrets scrubbed) instead of
  writing to a different file or terminal stream.

Do NOT reach for it when:

- You need supervision (restart-on-crash, backoff). That's
  systemd / supervisord / Procfile territory; this module is
  out-of-scope by design.
- You need a multi-service declarative orchestrator (everything
  docker-compose does). Compose multiple `Service` context
  managers via `contextlib.ExitStack` instead.
- You want async-native subprocess semantics. `mgf.common.process._aio`
  is on the roadmap; today's API is sync.

---

## At a glance

```python
from mgf.common.process import run

# One-shot — replaces `subprocess.run` for tools and scripts.
result = run(["alembic", "upgrade", "head"], timeout=60.0)
print(result.returncode, result.duration_s, result.command_id)
```

```python
from mgf.common.process import Service, AllOf, PortReady, LogLineReady

# Long-lived — replaces "spin up a dev server for a test" bash.
with Service(
    ["postgres", "-D", str(db_dir), "-p", "5432"],
    name="pg-test",
    ready=AllOf([
        PortReady("127.0.0.1", 5432),
        LogLineReady(r"database system is ready to accept connections"),
    ]),
    timeout_s=20.0,
) as svc:
    run_my_integration_tests(pg_url=f"postgres://localhost:{5432}/test")
# Service is stopped on context-exit (SIGTERM → 5s grace → SIGKILL),
# even if the test inside raised. Process group cascades — no orphaned
# children.
```

---

## `run()` — one-shot subprocess wrapper

The list-only contract eliminates command injection at the API
surface; if you want a shell, pass `["/bin/sh", "-c", "..."]`
explicitly:

```python
from mgf.common.process import run

# Capture output + check returncode (raise on non-zero):
result = run(["pip-compile", "requirements.in"], timeout=120.0)
print(result.stdout)

# check=False — inspect returncode yourself:
result = run(["mypy", "src/"], check=False)
if result.returncode != 0:
    print(f"mypy failed; stdout: {result.stdout}")

# Override env without leaking the parent's:
result = run(
    ["env"],
    env_extra={"MY_VAR": "value"},  # merged on top of os.environ
)

# Quiet stdout/stderr — capture still works via the logger:
result = run(["loud-tool"], capture_output=False)
# result.stdout is "" but the logger received every line.
```

### Failure modes (all raise from `mgf.common.exceptions`)

```python
from mgf.common.exceptions import (
    ProcessNonZeroExit,    # check=True and returncode != 0
    ProcessSignaled,       # check=True and child killed by signal
    ProcessTimeout,        # timeout= elapsed, child terminated
)

try:
    run(["sleep", "30"], timeout=2.0)
except ProcessTimeout as exc:
    # exc.command, exc.command_id, exc.timeout
    print(f"hung command: {exc.command}, id={exc.command_id}")

try:
    run(["false"])
except ProcessNonZeroExit as exc:
    # exc.returncode, exc.stderr_tail (last 10 stderr lines)
    print(f"exited {exc.returncode}; tail: {exc.stderr_tail}")
```

All four `Process*` errors inherit `OperationError` — your CLI's
top-level handler that maps `OperationError` → `EXIT_OPERATION_ERROR`
(rule EH-04) catches them unchanged.

### Output streaming + redaction

Every line of stdout / stderr from the child becomes a log record
on `logging.getLogger("mgf.process.<name>")`:

- **stdout** lines log at `INFO`.
- **stderr** lines log at `WARNING` (so aggregator filters surface
  child stderr without over-warning on stdout).
- Each record carries `extra = {"child_pid": int, "stream":
  "stdout"|"stderr", "command_id": str}` for grep / filter.

When `mgf.common.bootstrap()` is active, the configured `Redactor`
scrubs each line BEFORE it lands in the log — so a child that
prints `DATABASE_URL=postgres://user:hunter2@host/db` in its
banner does not leak the password into your aggregator. Override
explicitly:

```python
run(["my-tool"], redact=False)  # debug only — never in production
run(["my-tool"], redact=True)   # require Redactor; raises if no bootstrap
```

### Correlation via `command_id`

Every launched process gets an 8-hex-char `command_id` stamped on
every log record. Use it to retrieve the captured output from
your structured log:

```python
result = run(["alembic", "upgrade", "head"], timeout=60.0)
# Later, when something went wrong:
# $ jq 'select(.extra.command_id == "ab12cd34")' logs/myapp.log
# … prints every line the alembic process emitted, in order, with
#   stream + level intact.
```

### Process-group cascade

`process_group=True` (the POSIX default) launches the child in
its own session group via `start_new_session=True`. On timeout /
explicit kill, SIGTERM goes to the whole group via `os.killpg`.
This prevents the canonical "I started docker-compose, my test
failed, now I have orphaned containers" pain.

```python
# docker-compose spawns several child containers. Without the
# group cascade, a SIGTERM to docker-compose itself doesn't kill
# its children — orphans linger.
run(
    ["docker", "compose", "up", "--abort-on-container-exit"],
    timeout=300.0,
    process_group=True,  # default; SIGTERM cascades to compose's children
)
```

Set `process_group=False` for processes you want to outlive the
caller (rare; usually a smell).

---

## `Service` — long-lived processes

`Service` orchestrates the "launch + wait until ready + run scope
+ tear down" lifecycle. Use as a context manager (recommended;
teardown on exit guaranteed) or via explicit `start()` / `stop()`
when the lifecycle doesn't fit a single scope.

### Common patterns

#### Postgres for an integration test

Postgres binds the port BEFORE it's actually accepting
connections. Pair `PortReady` with `LogLineReady` for the
authoritative ready signal:

```python
from mgf.common.process import Service, AllOf, PortReady, LogLineReady

with Service(
    ["postgres", "-D", str(db_dir), "-p", str(port), "-h", "127.0.0.1"],
    name="pg",
    ready=AllOf([
        PortReady("127.0.0.1", port),
        LogLineReady(r"database system is ready to accept connections"),
    ]),
    timeout_s=20.0,
) as svc:
    # Postgres is fully ready. Run your migrations, fixtures, tests.
    run_migrations(port=port)
    test_repository_layer(port=port)
# On exit: SIGTERM → 5s graceful checkpoint → SIGKILL if needed.
```

#### Uvicorn FastAPI smoke test

```python
from mgf.common.process import Service, HttpReady

with Service(
    ["uvicorn", "myapp.main:app", "--port", "8001"],
    name="api",
    ready=HttpReady("http://127.0.0.1:8001/healthz"),
    timeout_s=10.0,
    env_extra={"APP_ENV": "test"},
) as svc:
    smoke_test_against_url("http://127.0.0.1:8001")
```

#### Multi-service stack via `ExitStack`

For "start postgres, then start the app that depends on it":

```python
from contextlib import ExitStack
from mgf.common.process import Service, PortReady, HttpReady

with ExitStack() as stack:
    pg = stack.enter_context(Service(
        ["postgres", ...],
        name="pg",
        ready=PortReady("127.0.0.1", 5432),
    ))
    api = stack.enter_context(Service(
        ["uvicorn", "myapp.main:app", "--port", "8001"],
        name="api",
        ready=HttpReady("http://127.0.0.1:8001/healthz"),
        env_extra={"DATABASE_URL": "postgres://localhost:5432/test"},
    ))
    # Both ready. Teardown on stack exit happens in reverse order:
    # api first, then pg. Standard ExitStack LIFO semantics.
    run_smoke_suite(api_url="http://127.0.0.1:8001")
```

#### Custom readiness via `CallableReady`

Escape hatch when the built-in probes don't cover your case (a
file appearing on disk, a Unix socket accepting, a custom RPC):

```python
from mgf.common.process import Service, CallableReady
from pathlib import Path

socket_path = Path("/tmp/myapp.sock")  # noqa: S108 — example

with Service(
    ["my-daemon", "--socket", str(socket_path)],
    name="daemon",
    ready=CallableReady(
        socket_path.exists,
        description_str=f"unix socket {socket_path} exists",
    ),
    timeout_s=10.0,
) as svc:
    run_against_socket(socket_path)
```

#### Composite readiness via `AnyOf`

Useful when the canonical ready signal is platform-dependent:

```python
from mgf.common.process import Service, AnyOf, HttpReady, LogLineReady

with Service(
    ["my-service"],
    name="ms",
    ready=AnyOf([
        HttpReady("http://127.0.0.1:8000/-/ready"),
        LogLineReady(r"now serving on .* port 8000"),
    ]),
    timeout_s=10.0,
) as svc:
    ...
```

### What happens on readiness failure

If `ready.check()` doesn't return True within `timeout_s`,
`Service.start()` raises `ServiceNotReady` — but only AFTER
fully tearing down the launched process. **No leaked process on
failure** is a contract:

```python
from mgf.common.exceptions import ServiceNotReady

try:
    Service([...], ready=PortReady("127.0.0.1", 5432), timeout_s=5.0).start()
except ServiceNotReady as exc:
    # exc.command, exc.command_id, exc.probe_description, exc.timeout,
    # exc.last_error (any exception raised by the probe during the wait)
    print(f"service didn't come up: {exc.probe_description}")
    print(f"last_error: {exc.last_error}")
    # The process is dead. No cleanup needed.
```

### Properties

```python
svc = Service([...], ready=...)
svc.start()
print(svc.is_running)    # True / False
print(svc.pid)           # int while running, None before start
print(svc.returncode)    # None while running, int after stop
print(svc)               # Service(name='svc', command=[...], status=running, pid=…)
svc.stop()
print(svc.returncode)    # negative on POSIX SIGTERM kill (-15)
```

`Service.__repr__` (v0.23.0) shows `name`, `command`, `status`
(`not-started` / `running` / `exited(<rc>)`), and `pid` — useful
in logs and pdb.

### Letting a service run to completion — `Service.wait()`

`with Service(...)` always calls `stop()` on exit (kills the
child even if it was about to exit cleanly). When you want
"start the service, let it run until it crashes / completes
naturally," use `wait()`:

```python
with Service(["my-batch-job"], ready=PortReady(...)) as svc:
    rc = svc.wait(timeout=600.0)  # let it run up to 10 min
    if rc is None:
        print("still alive after 10 min — caller should investigate")
    elif rc != 0:
        print(f"crashed with code {rc}")
    else:
        print("completed cleanly")
# Context-exit calls stop() — graceful even after wait() returned a code.
```

`wait()` returns the `returncode` on natural exit, `None` on
timeout (process still running). It does NOT raise on non-zero
exits — the caller decides what to do with the code.

### Best-effort OTel correlation (v0.23.0)

When `mgf.common.bootstrap()` is active AND the OpenTelemetry SDK
is installed, every `run()` and `Service.start()` opens an
`operation_span` named `process.run.<name>` /
`service.start.<name>` with `command_id` as a span attribute.
The span captures the launch's wall-clock duration and any
exception that escaped (`ProcessTimeout` / `ServiceNotReady` /
etc.) so failed launches are visible in the trace UI alongside
the structured log.

When bootstrap is **not** active, the span is silently skipped —
no `UnconfiguredWarning`, no overhead, no behaviour change. The
ergonomic is "trace-aware when wired; transparent when not."

---

## Testing notes

The library's own test suite (in `tests/unit/process/`) demonstrates
the patterns without external dependencies — it uses small Python
one-liner subprocesses as the "service under supervision":

```python
import socket
import sys
import textwrap

def _free_port():
    sock = socket.socket()
    sock.bind(("127.0.0.1", 0))
    port = sock.getsockname()[1]
    sock.close()
    return port

_TINY_LISTENER = textwrap.dedent("""
    import socket, sys
    sock = socket.socket()
    sock.bind(("127.0.0.1", PORT))
    sock.listen(1)
    print("READY", flush=True)
    while True:
        try:
            conn, _ = sock.accept(); conn.close()
        except KeyboardInterrupt:
            break
""").strip()

def test_my_thing():
    port = _free_port()
    with Service(
        [sys.executable, "-c", _TINY_LISTENER.replace("PORT", str(port))],
        ready=PortReady("127.0.0.1", port),
        timeout_s=5.0,
    ) as svc:
        # …
        pass
```

The pytest plugin's `mgf_context` fixture wires bootstrap so
redaction is active during the test. If you need a raw redactor-
free run, pass `redact=False` explicitly.

---

## Cross-references

- **`mgf.common.exceptions`** — the `ProcessError` category + 4
  leaves (`ProcessTimeout`, `ProcessNonZeroExit`,
  `ProcessSignaled`, `ServiceNotReady`).
- **`docs/standards/ERROR_HANDLING.md`** — EH-04 (top-level handler
  maps `OperationError` → exit code) catches `Process*` errors
  unchanged because they all inherit `OperationError`.
- **`docs/standards/web/README.md`** — for FastAPI consumers,
  `Service` plus `HttpReady` is the canonical "launch the app for
  smoke testing" shape.
- **`docs/public/recipes/framework-owned-logging.md`** — when
  `bootstrap(setup_logging=False)`, redaction does not run on the
  `mgf.process.*` logger output unless your framework's handlers
  install the redactor themselves.
- **`PUBLIC_API.md`** — full surface table for `mgf.common.process`.

---

## Out-of-scope reminders

By design, `mgf.common.process`:

- Does **not** restart processes on crash. Use systemd /
  supervisord for that.
- Does **not** model service dependency graphs. Use `ExitStack`
  to chain `Service` context managers.
- Does **not** accept shell-string commands. Lists only —
  eliminates command-injection by construction.
- Is **not** async. The module is sync today; an `_aio` variant
  may land later.
- Supports POSIX fully; **Windows is best-effort** —
  `process_group=True` is silently downgraded to per-PID kill,
  signal cascade semantics differ. If your tooling targets both,
  document the asymmetry consumer-side.
