# 05 — Observability: logging + OTel + redaction

The structured-logging + OpenTelemetry tracing stack. Wired by
`bootstrap()`; consumers use it via `get_logger()`,
`operation_span()`, and the span helpers.

> **Read first:** [`library_common.md`](library_common.md),
> [`01_lifecycle.md`](01_lifecycle.md).

---

## Motivation

Structured logging (JSON) for log aggregators (Loki, ELK,
Datadog). Trace correlation (OTel's `trace_id` / `span_id` in
every record). Sensitive-field redaction by default. Multiple
sinks (console, rotating file, journald, syslog, OTLP, HTTP
webhook) — each opt-in. All of it works without consumers
importing `logging`, `opentelemetry`, or `sentry_sdk` directly.

The closed-box principle is load-bearing here. Re-exports of
the OTel types (`Span`, `SpanContext`, `SpanKind`,
`SpanStatus`, `StatusCode`) live on `mgf.common.observability`
so consumers can type-hint span code without `import
opentelemetry`.

---

## Goal

```python
from mgf.common.observability import get_logger, operation_span

LOG = get_logger(__name__)

def fetch_user(user_id: str) -> dict:
    with operation_span("fetch_user", user_id=user_id):
        LOG.info("starting fetch")
        # ... real work ...
```

If OTel is enabled, `operation_span` opens a real span; the
log record carries `trace_id` + `span_id` automatically. If
OTel is disabled, the span is a no-op and the log record still
emits with the right level, formatter, and redactor.

---

## Quick start

### App entry point

`bootstrap()` wires logging + OTel from `MgfSettings`:

```python
from mgf.common import bootstrap
from mgf.common.settings import MgfSettings, OtelSettings

with bootstrap(
    settings=MgfSettings.production(
        otel=OtelSettings(enabled=True, endpoint="http://otel-collector:4318"),
    ),
) as ctx:
    do_app_work()
```

### Use logger + spans

```python
from mgf.common.observability import get_logger, operation_span

LOG = get_logger(__name__)

def process_request(req_id: str, payload: dict) -> dict:
    with operation_span("process_request", req_id=req_id):
        LOG.info(
            "processing request",
            extra={"req_id": req_id, "payload_size": len(payload)},
        )
        result = _do_work(payload)
        LOG.info("processed", extra={"req_id": req_id})
        return result
```

Every log record under that span carries `trace_id` + `span_id`
in JSON output (when OTel is enabled).

### Span attributes + events from inside

```python
from mgf.common.observability import set_attribute, add_event, set_status, StatusCode

def process_with_attributes(item) -> None:
    with operation_span("process"):
        set_attribute("item.id", item.id)
        set_attribute("item.size_bytes", item.size)
        try:
            _do(item)
            add_event("processed")
        except Exception as e:
            set_status(StatusCode.ERROR, description=str(e))
            raise
```

Every helper is no-op-safe when OTel is disabled.

---

## Public API

### Wiring (called by `bootstrap()`; consumers rarely call directly)

| Name | Purpose |
|---|---|
| [`setup_logging()`](../../PUBLIC_API.md#mgfcommonobservability) | Console + rotating file by default; opt-in journald / OTLP / syslog / HTTP via `MgfSettings`. |
| [`setup_otel()`](../../PUBLIC_API.md#mgfcommonobservability) | Init OTel SDK + OTLP exporter. No-op when SDK absent or no endpoint. |

### Use from app / library code

| Name | Purpose |
|---|---|
| [`get_logger(name=None)`](../../PUBLIC_API.md#mgfcommonobservability) | Returns `logging.getLogger(name)`. Use in lieu of `import logging`. |
| [`operation_span(name, **attrs)`](../../PUBLIC_API.md#mgfcommonobservability) | Context manager wrapping a use case in a span. No-op-safe. |
| [`get_current_span()`](../../PUBLIC_API.md#mgfcommonobservability) | Active `Span` or `None`. |
| [`set_attribute(key, value)`](../../PUBLIC_API.md#mgfcommonobservability) | Set on current span. No-op when no span. |
| [`add_event(name, attributes=None, timestamp=None)`](../../PUBLIC_API.md#mgfcommonobservability) | Add named event to current span. |
| [`set_status(status, description="")`](../../PUBLIC_API.md#mgfcommonobservability) | Set current span's status (typically `(StatusCode.ERROR, "...")`). |

### OTel re-exports (closes CB-12; no `import opentelemetry` needed)

[`Span`](../../PUBLIC_API.md#mgfcommonobservability) /
[`SpanContext`](../../PUBLIC_API.md#mgfcommonobservability) /
[`SpanKind`](../../PUBLIC_API.md#mgfcommonobservability) /
[`SpanStatus`](../../PUBLIC_API.md#mgfcommonobservability) /
[`StatusCode`](../../PUBLIC_API.md#mgfcommonobservability).
When `[observability]` extra isn't installed, these are `Any`
sentinels — type-checkers stay happy; runtime code reaching
into them no-ops via `getattr`.

### Redaction

| Name | Purpose |
|---|---|
| [`Redactor`](../../PUBLIC_API.md#mgfcommonobservability) | Sensitive-field redaction. Frozen dataclass; thread-safe. Composes via `\|`. |
| [`DEFAULT_REDACTION_KEYS`](../../PUBLIC_API.md#mgfcommonobservability) | Built-in secret-bearing key set as `frozenset[str]`. |
| [`LogFormat`](../../PUBLIC_API.md#mgfcommonobservability) | `Literal["text", "json", "auto"]`. |

### Pluggable sinks (Phase 3 registries)

| Name | Purpose |
|---|---|
| [`register_crash_sink(name)`](../../PUBLIC_API.md#mgfcommonobservability) | Decorator: register a custom crash-sink handler. See [`06_crash_reporting.md`](06_crash_reporting.md). |
| [`register_log_handler(name)`](../../PUBLIC_API.md#mgfcommonobservability) / [`register_redaction_pattern_group(name)`](../../PUBLIC_API.md#mgfcommonobservability) / [`register_span_processor(name)`](../../PUBLIC_API.md#mgfcommonobservability) | Same shape; consumer-registered names enabled via `MgfSettings`. |

Code:
[`src/mgf/common/observability/`](../../src/mgf/common/observability/).

---

## Patterns

### Per-component log levels

```python
from mgf.common.settings import LoggingSettings, MgfSettings

settings = MgfSettings(
    logging=LoggingSettings(
        level="INFO",
        level_overrides={
            "urllib3.connectionpool": "WARNING",
            "myapp.noisy_module": "DEBUG",
        },
    ),
)
```

### Custom redaction keys

```python
from mgf.common.observability import Redactor

# Compose default + project-specific:
custom = Redactor.default() | Redactor(extra_keys=frozenset({
    "db_password", "vault_key", "session_token",
}))
```

Pass via `MgfSettings.redaction.extra_keys` for the wired
formatter to use it; or pass `redactor=custom` to
`setup_logging` directly.

### Span-aware error reporting

```python
from mgf.common.observability import (
    operation_span,
    set_status,
    StatusCode,
    add_event,
)

def risky_op() -> None:
    with operation_span("risky_op") as span:
        try:
            _might_fail()
        except Exception as e:
            add_event("exception", attributes={"exception.type": type(e).__name__})
            set_status(StatusCode.ERROR, description=str(e))
            raise
```

### Library code (no bootstrap)

```python
from mgf.common.observability import get_logger, operation_span

LOG = get_logger(__name__)

def my_library_func() -> None:
    # Works whether or not the host has bootstrapped:
    # - bootstrapped: records flow through host's wired handlers
    # - not bootstrapped: records go to whatever Python's default is
    # - operation_span is no-op-safe in both cases
    with operation_span("my_lib.work"):
        LOG.info("doing work")
```

---

## Reserved `LogRecord` attribute names

Stdlib `logging.LogRecord` reserves `name`, `msg`, `args`,
`levelname`, `levelno`, `pathname`, `filename`, `module`,
`exc_info`, `exc_text`, `stack_info`, `lineno`, `funcName`,
`created`, `msecs`, `relativeCreated`, `thread`, `threadName`,
`processName`, `process`. Using any of these as `extra={...}`
keys raises `KeyError`. Use distinct names:

```python
# DON'T:
LOG.info("created", extra={"name": user.username}) # KeyError

# DO:
LOG.info("created", extra={"username": user.username})
LOG.info("created", extra={"subject": user.username})
```

This is documented in
[`docs/standards/LOGGING.md`](../standards/LOGGING.md) §12.1.

---

## Links

- **Public API:** [`PUBLIC_API.md` `mgf.common.observability` section](../../PUBLIC_API.md#mgfcommonobservability)
- **Engineering standards:**
  [`LOGGING.md`](../standards/LOGGING.md) (LG-01..LG-14),
  [`DESIGN_PRINCIPLES.md`](../standards/DESIGN_PRINCIPLES.md) DP-12 (UTC time)
- **Examples:**
  [`examples/03_logging/`](../../examples/03_logging/),
  [`examples/04_observability/`](../../examples/04_observability/)
  — three layers each
- **Code:**
  [`logging_setup.py`](../../src/mgf/common/observability/logging_setup.py),
  [`otel_setup.py`](../../src/mgf/common/observability/otel_setup.py),
  [`redaction.py`](../../src/mgf/common/observability/redaction.py),
  [`json_formatter.py`](../../src/mgf/common/observability/json_formatter.py)
- **Related docs:**
  - [`06_crash_reporting.md`](06_crash_reporting.md) — crash
    reporter shares the redactor + the sink-registry pattern
  - [`04_settings.md`](04_settings.md) — `LoggingSettings` /
    `OtelSettings` / `RedactionSettings` shapes
  - [`recipes/observability_shim.md`](recipes/observability_shim.md)
    — when your project needs to layer custom redactor /
    handler / formatter extensions on top of this stack
    without forking
