# Configuration

> **Conformance:** L0 / L1 / L2 (see `README.md`)
> **Audience:** Human engineers AND AI coding agents.

> **📦 Post-extraction note:** The reusable base lives in **`mgf.common.config`**: `BaseAppSettings` (pydantic-settings base with layered sources, YAML override, version-check validator), `make_settings_config(env_prefix=...)` (the canonical helper for setting `model_config` on a subclass — DO NOT use `**BaseAppSettings.model_config` spread; it doesn't work, see `src/mgf/common/config/_settings.py` docstring), `load_settings` / `save_settings` (typed loader + atomic writer with optional migration callback), `platform_dir` / `default_config_path` (cross-OS dirs), and `expand_path` (path-validator helper). Citations to `src/vmanager/config.py::AppConfig` in the body describe how the consumer (VManager) subclasses the base for its own typed fields — the reusable base is at `src/mgf/common/config/_settings.py`.

---

## Rules box (read this first)

CF-* rules apply to consumers that **own their config schema**
(the canonical mgf-common shape: pydantic-settings + typed
nested fields + atomic save). For consumers using
**framework-native config** (Django's `from django.conf import
settings`, Rails-equivalent), a subset of rules is N/A — see
[§0a Framework-native config](#0a-framework-native-config-django-rails-equivalent)
below. The `Applies to` column documents the scope.

| ID | Rule | Applies to |
|---|---|---|
| **CF-01** | Application configuration MUST live in a single typed object — a `pydantic_settings.BaseSettings` subclass. **No module reads the config file or env vars directly.** | consumer-owned-config |
| **CF-02** | Precedence MUST be, top-to-bottom: **CLI flags → environment variables → `.env` file → on-disk config file → typed defaults in code.** Higher entries override lower. | all |
| **CF-03** | The config schema MUST carry a `version` field. Loading a higher version than the code knows MUST fail loud. Loading a lower version MUST run a migration chain that brings the dict up to current. | consumer-owned-config |
| **CF-04** | Writes to the on-disk config file MUST be atomic (temp file + rename). Mode MUST be `0o600`. Path expansion (`~`, `$VARS`) MUST happen lazily (at call time, not module import). | consumer-owned-config |
| **CF-05** | Secrets MUST NOT live in the config file. They live in a vault (system keyring / Keychain / Credential Manager / encrypted file fallback) accessed through a separate `Vault` interface. | all |
| **CF-06** | Loading an **unknown key** from the config file MUST raise (typo protection). Missing keys MUST fall back to typed defaults silently. | consumer-owned-config |
| **CF-07** | Sub-config files (hooks, schedules, alerts, profiles, lab specs, etc.) MUST follow the same shape: typed model + atomic write + version field + unknown-key rejection. | consumer-owned-config |
| **CF-08** | The `Settings` object MUST be loaded **once** at the entry point and passed down by constructor injection. Module-global singletons are forbidden. | consumer-owned-config |
| **CF-09** | A `--config <path>` CLI flag MUST always exist. The default path MUST honour XDG / OS conventions (`$XDG_CONFIG_HOME` / `~/.config/<app>/config.yaml` on Linux; `%APPDATA%\<app>\config.yaml` on Windows; `~/Library/Application Support/<app>/config.yaml` on macOS; sandboxed app dir on iOS/Android). | consumer-owned-config |
| **CF-10** | Tests MUST construct `Settings` with explicit values (no env-var leakage). Use `Settings(**overrides)` or `Settings.model_construct` for unit tests. | all |
| **CF-11** | Renaming a config field MUST NOT break existing on-disk configs. Use pydantic `Field(alias=...)` + `validation_alias` to accept the old name; emit a one-shot DEPRECATION WARNING; rewrite to the new name on next save. After two MINOR releases, the alias MAY be removed. | consumer-owned-config |
| **CF-12** | When a config field has multiple mutually-exclusive backend variants (`vault.backend = "keyring" \| "file" \| "kms"`), the variants MUST be modelled as a **discriminated union** with a `kind` discriminator field. Each variant carries only its own backend's settings. See §16. | all |

---

## 0a. Framework-native config (Django, Rails-equivalent)

Closes FEEDBACK CONFIG-01. Web frameworks like Django own the
config schema themselves (settings as Python module + env-var
overlays). Several CF-* rules conflict with the framework's
canonical pattern — applying them literally would mean
abandoning the framework's intended use.

### Which CF-* rules are N/A for framework-native config

The four rules that encode the *consumer-owned, on-disk YAML,
CLI-loadable* shape are explicitly N/A under framework-native
config:

| Rule | Why it's N/A under framework-native config |
|---|---|
| **CF-01** | Django's `config/settings/*.py` IS where env-var reads happen. The settings module IS the consumer-of-env-vars by design. |
| **CF-04** | Django settings are version-controlled Python code, deployed via Ansible/Terraform/Docker. There is no runtime write of "the config file." |
| **CF-08** | `from django.conf import settings; settings.X` IS a module-global singleton lookup. It's the framework's documented access pattern; rewriting it would touch every call site for no behavioural improvement. |
| **CF-09** | Django uses `DJANGO_SETTINGS_MODULE` env var to point at a Python module, not a config file path. ASGI/WSGI processes don't take CLI args. |

### Which CF-* rules still apply

Most of the standard's intent translates cleanly:

- **CF-02** (precedence) — Django reads env vars in
  `settings.py` via `os.environ.get(...)` or
  `decouple.config(...)`. The precedence chain becomes
  *env > .env > settings.py defaults*. Same idea, framework-native
  mechanics.
- **CF-05** (secrets in vault) — Always applies. Django
  `settings.py` MUST NOT contain literal secrets; use
  Ansible Vault / .env / Cloud secrets manager.
- **CF-10** (tests with explicit values) — Pytest fixtures use
  Django's `@override_settings` decorator + per-test env-var
  overrides. Same intent as the canonical pattern.
- **CF-12** (discriminated unions) — Applies wherever the
  consumer has mutually-exclusive backend variants in their
  Django config (cache backend, vault backend, etc.).

CF-03 (schema version), CF-06 (unknown-key reject), CF-07
(sub-config shape), CF-11 (rename via alias) are also N/A for
framework-native — Django settings are Python code with no
schema-version marker, attribute access on a missing key
already raises `AttributeError`, sub-config files don't apply
the same way, and renaming a setting is a code refactor, not
a data migration.

### Framework-native isn't an exemption from discipline

The carve-out narrows the rule scope; it doesn't reduce the
discipline. A framework-native consumer is still expected to:

- **Lint-enforce** that env-var reads happen ONLY inside the
  framework's settings module(s). The
  `inthewords/apps/core/tests/test_central_interfaces.py::TestEnvVarsOnlyInSettings`
  pattern is the canonical shape — bans `os.environ.get(`,
  `os.environ[`, `decouple.config(` outside `config/settings/`.
- **Document** which CF-* rules apply and which are N/A in the
  project's adoption tracker (e.g.,
  `docs/MGF_COMMON_ADOPTION.md`). The §"How `inthewords` adopts
  CF-*" worked example below is the recommended shape.
- **Adopt CF-05** (secrets in vault) without exception. Vault
  discipline is framework-agnostic.

### Worked example — `inthewords` (Django) adopts CF-*

```
CF-01 — N/A (Django-owned). Env-var reads happen ONLY in
        config/settings/*.py; lint-enforced via
        TestEnvVarsOnlyInSettings.
CF-02 — APPLIES. Order is: env > .env > settings.py defaults.
CF-03 — N/A (Django settings have no schema_version field;
        settings ARE Python code).
CF-04 — N/A (settings are deployed code, not runtime data).
CF-05 — APPLIES. Secrets in Ansible Vault (prod) / .env (dev),
        never in settings.py.
CF-06 — N/A (Django attribute access on missing key already
        raises AttributeError).
CF-07 — Project chooses; inthewords keeps tools/lib/test_config.py
        on the typed-config pattern (CF-* compliant) for the
        test harness.
CF-08 — N/A (Django mandates module-global access).
CF-09 — N/A (DJANGO_SETTINGS_MODULE env var replaces it).
CF-10 — APPLIES. pytest fixtures construct Settings with
        explicit overrides; env leaks banned by conftest.
CF-11 — N/A (settings rename is a code refactor, not data
        migration).
CF-12 — APPLIES where relevant (e.g., CACHE_BACKEND alternatives).
```

### Future framework support

Today the carve-out is documented for Django specifically.
Rails (via Ruby), Flask, and FastAPI have similar but distinct
shapes; consumers in those domains MAY follow the same
pattern (declare which CF-* apply, which are N/A) and file
`FRAMEWORK-NN` papers if the carve-out needs framework-specific
clarification.

---

## 0. Flat vs nested Settings — explicit choice

The drop-in template in §3.1 below shows a **nested** schema
(`LoggingSettings`, `StorageSettings`, ...). That is the *recommended*
shape for projects starting from scratch. It is **not** required by
any CF-* MUST.

For projects migrating from an existing flat config (like VManager's
round-3 migration from a stdlib dataclass), a **flat** `BaseSettings`
with the same field names is equally compliant — every CF MUST
(typed model, layered precedence, schema versioning, atomic write,
secrets→vault, unknown-key reject) is satisfied either way. The flat
shape avoids touching every `cfg.X` consumer in the project.

VManager's `AppConfig` is flat (round 3). The decision is documented
at the head of `src/vmanager/config.py`. New projects SHOULD pick
nested upfront because it groups related fields and reads better in
a deep configuration; existing projects MAY stay flat unless they
have a concrete reason to migrate.

The rest of this document describes the nested template. Where the
flat shape differs (almost nowhere — the model_config, validators,
loader, save, and migration patterns are identical), the differences
are noted in line.

## 1. Mental model

Configuration is a **typed, validated, versioned object** built once at process startup from layered sources, then passed by reference to anything that needs it. Everything else is a code smell:

- "Reading the env var here because it's convenient" → CF-01 violation.
- "Loading config.yaml in this module to check one field" → CF-08 violation.
- "Adding a string field to the YAML and parsing it ad-hoc" → CF-01 + CF-06 violation.

The standard prescribes:

1. **`pydantic-settings.BaseSettings`** as the schema.
2. **Layered loading** with documented precedence (CF-02).
3. **Schema versioning** with migration (CF-03) so you can refactor the schema without breaking users.
4. **Atomic writes** (CF-04) so a crash never corrupts state.
5. **Vault separation** (CF-05) so secrets never leak into config dumps, version control, or logs.
6. **Same shape for satellite files** (CF-07) — hooks.yaml, schedules.yaml, etc. each get their own typed model with the same load/save/migrate/validate machinery.

---

## 2. Why pydantic-settings (not stdlib dataclass)

VManager today uses a stdlib `@dataclass` (`src/vmanager/config.py::AppConfig`). That works for L0 / L1 but does not scale to:

- **Env-var merging.** `pydantic-settings` reads env vars natively (`SECRET_KEY=...` → `Settings.secret_key`) with prefix support (`<APP>_SECRET_KEY`).
- **`.env` file support.** Built in. Critical for local development and 12-factor-style deployment.
- **Validation messages.** Pydantic's errors are precise and aggregated — every bad field reported, not just the first one.
- **Nested config.** Sub-models compose cleanly. A `LoggingSettings`, `DatabaseSettings`, `VaultSettings` group themselves.
- **JSON / YAML round-trip.** `model_dump(mode="json")` produces a serializable dict; `Settings.model_validate(dict)` parses it back.
- **Discriminated unions.** Per-backend variants (e.g., `LibvirtBackend | MockBackend`) are first-class.
- **Aliases.** Field renames without breaking older configs (`Field(alias="old_name")`).

Cost: one extra dependency (`pydantic`, `pydantic-settings`). Both are mainstream, well-maintained, and add ~3 MB. Worth it.

---

## 3. The `Settings` schema

### 3.1 Drop-in template — `<app>/settings.py`

```python
"""Application settings — single typed source of truth.

Loaded once at startup by the CLI / GUI / service entry point and
passed down by constructor injection. No other module reads env
vars, the config file, or .env directly (rule CF-01).

Precedence (rule CF-02): CLI flags > env vars > .env > config file > defaults.
Implementation: settings_customise_sources() defines the order.

Sensitive fields: NOT stored here. Use the Vault (rule CF-05).

See ``docs/standards/CONFIGURATION.md`` for the full standard.
"""

from __future__ import annotations

import os
from pathlib import Path
from typing import Any, Literal

import yaml
from pydantic import Field, ValidationError, field_validator
from pydantic_settings import (
    BaseSettings,
    PydanticBaseSettingsSource,
    SettingsConfigDict,
    YamlConfigSettingsSource,
)

from <app>.exceptions import AppConfigError, SchemaValidationError

CURRENT_SCHEMA_VERSION = 1

# ---------------------------------------------------------------------------
# Sub-models — group related fields into nested schemas
# ---------------------------------------------------------------------------

class LoggingSettings(BaseSettings):
    model_config = SettingsConfigDict(extra="forbid")

    level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] = "INFO"
    format: Literal["text", "json", "auto"] = "auto"
    file_path: Path | None = None  # None → default <state>/logs/<app>.log
    enable_otlp: bool = False
    redact_extra_keys: list[str] = Field(default_factory=list)

class StorageSettings(BaseSettings):
    model_config = SettingsConfigDict(extra="forbid")

    image_store: Path = Path("~/.local/share/<app>/images")
    state_dir: Path = Path("~/.local/state/<app>")

class GuiSettings(BaseSettings):
    model_config = SettingsConfigDict(extra="forbid")

    theme: Literal["dark", "light", "system"] = "dark"
    refresh_interval_ms: int = Field(default=2000, ge=100)

# ---------------------------------------------------------------------------
# Top-level Settings
# ---------------------------------------------------------------------------

class Settings(BaseSettings):
    """Single typed source of truth for the application config.

    Construct via ``load_settings()``, NOT by calling ``Settings()``
    directly — load_settings wires the file path and the migration
    chain.
    """

    model_config = SettingsConfigDict(
        env_prefix="<APP>_",                        # SECRET → <APP>_SECRET
        env_nested_delimiter="__",                  # GUI__THEME → gui.theme
        env_file=".env",                            # local dev
        env_file_encoding="utf-8",
        extra="forbid",                             # rule CF-06
        case_sensitive=False,
        validate_default=True,
        # The yaml_file is set dynamically via _yaml_file_path; see
        # settings_customise_sources below.
    )

    # ── Schema version (rule CF-03) ────────────────────────────────
    version: int = CURRENT_SCHEMA_VERSION

    # ── Sub-models ─────────────────────────────────────────────────
    logging: LoggingSettings = Field(default_factory=LoggingSettings)
    storage: StorageSettings = Field(default_factory=StorageSettings)
    gui: GuiSettings = Field(default_factory=GuiSettings)

    # ── Top-level fields ───────────────────────────────────────────
    backend_uri: str = "qemu:///system"
    ssh_timeout_seconds: int = Field(default=30, ge=1, le=600)
    health_check_timeout_seconds: int = Field(default=120, ge=1)

    # ── Validators ─────────────────────────────────────────────────

    @field_validator("version")
    @classmethod
    def _check_version(cls, v: int) -> int:
        if v > CURRENT_SCHEMA_VERSION:
            raise ValueError(
                f"config version {v} is newer than supported "
                f"version {CURRENT_SCHEMA_VERSION}; upgrade <app>"
            )
        return v

    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        """Define precedence (rule CF-02).

        Order: init (CLI flag overrides) > env > .env > YAML file > defaults.
        Pydantic resolves left-to-right (left wins).
        """
        yaml_path = _yaml_file_path()
        yaml_source = YamlConfigSettingsSource(settings_cls, yaml_file=yaml_path) if yaml_path.exists() else None
        sources: list[PydanticBaseSettingsSource] = [
            init_settings,
            env_settings,
            dotenv_settings,
        ]
        if yaml_source is not None:
            sources.append(yaml_source)
        sources.append(file_secret_settings)
        return tuple(sources)

    # ── Path expansion happens lazily on demand (rule CF-04) ───────

    def expanded_image_store(self) -> Path:
        return _expand(self.storage.image_store)

    def expanded_state_dir(self) -> Path:
        return _expand(self.storage.state_dir)

# ---------------------------------------------------------------------------
# Load / save / migrate
# ---------------------------------------------------------------------------

def default_config_path() -> Path:
    """OS-aware default path (rule CF-09).

    Linux  : $XDG_CONFIG_HOME/<app>/config.yaml or ~/.config/<app>/config.yaml
    macOS  : ~/Library/Application Support/<app>/config.yaml
    Windows: %APPDATA%/<app>/config.yaml
    """
    if os.name == "nt":
        base = Path(os.environ.get("APPDATA", Path.home() / "AppData" / "Roaming"))
    elif (xdg := os.environ.get("XDG_CONFIG_HOME")):
        base = Path(xdg)
    else:
        base = Path.home() / ".config"
    return base / "<app>" / "config.yaml"

def _yaml_file_path() -> Path:
    """Used by settings_customise_sources to know which YAML to read.

    Honours the <APP>_CONFIG_PATH env var override, falls back to the
    OS default. Tests override via monkeypatch on this function.
    """
    override = os.environ.get("<APP>_CONFIG_PATH")
    return Path(override) if override else default_config_path()

def load_settings(
    *,
    config_path: Path | None = None,
    overrides: dict[str, Any] | None = None,
) -> Settings:
    """Build the Settings object from layered sources.

    Parameters
    ----------
    config_path : Path or None
        Override the YAML file path. When provided, sets the
        <APP>_CONFIG_PATH env var so settings_customise_sources sees
        it. (We do not pass config_path through Settings(...) because
        pydantic-settings does not support per-call yaml_file overrides
        on YamlConfigSettingsSource.)
    overrides : dict or None
        CLI-level field overrides. These win over env vars (rule CF-02).
    """
    if config_path is not None:
        os.environ["<APP>_CONFIG_PATH"] = str(config_path)

    yaml_path = _yaml_file_path()
    if yaml_path.exists():
        try:
            raw = yaml.safe_load(yaml_path.read_text(encoding="utf-8")) or {}
        except yaml.YAMLError as e:
            raise AppConfigError(f"could not parse {yaml_path}: {e}") from e
        if not isinstance(raw, dict):
            raise AppConfigError(
                f"{yaml_path} must contain a YAML mapping at the top level, got {type(raw).__name__}"
            )
        # Migration chain (rule CF-03).
        version = raw.get("version", CURRENT_SCHEMA_VERSION)
        if isinstance(version, int) and version < CURRENT_SCHEMA_VERSION:
            raw = _migrate(raw, from_version=version)
            yaml_path.write_text(yaml.safe_dump(raw, sort_keys=False), encoding="utf-8")  # rewrite migrated

    # pydantic-settings reads everything; we just pass overrides for the highest-precedence layer.
    try:
        return Settings(**(overrides or {}))
    except ValidationError as e:
        raise AppConfigError(f"invalid configuration: {e}") from e

def save_settings(settings: Settings, *, path: Path | None = None) -> None:
    """Atomic write (rule CF-04).

    Writes to a temp file in the same directory then renames, so a
    crash mid-write never leaves a truncated YAML file. File mode is
    0o600 to keep the config user-private.
    """
    target = path if path is not None else _yaml_file_path()
    target.parent.mkdir(parents=True, exist_ok=True)

    payload = settings.model_dump(mode="json", exclude_unset=False)
    payload["version"] = CURRENT_SCHEMA_VERSION  # always written

    tmp = target.with_suffix(target.suffix + ".tmp")
    try:
        tmp.write_text(
            yaml.safe_dump(payload, sort_keys=False, default_flow_style=False, allow_unicode=True),
            encoding="utf-8",
        )
        tmp.chmod(0o600)
        tmp.replace(target)
    except Exception:
        tmp.unlink(missing_ok=True)
        raise

def _migrate(raw: dict[str, Any], *, from_version: int) -> dict[str, Any]:
    """Walk the migration chain from `from_version` to CURRENT_SCHEMA_VERSION.

    Each step: read the current shape, write the next shape. Steps
    are idempotent and ordered. Add a new step here when you bump
    CURRENT_SCHEMA_VERSION.
    """
    if from_version == 1 == CURRENT_SCHEMA_VERSION:
        return raw
    # Example placeholder for future bumps:
    # if from_version == 1:
    #     raw["new_field"] = "default"
    #     raw["version"] = 2
    #     from_version = 2
    return raw

def _expand(p: Path) -> Path:
    """Lazy ~ / $VAR expansion (rule CF-04 — never at module import time)."""
    return Path(os.path.expandvars(str(p))).expanduser()

__all__ = [
    "CURRENT_SCHEMA_VERSION",
    "GuiSettings",
    "LoggingSettings",
    "Settings",
    "StorageSettings",
    "default_config_path",
    "load_settings",
    "save_settings",
]
```

### 3.1.1 Per-call YAML override via `ClassVar`

Pydantic interprets any annotated class-body assignment as a model
field or `ModelPrivateAttr` (per-instance). A normal `_x: Path | None = None`
declaration becomes a private attribute, NOT the class-level
attribute that `settings_customise_sources` needs.

The right shape: declare it as `ClassVar`:

```python
from typing import ClassVar

class Settings(BaseSettings):
    # NOT a field, NOT a private attr — a real class attribute that
    # ``settings_customise_sources`` (a classmethod) can read + write
    # without per-instance state.
    _yaml_file_override: ClassVar[Path | None] = None
```

Then `load_settings(...)` sets it before constructing the model:

```python
def load_settings(*, config_path: Path | None = None) -> Settings:
    target = config_path or default_config_path()
    # ... (read + migrate the file shape if needed) ...
    prior = Settings._yaml_file_override
    Settings._yaml_file_override = target if target.exists() else None
    try:
        return Settings()
    finally:
        Settings._yaml_file_override = prior
```

And `settings_customise_sources` reads it:

```python
@classmethod
def settings_customise_sources(cls, settings_cls, init_settings, env_settings,
                                dotenv_settings, file_secret_settings):
    sources = [init_settings, env_settings, dotenv_settings]
    path = cls._yaml_file_override
    if path is not None and path.exists():
        sources.append(YamlConfigSettingsSource(settings_cls, yaml_file=path))
    sources.append(file_secret_settings)
    return tuple(sources)
```

**MUST NOT:** load YAML by passing values as init kwargs to
`Settings(...)`. That makes them top-precedence and inverts CF-02
(env should win over YAML).

### 3.2 Wiring at the entry point

```python
# src/<app>/cli/__main__.py
@click.group(...)
@click.option("--config", "config_path", type=click.Path(path_type=Path), default=None)
@click.option("--log-level", default=None)
@click.pass_context
def main(ctx, config_path: Path | None, log_level: str | None) -> None:
    overrides: dict = {}
    if log_level is not None:
        overrides["logging"] = {"level": log_level.upper()}

    settings = load_settings(config_path=config_path, overrides=overrides)

    setup_logging(
        level=settings.logging.level,
        fmt=settings.logging.format,
        log_file=settings.logging.file_path,
        enable_otlp=settings.logging.enable_otlp,
    )

    ctx.ensure_object(dict)
    ctx.obj["settings"] = settings
    # provider, history, vault factories etc. — all take `settings`.
```

---

## 4. Sub-config files (rule CF-07)

Sub-config files are *not* the same as the main `Settings` — they have their own loaders because they are typically **content** (rules, scripts, schedules) rather than **settings**. But they MUST follow the same shape.

### 4.1 The shape every satellite file follows

| Aspect | Requirement |
|---|---|
| Schema | A typed pydantic model (or frozen dataclass for very simple cases) |
| Version field | Yes, top-level `version: int` |
| Loader | `<Type>.from_yaml_file(path) -> <Type>` |
| Saver | `<Type>.save_to_file(path)` (atomic write, mode 0o600) |
| Missing file | Returns an empty default silently if appropriate (e.g., "no hooks defined") |
| Bad YAML | Raises `<App>SpecError` with file path + cause |
| Unknown key | Raises `<App>SpecError` with the offending key name |
| Bad version | Raises `<App>SpecError` (forward-incompatible) or runs migration (backward-compatible) |
| Validation errors | Aggregated — all errors reported in one exception (`SchemaValidationError(errors)`) |

### 4.2 Drop-in template — sub-config loader

```python
# src/<app>/scheduler/store.py — example shape every satellite uses
from __future__ import annotations
from pathlib import Path
from typing import Any
import yaml
from pydantic import BaseModel, ConfigDict, Field, ValidationError

from <app>.exceptions import SchemaValidationError

CURRENT_SCHEDULES_VERSION = 1

class ScheduleEntry(BaseModel):
    model_config = ConfigDict(extra="forbid", frozen=True)

    name: str
    cron: str
    action: str
    target: str
    enabled: bool = True

class ScheduleStore(BaseModel):
    model_config = ConfigDict(extra="forbid")

    version: int = CURRENT_SCHEDULES_VERSION
    entries: list[ScheduleEntry] = Field(default_factory=list)

    @classmethod
    def from_yaml_file(cls, path: Path) -> "ScheduleStore":
        if not path.exists():
            return cls()  # empty default (rule CF-07 missing-file)
        try:
            raw = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
        except yaml.YAMLError as e:
            raise SchemaValidationError([f"{path}: invalid YAML: {e}"]) from e
        if not isinstance(raw, dict):
            raise SchemaValidationError([f"{path}: top level must be a mapping"])

        version = raw.get("version", CURRENT_SCHEDULES_VERSION)
        if isinstance(version, int) and version > CURRENT_SCHEDULES_VERSION:
            raise SchemaValidationError([
                f"{path}: schedules version {version} is newer than supported "
                f"version {CURRENT_SCHEDULES_VERSION}; upgrade <app>"
            ])

        try:
            return cls.model_validate(raw)
        except ValidationError as e:
            errors = [f"{path}: {err['loc']}: {err['msg']}" for err in e.errors()]
            raise SchemaValidationError(errors) from e

    def save_to_file(self, path: Path) -> None:
        """Atomic write (rule CF-04)."""
        path.parent.mkdir(parents=True, exist_ok=True)
        payload = self.model_dump(mode="json")
        payload["version"] = CURRENT_SCHEDULES_VERSION

        tmp = path.with_suffix(path.suffix + ".tmp")
        try:
            tmp.write_text(yaml.safe_dump(payload, sort_keys=False), encoding="utf-8")
            tmp.chmod(0o600)
            tmp.replace(path)
        except Exception:
            tmp.unlink(missing_ok=True)
            raise
```

**MUSTs:**
- Every satellite loader MUST aggregate errors — never raise on the first one.
- Every saver MUST be atomic (`tmp.replace(path)`).
- The path MUST come from `Settings`, not from a hardcoded constant. (Tests override `Settings.<sub>_file` for isolation.)

**AS-IS in VManager:**
- `src/vmanager/scheduler/store.py::ScheduleStore`, `src/vmanager/monitor/alerts.py::AlertStore`, `src/vmanager/hooks/registry.py::HookRegistry`, `src/vmanager/lab/spec.py::LabSpec`, `src/vmanager/profiles/store.py::ProfileStore` — all follow the spirit of this pattern but use stdlib dataclass + manual validation. Migration to pydantic models is part of the gap-closure plan.
- **Gap:** `ProfileStore.save()` is **not atomic** (uses plain `write_text` rather than temp+rename). MUST be fixed.

---

## 5. The Vault — secrets boundary (rule CF-05)

Secrets MUST NOT live in `config.yaml`. The boundary:

| Storage | What goes there |
|---|---|
| `config.yaml` | Non-secret settings, paths, intervals, feature flags, schema version |
| `<state>/credentials.yaml` (encrypted) or system keyring | Passwords, tokens, private keys, API keys |
| `.env` (local dev only — `.gitignore`d) | Per-developer overrides; SHOULD NOT contain secrets in production deploys |
| `secrets/` files (read by pydantic-settings via `file_secret_settings`) | Container/Kubernetes-style file-mounted secrets |

### 5.1 The Vault interface

```python
# src/<app>/vault/__init__.py
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass

from <app>.exceptions import VaultError, KeyringUnavailableError

@dataclass(frozen=True)
class Credential:
    """Opaque credential object — never logged, never serialized to YAML."""
    username: str
    secret: str  # password or private key text

    def __repr__(self) -> str:
        return f"Credential(username={self.username!r}, secret=[REDACTED])"

class Vault(ABC):
    """The credential storage interface.

    Implementations:
      - SystemKeyringVault — libsecret / Keychain / Credential Manager
      - EncryptedFileVault — Fernet AES-128 fallback
    """

    @abstractmethod
    def get(self, key: str) -> Credential: ...

    @abstractmethod
    def set(self, key: str, credential: Credential) -> None: ...

    @abstractmethod
    def delete(self, key: str) -> None: ...

    @property
    @abstractmethod
    def backend(self) -> str:  # "keyring" | "file"
        ...
```

**MUSTs:**
- `Credential.__repr__` MUST redact `secret`.
- The `Vault` instance MUST be constructed from `Settings` paths and injected — same rule as the provider (CF-08).
- The active backend MUST be queryable (`vault.backend`) and SHOULD be logged at startup (rule **DP-10** no silent fallbacks).

**AS-IS in VManager:** `src/vmanager/vault/` implements this with `secretstorage` (libsecret) primary and `Fernet`-encrypted JSON fallback. Backend is queryable. **Gap:** `CredentialVault.__init__` does NOT log the active backend at startup (per VManager's `docs/claude/CLAUDE.md` Gotchas §16).

---

## 6. CLI flag overrides

Every project ships a small standard set of flags that override the corresponding `Settings` field. These are the only places the CLI **overrides** config:

| Flag | Field | Purpose |
|---|---|---|
| `--config <path>` | (file path) | Use this file instead of the default |
| `--log-level <level>` | `logging.level` | Bump verbosity for this run |
| `--log-format <text\|json\|auto>` | `logging.format` | Force structured output for piping |
| `--no-color` | (TextFormatter) | Disable ANSI colours (also honours `NO_COLOR` env) |

Wire them via the `overrides` parameter of `load_settings()`:

```python
@click.group(...)
@click.option("--log-level", default=None, type=click.Choice(["DEBUG", "INFO", "WARNING", "ERROR"]))
@click.pass_context
def main(ctx, log_level: str | None) -> None:
    overrides: dict = {}
    if log_level is not None:
        overrides.setdefault("logging", {})["level"] = log_level
    ctx.obj["settings"] = load_settings(overrides=overrides)
```

**MUSTs:**
- A CLI flag MUST NOT bypass the `Settings` object. It MUST always feed into `overrides`.
- Adding a new CLI flag without a corresponding `Settings` field is a defect (the field is the source of truth; the flag is just a way to set it).

---

## 7. Environment variables

`pydantic-settings` reads env vars natively. The convention:

- **Prefix** = `<APP>_` (uppercase). Example: `VMANAGER_BACKEND_URI`.
- **Nested fields** use the configured `env_nested_delimiter` (we use `__`). Example: `VMANAGER_LOGGING__LEVEL=DEBUG`.
- **Booleans** accept `1/0`, `true/false`, `yes/no` (case-insensitive).
- **Lists** are comma-separated (`VMANAGER_LOGGING__REDACT_EXTRA_KEYS=foo,bar,baz`) or JSON (`'["foo","bar"]'`).

**MUSTs:**
- Env vars MUST follow the prefix convention. No bare `LOG_LEVEL` — too prone to collision with other apps.
- Sensitive env vars (`<APP>_DB_PASSWORD`, `<APP>_API_TOKEN`) MUST go through the Vault, not into `Settings`.
- `.env` files MUST be `.gitignore`d. A `.env.example` SHOULD ship with the project showing the available knobs (without real values).

---

## 8. Schema versioning + migration (rule CF-03)

When the schema changes, you have two paths:

### 8.1 Backward-compatible (most common)

Add a new field with a default. Older configs load fine because the field is missing → default applies. No version bump needed.

```python
# Before
class Settings(BaseSettings):
    version: int = 1
    backend_uri: str = "qemu:///system"

# After (no version bump)
class Settings(BaseSettings):
    version: int = 1
    backend_uri: str = "qemu:///system"
    new_optional_field: int = 30  # default → backward-compat
```

### 8.2 Breaking change (rare)

Bump `CURRENT_SCHEMA_VERSION`. Add a migration step. Older configs load → migration runs → file is rewritten on disk.

```python
CURRENT_SCHEMA_VERSION = 2

def _migrate(raw: dict[str, Any], *, from_version: int) -> dict[str, Any]:
    if from_version == 1:
        # v1 had `timeout_ms`; v2 split into ssh_timeout_seconds + http_timeout_seconds
        old = raw.pop("timeout_ms", 30_000)
        raw["ssh_timeout_seconds"] = old // 1000
        raw["http_timeout_seconds"] = old // 1000
        raw["version"] = 2
        from_version = 2
    return raw
```

**MUSTs:**
- Loading a higher version than known MUST raise `AppConfigError` with a message telling the user to upgrade the app.
- Loading a lower version MUST run migration silently and rewrite the file with the migrated content.
- A migration step MUST be idempotent (running it twice produces the same result).
- A unit test MUST exercise every migration step end-to-end (load v(N-1) → assert fields after migration).

### 8.3 Accepting files that predate the `version` field (VManager round 1)

When you **add** the `version:` field to an existing sub-config
format (i.e., you are tightening CF-07 on a file format users are
already writing), every user-authored file on disk predates the
rule. Three choices:

1. **Fail loud.** Refuse to load any file without `version:`.
   Worst UX — users who upgrade the app see every existing
   `profile.yaml` / `hooks.yaml` / `schedules.yaml` reject on
   first run.
2. **Silent default.** Treat missing `version:` as the current
   version with no signal. Best UX but costs you a diagnostic —
   users who later hit a migration bug have no way to tell
   whether their file was ever versioned.
3. **WARN and default** (recommended). Log a WARNING that names
   the file + the current version, treat the load as
   `CURRENT_VERSION`, then rewrite the file with `version:` at
   the top on the next save. This is what VManager's Round 1
   CF-07-profiles close did.

```python
@classmethod
def from_dict(cls, raw: dict[str, Any]) -> MyConfig:
    # ... unknown-key rejection ...

    raw_version = raw.get("version")
    if raw_version is None:
        LOG.warning(
            "config %r: missing 'version' field — treating as v%d. "
            "Re-save the file to add 'version: %d' at the top.",
            raw.get("name", "<unknown>"),
            CURRENT_VERSION,
            CURRENT_VERSION,
        )
        version = CURRENT_VERSION
    elif isinstance(raw_version, bool) or not isinstance(raw_version, int):
        # bool is a subclass of int in Python — treat it as a
        # type error rather than silently accepting True as 1.
        raise SchemaValidationError(
            [f"version must be an int, got {type(raw_version).__name__}"],
        )
    else:
        version = raw_version

    if version > CURRENT_VERSION:
        raise SchemaValidationError(
            [f"version {version} is not supported; expected <= {CURRENT_VERSION}"],
        )
    if version < CURRENT_VERSION:
        raw = _migrate(raw, from_version=version)

    # ... rest of parsing ...
```

**MUSTs for this path:**
- The WARNING MUST carry the file identifier (path or name) so the
  user can find the right file to fix.
- The next `save_*` call MUST emit `version: <CURRENT_VERSION>` so
  re-saving a legacy file silently upgrades it on disk.
- A unit test MUST pin the "missing-version accepted with WARNING"
  behaviour — otherwise a future PR that switches to "fail loud"
  silently breaks every existing user's install on upgrade.
- `bool` MUST be rejected as a version value even though it is an
  `int` subclass in Python — `True` / `False` in the YAML is
  almost always a typo.

The pattern is idempotent: once a legacy file is re-saved, the
WARNING stops firing and the file is indistinguishable from a
natively-versioned one.

---

### 8.4 Serialising nested sub-models — prefer `model_dump(mode="json")`

`mgf.common.config.BaseAppSettings.to_yaml_dict()` is **shallow**
— it iterates `model_fields` and converts only `Path` instances
explicitly, returning every other value as-is. Nested
`BaseModel` sub-fields (`LoggingSettings`, the discriminated
`VaultConfig` from §16, etc.) emerge as Python objects that
`yaml.safe_dump` cannot represent. Calling `save_settings(cfg)`
on a `BaseAppSettings` subclass with nested sub-models therefore
raises `yaml.representer.RepresenterError`.

This is a known limitation of the helper. Two paths:

1. **Use `model_dump(mode="json")` directly** (the recommended
   shape per `to_yaml_dict`'s own docstring). Then write
   atomically via the canonical temp + rename pattern:

   ```python
   payload = settings.model_dump(mode="json")
   payload["version"] = type(settings).CURRENT_VERSION
   tmp = path.with_suffix(path.suffix + ".tmp")
   tmp.write_text(yaml.safe_dump(payload, sort_keys=False))
   tmp.chmod(0o600)
   tmp.replace(path)
   ```

2. **Stay flat** until `to_yaml_dict` recurses. Flat schemas
   (every field a scalar / Path / `Literal`) work with
   `save_settings` directly. The discriminated-union pattern
   (§16) also depends on nested models, so this only buys time.

**Forward path:** a future `mgf-common` release SHOULD make
`to_yaml_dict` use `model_dump(mode="json")` internally so
`save_settings` works for any pydantic model shape. Tracked in
[`FEEDBACK.md`](../../FEEDBACK.md).

---

## 9. Atomic writes (rule CF-04)

**MUSTs:**
- Any state file the app owns MUST be written atomically: write to `<file>.tmp` in the same directory, then `tmp.replace(target)`.
- File mode MUST be `0o600` for any file that may carry private info.
- The temp file MUST be cleaned up on exception (try/except around the write, `tmp.unlink(missing_ok=True)` in the except).

This applies to `config.yaml`, every satellite YAML, the alert state cursor, the scheduler tick cursor, the port-forward state file, the encrypted vault file — **everything** the application writes.

**AS-IS in VManager:** mostly compliant. `config.save_config`, `AlertState`, `TickState`, `ScheduleStore.save_to_file`, `AlertStore.save_to_file` all use the temp+rename pattern. **Gap:** `ProfileStore.save()` does not — see gap plan.

---

## 10. Lazy path expansion (rule CF-04)

```python
# WRONG — captures $HOME at module import time, breaks tests that monkeypatch HOME per test
DEFAULT_PATH = Path("~/.config/<app>/x.yaml").expanduser()

# RIGHT — resolves at call time
def _default_path() -> Path:
    return Path("~/.config/<app>/x.yaml").expanduser()
```

This rule applies *both* to module-level constants *and* to dataclass `default_factory`. Every default that depends on `$HOME` or `$XDG_*` MUST be wrapped in a function called at use time.

**Why:** unit tests monkeypatch `HOME` per test (`monkeypatch.setenv("HOME", str(tmp_path / "home"))`). Module-load expansion captures the parent process's `$HOME` once and breaks per-test isolation — every test ends up writing to the developer's real home directory.

**AS-IS in VManager:** the rule is followed for every subsystem
default — `creator/builder.py::_default_state_root`,
`monitor/history.py::_default_db_path`,
`operations/port_forward.py::_default_state_path`,
`profiles/store.py::_default_user_dir`,
`vault/keyring.py::_default_vault_file`,
`vault/keyring.py::_default_key_file`. Pinned by
`tests/unit/test_default_path_resolution.py` (see §10.1 below).
Documented in `docs/claude/CLAUDE.md` Cross-cutting patterns §6.

### 10.1 Pinning the lazy-default pattern (VManager round 3)

When a project has many `_default_X()` helpers — one per
subsystem that caches something under `$HOME` — the regression
protection is best concentrated in a **single consolidated pin
test file**, not scattered across each subsystem's own test
module.

VManager's `tests/unit/test_default_path_resolution.py` is the
reference shape: one class `TestLazyHomeResolution` with one test
per `_default_X()` helper, each of the same shape:

```python
def test_history_db_path_follows_home_monkeypatch(
    self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
    """_default_db_path() resolves HOME lazily."""
    fake_home = tmp_path / "alternate-home"
    monkeypatch.setenv("HOME", str(fake_home))
    resolved = _history._default_db_path()
    assert str(resolved).startswith(str(fake_home)), (
        f"_default_db_path() returned {resolved!r} — expected a path "
        f"under monkeypatched HOME ({fake_home!r})."
    )
```

**Why one file, not one per module:**
- Future readers see the full "lazy HOME default" pattern
  enforced in one visible place. Scattered tests make the rule
  feel like it applies only where the specific test happens to
  cover.
- Adding a new default-path helper in a new subsystem becomes a
  one-line-in-one-file obligation: add a row here. A new test
  file per subsystem invites the "I'll add the test later"
  anti-pattern.
- The failure mode under regression is legible — "the builder's
  default path isn't HOME-isolated" is easier to act on when the
  test is named `test_builder_state_root_follows_home_monkeypatch`
  in a file labelled `test_default_path_resolution.py`.

**Bonus:** add a second test class `TestBackwardsCompatAliases`
that asserts every public `DEFAULT_X` module-level constant is
still importable (for callers that grep for the legacy name).
Those constants still capture `$HOME` at module load, but keeping
them importable preserves source-compat. The pin test is your
safety net against someone removing the backwards-compat alias
without a deprecation cycle.

---

## 11. Testing config

**MUSTs:**
- Tests MUST construct `Settings` with explicit values:

```python
def test_some_thing(monkeypatch, tmp_path):
    # Don't read the developer's actual config
    monkeypatch.delenv("VMANAGER_CONFIG_PATH", raising=False)
    monkeypatch.setattr("vmanager.settings._yaml_file_path", lambda: tmp_path / "x.yaml")
    settings = load_settings(overrides={"backend_uri": "mock://"})
    assert settings.backend_uri == "mock://"
```

- Tests MUST NOT assume the developer's local `.env` or `~/.config/<app>/config.yaml` exists.
- A test that needs a temporary config file MUST use `tmp_path` and pass it via `config_path=`.

**SHOULDs:**
- Property-test the round-trip: `Settings(...)` → `save_settings(s, path)` → `load_settings(config_path=path)` → equal.

---

## 12. Anti-patterns (forbidden)

| Anti-pattern | What's wrong | Right shape |
|---|---|---|
| `os.environ.get("X_THING")` in a use-case module | Config bypass; inconsistent | Add field to `Settings`; consume from there |
| `yaml.safe_load(open("config.yaml"))` outside the loader | CF-01 violation | `ctx.obj["settings"]` |
| Plain `path.write_text(yaml.dump(...))` for owned state | Non-atomic; partial write on crash | `tmp = path.with_suffix(...+".tmp"); ...; tmp.replace(path)` |
| `DEFAULT_PATH = Path("~/.x").expanduser()` at module top | Captures HOME at import | `def _default_path(): return Path("~/.x").expanduser()` |
| Storing a password/token in `config.yaml` | Plaintext secret in version control / logs | Vault |
| Swallowing `ValidationError` to "be lenient" | Silent typos; user thinks setting was applied | Re-raise as `SchemaValidationError(errors)` |
| Module-level `Settings()` singleton | Untestable; first-import wins | Constructor injection from entry point |
| Adding a new sub-config file with bespoke parsing | Drift from the standard shape | Use the §4.2 template |

---

## 13. AS-IS in VManager

- **Schema:** stdlib `@dataclass` (`src/vmanager/config.py::AppConfig`). 30+ fields; flat (not nested into sub-models).
- **Loader:** `load_config(path) -> AppConfig` reads YAML + validates manually. Unknown keys rejected (good). Atomic write (good). Lazy path expansion in `__post_init__` (good).
- **Schema versioning:** `CURRENT_VERSION = 1` + `_migrate()` placeholder (good).
- **Injection rule:** upheld — verified by audit. (See `docs/standards/COMMON_COMPONENTS.md`.)
- **Sub-config files:** hooks.yaml, schedules.yaml, alerts.yaml, lab specs, profiles, VM specs — all custom loaders, all roughly follow the shape but with stdlib dataclass + manual validation.
- **Vault:** `src/vmanager/vault/` — keyring + encrypted file fallback. Clean boundary.
- **Env vars:** scattered (`os.environ.get` for `XDG_CONFIG_HOME`, `VMANAGER_USE_MOCK`, `EDITOR`) — not centralised in config.

**Gaps:**
- Not on `pydantic-settings` (this standard's choice).
- No `.env` support.
- Env-var reads scattered, not via `Settings`.
- `ProfileStore.save()` non-atomic.
- No backend-startup log line for the Vault.

---

## 14. Gap-closure plan for VManager

### Priority P0 — migration to pydantic-settings

1. **Add `pydantic` and `pydantic-settings` to `pyproject.toml` core deps**.
2. **Create `src/vmanager/settings.py`** alongside the existing `config.py` — paste from §3.1, rename `<app>` → `vmanager`, port every `AppConfig` field with the same default. Group fields into nested sub-models: `LoggingSettings`, `StorageSettings`, `MetricsSettings`, `GuiSettings`, `VaultSettings`, `LabSettings`.
3. **Map every existing `cfg.X` reference to `settings.<sub>.X`** — automated rewrite. This is a large diff but mechanical.
4. **Replace `load_config` with `load_settings`** in CLI and GUI entry points.
5. **Delete `src/vmanager/config.py`** once all call sites are migrated. Keep a deprecation shim for one release if external code depends on `AppConfig` (none does today).
6. **Migration step in `_migrate(from_version=1)`** for the `AppConfig → Settings` field reshape — translate the flat YAML into the nested form (e.g., `gui_theme` → `gui.theme`).

### Priority P1 — close gaps

7. **`ProfileStore.save()` atomic-write**. Patch `src/vmanager/profiles/store.py` to use temp+rename, mode 0o600.
8. **Vault startup log**. Add `LOG.info("vault backend: %s", self.backend)` at the end of `CredentialVault.__init__`.
9. **Centralise env-var reads**. The few env vars that bypass config (`VMANAGER_USE_MOCK`, `EDITOR`, `XDG_CONFIG_HOME`) move into `Settings` fields driven by `Field(validation_alias="VMANAGER_USE_MOCK")` etc. The `os.environ.get(...)` calls scattered across `cli/__main__.py`, `gui/app.py`, and `cli/groups/vm_config.py` MUST go.

### Priority P2 — sub-configs as pydantic models

10. **Convert each satellite store to a pydantic model** following §4.2. One PR per file:
   - `scheduler/store.py::ScheduleStore`
   - `monitor/alerts.py::AlertStore`
   - `hooks/registry.py::HookRegistry`
   - `lab/spec.py::LabSpec`
   - `profiles/store.py::ProfileStore`
11. **Lift the validation paths** — the existing manual validators become field validators on the pydantic model.

### Priority P3 — `.env` support + docs

12. **Add `.env.example`** at repo root listing every overridable env var with a short comment.
13. **Add `.env` to `.gitignore`**.
14. **README section "Configuration via env vars"** — list the prefix, the nested delimiter, and link to this standard.

### Priority P4 — testing

15. **Property-test config round-trip**: Hypothesis-generated `Settings` → save → load → equal.
16. **Test every migration step**: synthesise a v(N-1) YAML, load, assert v(N) shape.
17. **Test that no module imports `os.environ`** other than the centralised settings + a small allowlist (logging journald detection, OTEL endpoint, crash sink). Lint with grep + allowlist file.

---

## 15. Cross-references

- The `Vault` interface lives next door to `Settings` in `<app>/vault/`. See [`COMMON_COMPONENTS.md`](COMMON_COMPONENTS.md) for the catalogue entry.
- Every sub-config file's parse error MUST raise `SchemaValidationError` from [`ERROR_HANDLING.md`](ERROR_HANDLING.md) §2.3.
- Logging is configured *from* `Settings.logging` — see [`LOGGING.md`](LOGGING.md) §5.
- Secrets boundary, vault backend selection, no silent fallback (rule SC-01..SC-05): [`SECURITY.md`](SECURITY.md) §2.
- Test patterns for config (HOME isolation, schema migration, round-trip property tests): [`TESTING.md`](TESTING.md) §11.3 + §5.

---

## 16. Discriminated unions for backend variants (rule CF-12)

When a config field has multiple mutually-exclusive backend variants
— a vault that is keyring / file / KMS, a metrics sink that is
prometheus / OTLP / disabled, a queue backend that is sqlite /
postgres / redis — the wrong shape is a flat structure with every
backend's fields as siblings:

```python
# WRONG — flat shape, ambiguous
class VaultSettings(BaseSettings):
    backend: Literal["keyring", "file", "kms"]
    keyring_service: str = "<app>"
    file_path: Path | None = None
    file_passphrase_env: str | None = None
    kms_endpoint: str | None = None
    kms_key_id: str | None = None
```

Problems:
- Most fields are `None` for any given backend; the type system
  doesn't enforce which.
- Adding a new backend touches the same flat class; the file grows
  without bound.
- A typo (`backend = "kerying"`) silently picks defaults from the
  wrong variant.
- Validation can't say "this field is required for the file
  backend" without an after-validator full of `if/elif`.

The right shape is a **discriminated union** with a `kind` field:

```python
# RIGHT — variants typed independently, discriminator picks
from typing import Literal, Annotated, Union

from pydantic import BaseModel, ConfigDict, Field

class KeyringVaultConfig(BaseModel):
    model_config = ConfigDict(extra="forbid")
    kind: Literal["keyring"] = "keyring"
    service: str = "<app>"

class FileVaultConfig(BaseModel):
    model_config = ConfigDict(extra="forbid")
    kind: Literal["file"] = "file"
    path: Path
    passphrase_env: str = "<APP>_VAULT_PASSPHRASE"

class KMSVaultConfig(BaseModel):
    model_config = ConfigDict(extra="forbid")
    kind: Literal["kms"] = "kms"
    endpoint: str
    key_id: str

VaultConfig = Annotated[
    Union[KeyringVaultConfig, FileVaultConfig, KMSVaultConfig],
    Field(discriminator="kind"),
]

class Settings(BaseAppSettings):
    vault: VaultConfig = KeyringVaultConfig()
```

In YAML:

```yaml
vault:
  kind: file
  path: ~/.local/share/myapp/vault.enc
  passphrase_env: MYAPP_VAULT_PASSPHRASE
```

Pydantic uses `kind` to pick the right model class, validates the
backend's required fields, and rejects fields that belong to a
different backend. `extra="forbid"` makes the rejection loud
(rule **CF-06**).

### 16.1 Adding a new backend

A new backend is a new class plus one entry in the union — the
rest of the codebase is untouched:

```python
class HSMVaultConfig(BaseModel):
    model_config = ConfigDict(extra="forbid")
    kind: Literal["hsm"] = "hsm"
    pkcs11_lib: Path
    slot_id: int

VaultConfig = Annotated[
    Union[KeyringVaultConfig, FileVaultConfig, KMSVaultConfig, HSMVaultConfig],
    Field(discriminator="kind"),
]
```

The factory that builds the runtime `Vault` from `Settings.vault`
matches on `kind` and constructs the right impl:

```python
def build_vault(cfg: VaultConfig) -> Vault:
    match cfg:
        case KeyringVaultConfig(service=s):
            return SystemKeyringVault(service=s)
        case FileVaultConfig(path=p, passphrase_env=env):
            return EncryptedFileVault(path=p, passphrase=os.environ[env])
        case KMSVaultConfig(endpoint=e, key_id=k):
            return CloudKMSVault(endpoint=e, key_id=k)
        case HSMVaultConfig(pkcs11_lib=lib, slot_id=s):
            return HSMVault(pkcs11_lib=lib, slot_id=s)
```

The `match` exhaustiveness is checked by mypy — adding a new
variant without updating the factory fails CI.

### 16.2 When NOT to use

A discriminated union is overkill when:
- There's only one variant (it's just a struct).
- Variants share most of their fields and differ only in one
  scalar (a single `Literal` field is simpler).
- The "variant" is really a future plan, not present-day requirement
  (YAGNI; introduce when the second variant exists).

---

## 17. Environment-tier layering (dev / staging / prod)

Most projects distinguish at least two environments — local
development and production. The standard's layered precedence
(rule **CF-02**) handles per-developer overrides via `.env`, but
team-wide environment tiers need a tier-aware loader.

### 17.1 Tier as a first-class concept

The active tier is a single environment variable (`<APP>_ENV`)
read at the entry point. Acceptable values: `dev`, `staging`,
`prod`, plus any custom tier the project needs (`ci`, `local`).

### 17.2 The tier-aware load order

For each tier, `load_settings` looks for a tier-specific YAML in
addition to the base config:

```
~/.config/<app>/config.yaml ← base
~/.config/<app>/config.<tier>.yaml ← tier overlay
.env, env vars, CLI flags ← per CF-02
```

Layered precedence (highest wins): CLI > env > `.env` > tier YAML
> base YAML > defaults. The tier YAML overrides the base; both are
checked into git for team-wide consistency. Per-developer secrets
go into `.env` (not committed).

### 17.3 Implementation

```python
def load_settings(
    settings_cls: type[T],
    *,
    path: Path | None = None,
    tier: str | None = None,
) -> T:
    base = path or default_config_path()
    tier = tier or os.environ.get("<APP>_ENV", "")
    tier_path = base.with_name(base.stem + f".{tier}" + base.suffix) if tier else None

    raw: dict[str, Any] = {}
    if base.exists():
        raw = _read_yaml(base)
    if tier_path and tier_path.exists():
        overlay = _read_yaml(tier_path)
        raw = _deep_merge(raw, overlay)

    # ... migration, validation, etc.
    return settings_cls.model_validate(raw)
```

`_deep_merge(base, overlay)` recurses into nested dicts; lists are
replaced wholesale (an "append" semantic invites surprises).

### 17.4 Tier-MUST conventions

| Tier | What MUST be true |
|---|---|
| `dev` | No real credentials; mocks where possible; full debug logging |
| `staging` | Production shape with reduced scale; test credentials only |
| `prod` | All credentials from vault; no debug logging by default; OTLP enabled |

Code MUST NOT branch on the tier value:

```python
# WRONG
if settings.tier == "prod":
    enable_security_features()

# RIGHT — security features on by default; tier-specific behaviour
# is expressed through config values, not code branches
enable_security_features(level=settings.security.level)
```

The principle: tiers are config layers, not feature flags. Code
that needs different behaviour reads a config value; the tier
files set the right value for each tier.

### 17.5 CI environments

CI loads `<APP>_ENV=ci` (or `test`) and a `config.ci.yaml` that
points every external endpoint at a mock or in-memory backend.
Tests MUST NOT need a tier file — they construct `Settings`
explicitly per rule **CF-10** and bypass the loader entirely.

---

## 18. Field aliases for renames-without-breaks (rule CF-11)

A field rename (`old_name` → `new_name`) breaks every existing
on-disk config that uses the old name. Pydantic's
`Field(validation_alias=...)` accepts both names while the schema
holds only the new one:

```python
from pydantic import AliasChoices, BaseSettings, Field

class Settings(BaseAppSettings):
    # New canonical name; legacy `connect_timeout_s` accepted for read.
    connection_timeout_seconds: int = Field(
        default=30,
        validation_alias=AliasChoices(
            "connection_timeout_seconds",  # new
            "connect_timeout_s",           # legacy
        ),
    )
```

A YAML with the old key loads cleanly:

```yaml
connect_timeout_s: 60 # OK — accepted via alias
```

### 18.1 The deprecation warning

The first time a legacy alias is used in a process, log a
DEPRECATION WARNING with the file identifier and the new name:

```python
@model_validator(mode="before")
@classmethod
def _warn_on_legacy_aliases(cls, values: dict[str, Any]) -> dict[str, Any]:
    legacy_map = {
        "connect_timeout_s": "connection_timeout_seconds",
        "log_format": "logging.format",
    }
    for legacy, current in legacy_map.items():
        if legacy in values:
            LOG.warning(
                "config field %r is deprecated; use %r. "
                "The legacy name will be removed in v3.0.",
                legacy, current,
            )
    return values
```

### 18.2 The rewrite

`save_settings` writes the canonical form. After the next save, the
on-disk file uses the new name and the warning stops firing.

### 18.3 The deprecation timeline

| Step | When | What |
|---|---|---|
| Add alias + warn | release N | Both names accepted; old name warns |
| Document removal date | release N+1 |notes the removal target |
| Remove alias | release N+2 (≥ 6 months after N) | Old name now an unknown-key error (rule **CF-06**) |

For breaking config changes that cannot be aliased (a field type
narrowing, e.g., `int` → `Literal[1, 2, 3]`), bump the schema
`version` and migrate (rule **CF-03**).

### 18.4 Anti-patterns

| Anti-pattern | What's wrong | Right shape |
|---|---|---|
| Rename without alias | Existing configs fail to load on upgrade | `validation_alias=AliasChoices(new, legacy)` |
| `if "old_name" in raw: raw["new_name"] = raw.pop("old_name")` in the loader | Every reader needs to know about the migration | Pydantic alias does it once, declaratively |
| Permanent alias (never removed) | Schema accumulates dead names; cognitive load | Plan removal in the deprecation timeline |
| Aliasing a field that changed semantics | Loads, but means something different now | Use a different name AND migrate via CF-03 |

---

*End of `CONFIGURATION.md`.*
