# Environment Seeding

The seed module provides an API and tooling for configuring Fidesplus
environments with sample data, integrations, and default settings. It is
used to stand up consistent demo, test, and benchmark environments.

## What Gets Seeded

The seed API (`POST /api/v1/plus/seed`) can configure any combination of:

- **Sample resources** -- systems, datasets, and policies from manifest files
  in one or more *sample data sources* (see below)
- **Messaging** -- Mailgun as the default messaging service
- **Storage** -- S3 as the default storage backend
- **Linked connections** -- sample database connections linked to systems
- **Consent notices and experiences** -- default privacy notices enabled
- **Compass vendors** -- default vendor systems created
- **Email templates** -- rich HTML email templates applied
- **Discovery monitors** -- BigQuery, website, Okta, and Entra monitors created and optionally executed. BigQuery is always seeded with LLM-based classification. Okta and Entra accept an `enable_llm` flag (default true) that, combined with the shared `llm_model_override`, controls LLM-based context classification at seed time — set it to `false` to opt out per monitor.

## Sample Data Sources

A *sample data source* is a named, self-contained data bundle under
`data/seed/sample_data/<name>/`. Sources are additive and composable —
data tasks accept a list of source names and load from each in order:

```jsonc
// POST /api/v1/plus/seed
{
  "tasks": { "sample_resources": true },
  "sample_resources": { "sources": ["default", "publisher"] }
}
```

When no `sample_resources` config is provided, the `default` source is
seeded (the pre-existing standard sample data). List available sources and
their contents with `GET /api/v1/plus/seed/sample-data`.

Sources are distinct from seed *profiles* (`secret_profile`): a profile is
a stored seed request payload (in 1Password) that composes task flags and
per-task source selections; a source is just data on disk — no credentials,
no external configuration. See `sample_data.py` for the directory layout.

## Commands

### Default Dev Session (with automatic seeding)

```bash
nox -s "dev(slim)"
```

The default dev session starts services in the background (detached mode),
validates your `.env` against the 1Password reference, and invokes the
seed API automatically (best-effort). If seeding fails — e.g., because
1Password CLI is not configured — a warning is printed but the
environment stays up and usable.

To skip seeding and `.env` validation entirely:

```bash
nox -s "dev(slim)" -- no-seed
```

### Full Demo Environment

```bash
nox -s "build(slim)" -- dev   # build the local slim image (required before first run)
nox -s "dev(slim)" -- demo
```

This is the primary command for running a local demo environment. It:

1. Validates environment variable prerequisites (see below)
2. Builds and starts all services (webserver, workers, sample databases,
   admin UI, privacy center, sample app)
3. Waits for all services to be healthy
4. Automatically invokes the seed API to configure the environment
5. Waits for Ctrl+C, then tears down containers

To start the demo environment without seeding:

```bash
nox -s "dev(slim)" -- demo no-seed
```

> **⚠️ Docker Desktop must have at least 12 GB of memory allocated.**
> The Admin UI runs a memory-intensive Next.js production build on
> container startup. If Docker doesn't have enough memory, the build
> will fail silently or OOM-kill the container.

> **Note: Admin UI and Privacy Center images**
>
> These frontend images do **not** hot-reload or rebuild automatically. If
> you make changes to the frontend code in the `fides` repo, you must
> manually rebuild:
>
> ```bash
> docker compose -f docker-compose.yml -f demo/docker-compose.demo.yml build fides-admin-ui fides-privacy-center
> ```
>
> The Admin UI also runs a Next.js production build on container startup
> that takes a few minutes. It will not be accessible until the full
> build completes (look for `✓ Ready in Xms` in its logs):
>
> ```bash
> docker compose -f docker-compose.yml -f demo/docker-compose.demo.yml logs -f fides-admin-ui
> ```

### Standalone Seed

```bash
nox -s seed
```

Use this to seed an **already-running** Fidesplus environment. This is
useful when:

- You restarted containers and need to re-seed
- You want more control over your local development environment than the full demo environment allows for, e.g. you don't need the entire set of demo services running locally (privacy center, sample app, etc.)
- You're iterating on seed configuration

The seed configuration (including any required secrets) is read from
1Password automatically.

## Prerequisites

All three prerequisites below must be satisfied before running either
command.

### 1. 1Password CLI

The `op` CLI must be installed and authenticated. Seed configuration and
the reference `.env` are both stored in 1Password.

**Install:**
<https://developer.1password.com/docs/cli/get-started/>

**Authenticate:**
```bash
op signin
```

**Verify:**
```bash
op account list
```

### 2. Monorepo layout (no sibling fides repo)

Fides core (`src/fides/`) and clients (`clients/`) live in this repository.
The default dev stack hot-reloads backend code via `docker-compose.yml` — you
do **not** need a separate fides checkout or `FIDES_REPO_PATH` for backend dev.

Demo frontend images still require monorepo Docker build targets under
`clients/` (see Phase 5 in the dev migration plan). Until those land, demo
mode sets `FIDES_REPO_PATH` to the repo root automatically.

### 3. Required Environment Variables

Copy the reference `.env` from 1Password into the **fidesplus repo root**:

```bash
op item get fidesplus-demo-dot-env \
  --vault fidesplus-demo-seed \
  --fields notesPlain > .env
```

That's it -- Docker Compose loads this file (`env_file: - .env`) into
every service container automatically, and `.env` is already in
`.gitignore` so it will never be committed.

Both commands run a pre-flight check that validates every variable in the
1Password reference is either present in your local `.env` or exported in
your shell. If any are missing, the session will error with a list of
the missing variables.

> **Note:** A `.env` file is not strictly required. The pre-flight check
> only verifies that the variables are *set*, not how. Shell exports,
> direnv, a secrets manager, or any other mechanism that makes the
> variables available to Docker Compose will work just as well.

## Troubleshooting

### Missing environment variables

```
Environment variable prerequisites not met!

The following required environment variables are not set:
  - SOME_VARIABLE
  ...
```

**Fix:** Fetch the reference `.env` from 1Password (see above) and save
it to the repo root. If you are running `nox -s seed` against
already-running containers, you must restart them after updating `.env`:

```bash
docker compose restart
```

### 1Password CLI not available

```
1Password CLI (op) is not installed or not authenticated.
```

**Fix:** Install the `op` CLI and run `op signin`. See the Prerequisites
section above.

### Seed API connection error

```
Failed to connect to seed API
```

**Fix:** Make sure the Fidesplus webserver is running and healthy. For
demo mode this happens automatically; for standalone seed, start the
environment first:

```bash
nox -s "dev(slim)"
```

### FIDES_REPO_PATH invalid (legacy fides-pkg only)

```
Invalid fides repository path
```

**Fix:** Omit the `fides-pkg` posarg for monorepo dev — `src/fides/` reloads
automatically. If you intentionally use the legacy sibling-repo overlay,
set `FIDES_REPO_PATH` to a valid external fides checkout.
