# Privacy Assessment Service Architecture

Service layer for privacy assessment lifecycle: template-driven generation, LLM-powered analysis, answer versioning, risk evaluation, re-evaluation, and PDF reporting.

## Overview

```
API Routes
    │
    ▼
PrivacyAssessmentService          PrivacyAssessmentTaskService
    │                                      │
    ├── Repository (data access)           ├── Celery tasks
    ├── LLM Executor (answer generation)   ├── Redis locks (concurrency)
    ├── RiskEvaluator (high-risk filter)   └── APScheduler (periodic re-eval)
    ├── ChangeDetector (re-evaluation)
    ├── SnapshotBuilder (context capture)
    ├── AnswerUtils (version persistence)
    ├── UserResolution (identity)
    └── PDFGenerator (reporting)
```

## Assessment Lifecycle

```
1. Template Selection
   Active AssessmentTemplate for requested type
        │
2. Declaration Discovery
   Find system declarations matching filters
        │
3. Assessment Creation
   PrivacyAssessment row + empty AssessmentAnswer per question
        │
4. Answer Generation (async Celery task)
   ├── Build PrivacyContext (system metadata, taxonomy, policies)
   ├── LLM generates answers per question (with confidence scores)
   ├── Persist as AnswerVersion (source=ai_analysis)
   └── Calculate completeness %
        │
5. SME Review
   ├── User edits in Admin UI (source=user_input)
   ├── Questionnaire chat (source=team_input)
   └── Each edit creates new AnswerVersion
        │
6. Re-evaluation (on context/template changes)
   ├── Snapshot diff detects material changes
   ├── LLM re-evaluates affected questions only
   └── New versions preserve prior answers
        │
7. Reporting
   └── PDF export (internal or external mode)
```

## Answer Versioning

Every answer modification creates an immutable `AnswerVersion` record. The `AssessmentAnswer.current_version_id` pointer tracks the active version.

**Sources** identify how an answer was produced:

| Source | Origin | Example |
|--------|--------|---------|
| `system` | Auto-populated from fides metadata | Data categories from system config |
| `ai_analysis` | LLM-generated during assessment | "The system processes employee health data..." |
| `user_input` | Manual edit in Admin UI | Reviewer corrects or approves |
| `team_input` | Questionnaire chat response | SME answer via Slack or in-app chat |

**Change types** track the nature of the edit:

| Type | Meaning |
|------|---------|
| `ai_generated` | Initial LLM generation or re-evaluation |
| `human_edited` | User or SME modified the answer |
| `approved` | Reviewer approved the answer |
| `rejected` | Reviewer rejected the answer |

**Revert** restores a previous version by copying it as a new version (history is append-only).

## Task System

Assessment generation runs asynchronously via Celery:

```
TaskService.queue_task()
    │
    ├── Redis lock (prevents concurrent tasks)
    ├── Pre-compute expected assessment count
    ├── Create PrivacyAssessmentTask record
    └── Dispatch Celery task
         │
         ▼
run_assessment_task()  (Celery worker)
    │
    ├── GENERATE:      service.generate_assessments()
    └── RE_EVALUATE:    service.re_evaluate_assessments()
         │
         ├── Yields one assessment at a time
         ├── Updates task progress after each
         └── Status: pending → in_processing → complete/error
```

**Scheduled re-evaluation** uses APScheduler with a configurable cron expression from `PrivacyAssessmentConfig`.

**Stuck task detection** runs periodically to mark abandoned tasks as errored.

## Risk Evaluation

`RiskEvaluator` determines whether a system/declaration pair is high-risk, used to filter which assessments to generate.

**Six trigger categories (A-F):**

| Trigger | Category | Examples |
|---------|----------|----------|
| A | Sensitive data categories | GDPR Art. 9/10 data, US state law sensitive data |
| B | High-risk data uses | Profiling, marketing, AI training, broad collect |
| C | Vulnerable data subjects | Employees, patients, job applicants |
| D | System attributes | DPA required, international transfers, profiling |
| E | Declaration attributes | Special category, third-party sharing, legitimate interests |
| F | Compound triggers | Behavior + advertising, location + analytics, financial + fraud |

Returns `RiskEvaluationResult` with matched triggers and overall risk determination.

## Re-evaluation & Change Detection

Assessments capture a **context snapshot** at generation time containing the full privacy context and template state. When re-evaluation runs:

```
Current State          Snapshot (at generation)
     │                        │
     └────── compute_diff() ──┘
                  │
                  ▼
          AssessmentDiff
          ├── context_changes (system metadata changed)
          └── template_changes (questions added/changed/removed)
                  │
                  ▼
          detect_material_changes()
          ├── Template changes → deterministic (always re-evaluate)
          └── Context changes → LLM judges materiality
                  │
                  ▼
          Re-evaluate affected questions only
```

Only questions affected by material changes are re-evaluated. The snapshot is updated after re-evaluation.

## LLM Integration

### Privacy Context Builder

Aggregates fides metadata into a structured `PrivacyContext` for LLM consumption:

```
PrivacyContextBuilder
    .include_sections(SYSTEMS, DATA_CATEGORIES, DATA_USES, ...)
    .filter_by_systems(["system_key"])
    .filter_by_declarations(["declaration_id"])
    .build(db)
    → PrivacyContext
```

Sections: `ORGANIZATION`, `DATA_CATEGORIES`, `DATA_USES`, `DATA_SUBJECTS`, `SYSTEMS`, `DATASETS`, `POLICIES`, `PRIVACY_NOTICES`, `CONNECTIONS`

### Assessment Executor

Generates answers for a set of questions given a template and privacy context:

```
PrivacyAssessmentExecutor
    .execute_for_declaration(declaration, context, questions)
    → DeclarationAssessmentResult
        ├── answers[] (text + confidence + evidence)
        └── risk_level
```

The executor formats a prompt with the template's legal reference, questions, guidance, and privacy context, then parses the LLM response into structured answers.

## Configuration

`PrivacyAssessmentConfigService` manages a singleton config row:

| Setting | Purpose |
|---------|---------|
| `slack_channel_name` | Default Slack channel for questionnaires |
| `reassessment_cron` | Cron expression for periodic re-evaluation |
| `tone_prompt` | Custom tone for questionnaire bot personality |
| `default_model` | LLM model override (default: `openrouter/anthropic/claude-opus-4-7`) |

## PDF Reporting

`generate_pdf()` renders an assessment as a downloadable PDF:

- **Internal mode**: Full details including evidence, version history, confidence scores
- **External mode**: Clean, signing-ready document for regulators

Uses Jinja2 HTML templates + WeasyPrint for rendering.

## Key Files

### Core Service

| File | Responsibility |
|------|---------------|
| `service.py` | Main orchestration: CRUD, generation, re-evaluation, answer management |
| `repository.py` | Data access returning domain entities (decoupled from ORM) |
| `entities.py` | Dataclass entities: AssessmentEntity, AssessmentTaskEntity, EvidenceItemEntity |
| `exceptions.py` | Domain exceptions: AssessmentTaskConflictError, TaskNotFoundError |

### Answer Management

| File | Responsibility |
|------|---------------|
| `answer_utils.py` | `persist_answer_version()` -- create version, update pointer, recalculate completeness |
| `user_resolution.py` | Resolve user identifiers to display names (trust hierarchy, anti-spoofing) |

### Task System

| File | Responsibility |
|------|---------------|
| `task_service.py` | Task CRUD, queuing, retry |
| `tasks.py` | Celery task definition (`run_assessment_task`) |
| `task_utils.py` | Redis locking, expected count pre-computation, scheduled jobs |

### Evaluation

| File | Responsibility |
|------|---------------|
| `risk_evaluator.py` | Six-trigger risk assessment for declarations |
| `change_detector.py` | Determine which questions need re-evaluation |
| `snapshot.py` | Build and diff context snapshots |

### LLM

| File | Responsibility |
|------|---------------|
| `llm/privacy_context.py` | PrivacyContextBuilder -- aggregate fides metadata for LLM |
| `llm/assessments/executor.py` | PrivacyAssessmentExecutor -- generate answers from template + context |
| `llm/prompt_builder.py` | Jinja2-based prompt construction |

### Supporting

| File | Responsibility |
|------|---------------|
| `config_service.py` | Singleton assessment configuration management |
| `assessment_types.py` | Active template type registry for API schema |
| `constants.py` | Default model, shared constants |
| `report/utils.py` | PDF generation (Jinja2 + WeasyPrint) |
