# Using the `ConfigurableTestWebsiteMonitor`

The Configurable Test Website Monitor generates predictable website monitoring data without requiring external web monitoring services. It simulates the discovery of cookies, scripts, images, and other web assets with configurable vendor assignments and consent scenarios. It does _not_ interact with Compass and exercise any of the asset "vendor identification" functionality of the real Website monitor service: it bypasses this step and assigns configured vendor IDs directly.

It supports a number of configuration parameters, including what kinds of assets to generate, what vendors the assets should belong to, and how many assets to generate. This is particularly useful when trying to generate large amounts of website monitor staged resources for performance testing in local environments. 

The ConfigurableTestWebsiteMonitor is only available when development mode is enabled (`FIDES__DEV_MODE=true`)


## Complete Configuration Example

This is a complete configuration example, all the parameters are explained in the Configuration Options section.

```json
{
  "datasource_params": {
    "num_cookies": 20,
    "num_javascript_requests": 15,
    "num_image_requests": 8,
    "num_iframe_requests": 3,
    "num_browser_requests": 5,
    "test_locations": ["us", "uk", "de"],
    "test_pages": [
      "https://example.com/",
      "https://example.com/about",
      "https://example.com/contact"
    ],
    "test_domains": [
      "example.com",
      "analytics.example.com",
      "cdn.example.com",
      "ads.testsite.com"
    ],
    "cookie_names": [
      "_ga",
      "_gid",
      "session_id",
      "user_prefs",
      "analytics"
    ],
    "cookie_values": [
      "GA1.2.123456789",
      "GA1.2.987654321",
      "abc123def456",
      "theme=dark",
      "enabled=true"
    ],
    "consent_granted_percentage": 0.75,
    "consent_denied_percentage": 0.1,
    "consent_cmp_error_percentage": 0.05,
    "vendor_match_percentage": 0.6,
    "test_vendor_ids": [
      "fds.1047",
      "fds.1053",
      "fds.1046",
      "gvl.755",
      "fds.1064"
    ],
    "locations": ["us"],
    "exclude_domains": [],
    "sitemap_url": null,
    "experiences_data": {
      "https://example.com/": {
        "us": {
          "notices": [
            {
              "notice_key": "data_sales_and_sharing",
              "data_uses": ["marketing.advertising.first_party.targeted", "marketing.advertising.third_party.targeted"],
              "mechanism": "opt_out"
            }
          ],
          "is_tcf_experience": false
        }
      }
    }
  }
}
```

## Setup Steps 

Because this is a testing utility, it cannot be configured through our usual integrations menu in the Admin UI. Instead, you have to rely on API calls to set up the monitor. Once you've successfully executed the `ConfigurableTestWebsiteMonitor`, then you will be able to see the results on the Admin UI Action Center as usual.

### Step 1: Create Test Website Connection

**Endpoint:** `PATCH /api/v1/connection`

**Request Body:**
```json
[
  {
    "name": "Test Website Connection",
    "key": "test_website_connection",
    "connection_type": "test_website",
    "access": "write",
    "disabled": false,
    "description": "Connection for configurable test website monitor",
    "secrets": {
      "url": "https://ethyca.com"
    }
  }
]
```

The url field in secrets won't actually be used but it's needed because the WebsiteMonitor expects a value in that field. 

### Step 2: Create Discovery Monitor

**Endpoint:** `PUT /api/v1/plus/discovery-monitor`

#### Option A: Use Default Parameters

To use all default configuration values, omit the datasource_params field and use the request body below (the connection_config_key is the key returned in the response of the request from Step 1):

```json
{
  "name": "Default Test Website Monitor",
  "connection_config_key": "test_website_connection",
  "enabled": true
}
```

#### Option B: Override Specific Parameters

To customize the monitor behavior, add a datasource_params object with the parameters you want to override. You only need to specify the parameters you want to change - any omitted parameters will use their default values. The configuration options section lists all available configuration parameters. 

**Example - Override Asset Counts:**
```json
{
  "name": "Custom Asset Count Monitor",
  "connection_config_key": "test_website_connection",
  "enabled": true,
  "datasource_params": {
    "num_cookies": 25,
    "num_javascript_requests": 20
  }
}
```

**Example - Override Vendor Configuration:**
```json
{
  "name": "High Vendor Match Monitor",
  "connection_config_key": "test_website_connection",
  "enabled": true,
  "datasource_params": {
    "vendor_match_percentage": 0.8,
    "test_vendor_ids": [
      "fds.1047",
      "fds.1053",
      "gvl.755"
    ]
  }
}
```

### Step 3: Execute Monitor

**Endpoint:** `POST /api/v1/plus/discovery-monitor/{monitor_id}/execute`

You'll need to use the monitor key from the response on Step 2.

### Step 4: View results

You can now go the Admin UI Action center to view the results. 

## Configuration Options

The configurable test website monitor supports extensive customization through the datasource_params field. All parameters are optional and will use defaults if not specified.

### Test Environment Configuration

Defines the simulated website environment:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `test_locations` | array | `["us", "uk", "de"]` | Geographic locations for consent simulation |
| `test_pages` | array | `["https://example.com/", "https://example.com/about", "https://example.com/contact"]` | Pages where assets are found |
| `test_domains` | array | `["example.com", "analytics.example.com", "cdn.example.com", "ads.testsite.com"]` | Domains used in generated assets |
| `cookie_names` | array | `["_ga", "_gid", "session_id", "user_prefs", "analytics"]` | Sample cookie names to use |
| `cookie_values` | array | `["GA1.2.123456789", "GA1.2.987654321", "abc123def456", "theme=dark", "enabled=true"]` | Sample cookie values to use |

**Example:**
```json
{
  "test_locations": ["us", "uk", "de", "fr"],
  "test_pages": [
    "https://example.com/",
    "https://example.com/products",
    "https://example.com/about",
    "https://example.com/contact"
  ],
  "test_domains": [
    "example.com",
    "cdn.example.com",
    "analytics.example.com",
    "ads.partner.com"
  ],
  "cookie_names": [
    "_ga",
    "_gid",
    "sessionId",
    "user_prefs"
  ],
  "cookie_values": [
    "sessionId_abc123",
    "userId_xyz789",
    "preferences_enabled"
  ]
}
```

### Asset Generation Configuration

Controls how many and what types of assets are generated:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `num_cookies` | integer | 15 | Number of cookie assets to generate |
| `num_javascript_requests` | integer | 10 | Number of JavaScript request assets |
| `num_image_requests` | integer | 5 | Number of image request assets |
| `num_iframe_requests` | integer | 3 | Number of iframe request assets |
| `num_browser_requests` | integer | 8 | Number of other browser request assets |

**Example:**
```json
{
  "num_cookies": 25,
  "num_javascript_requests": 15,
  "num_image_requests": 8,
  "num_iframe_requests": 4,
  "num_browser_requests": 5
}
```

### Vendor Assignment Configuration

Controls how vendor IDs are assigned to generated assets:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `vendor_match_percentage` | float | 0.6 | Percentage of assets that get vendor IDs (0.0 to 1.0) |
| `test_vendor_ids` | array | `["fds.1047", "fds.1053", "fds.1046", "gvl.755", "fds.1064"]` | Available vendor IDs for assignment |

**Example:**
```json
{
  "vendor_match_percentage": 0.7,
  "test_vendor_ids": [
    "fds.1047",
    "fds.1053",
    "fds.1046",
    "gvl.755",
    "fds.1064",
    "gvl.50",
    "fds.2001"
  ]
}
```

The default test_vendor_ids represent common vendors:

| Vendor ID | Vendor Name |
|-----------|-------------|
| `fds.1047` | Google Analytics |
| `fds.1053` | HubSpot |
| `fds.1046` | Google Tag Manager |
| `gvl.755` | Google Ads |
| `fds.1064` | LinkedIn Corporation |

The monitor uses deterministic hashing to assign vendor IDs based on domain patterns and asset characteristics. First, it determines which assets receive vendor IDs using the `vendor_match_percentage`. Then it applies pattern-matching rules:

- Domains containing "analytics" or "google" → Google Analytics/Ads/Tag Manager vendors
- Domains containing "ads" or "cdn" → Google Ads/Tag Manager vendors  
- Domains containing "hubspot" or "hs" → HubSpot vendor
- Domains containing "linkedin" → LinkedIn vendor

Assets that don't match specific patterns are assigned random vendors from the configured list. This approach ensures consistent, reproducible vendor assignments across test runs while simulating realistic vendor distribution patterns.

### Consent Configuration

Controls how consent states are assigned to generated assets:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `consent_granted_percentage` | float | 0.75 | Percentage of assets generated with granted consent (0.0 to 1.0) |
| `consent_denied_percentage` | float | 0.1 | Percentage of assets generated with denied consent (0.0 to 1.0) |
| `consent_cmp_error_percentage` | float | 0.05 | Percentage of assets generated with cmp_error consent (0.0 to 1.0) |

**Note:** Assets not assigned granted, denied, or cmp_error consent will have "default" consent.

### Experience Data Configuration

Controls the privacy experiences data used for calculating consent statuses.

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `experiences_data` | object | Default experiences data | JSON dictionary containing privacy experience data for consent notice mapping |

The `experiences_data` parameter allows you to provide custom privacy experience data that will be used as the basis to calculate the resource's consent status. If not provided, the monitor will use `DEFAULT_EXPERIENCES_DATA` data defined in `src/fidesplus/api/schemas/monitor_params.py`.

**Example experiences data:**
```json
{
  "experiences_data": {
    "https://example.com/": {
      "us": {
        "notices": [
          {
            "notice_key": "data_sales_and_sharing",
            "data_uses": ["marketing.advertising.first_party.targeted", "marketing.advertising.third_party.targeted"],
            "mechanism": "opt_out"
          }
        ],
        "is_tcf_experience": false
      },
      "eu": {
        "notices": [
          {
            "notice_key": "analytics_tracking",
            "data_uses": ["analytics"],
            "mechanism": "opt_in"
          },
          {
            "notice_key": "targeted_advertising",
            "data_uses": ["marketing.advertising.first_party.targeted"],
            "mechanism": "opt_in"
          }
        ],
        "is_tcf_experience": true
      }
    },
    "https://example.com/about": {
      "us": {
        "notices": [
          {
            "notice_key": "functional_storage",
            "data_uses": ["functional.storage"],
            "mechanism": "notice_only"
          }
        ],
        "is_tcf_experience": false
      }
    }
  }
}
```