# MediLink Gmail Pipeline - Fresh Setup Guide (Direct Service Account Access)

This guide provides step-by-step instructions for setting up the MediLink Gmail ingestion pipeline from scratch using the **direct service account access** approach.

## 🏗️ Architecture Overview

```
mariavidaud@gmail.com (Personal Gmail)
    ↓ (Gmail Forwarding Rule)
Service Account Gmail (medilink-gmail-orchestrator@PROJECT_ID.iam.gserviceaccount.com)
    ↓ (Workload Identity Federation)
Cloud Run Service → Gmail API → Pub/Sub → XP Daemon
```

## 📋 Prerequisites

- Google Cloud Project: `genial-analyzer-476721-f8`
- Google Cloud SDK installed and authenticated
- Python 3.11+ installed
- Access to Google Workspace Admin Console

## 🚀 Complete Setup Process

### Step 1: Run Comprehensive Setup Script

```powershell
cd cloud\orchestrator
.\setup_complete_pipeline.ps1 -ProjectId genial-analyzer-476721-f8
```

**What this script does:**
- ✅ Creates service account with proper IAM roles
- ✅ Enables required GCP APIs (Pub/Sub, Cloud Run, etc.)
- ✅ Creates Cloud Storage bucket
- ✅ Builds and deploys Cloud Run service
- ✅ Creates Pub/Sub topic and subscription
- ✅ Provides next steps for manual configuration

### Step 2: Create Service Account Gmail Account

**IMPORTANT:** The service account needs its own Gmail account to receive forwarded emails.

1. Go to [Gmail](https://gmail.com)
2. Click "Create account"
3. Choose "For work or my business"
4. Use this exact email address:
   ```
   medilink-gmail-orchestrator@genial-analyzer-476721-f8.iam.gserviceaccount.com
   ```
5. Complete the account creation process
6. **Keep this Gmail account active** - it's where forwarded emails will arrive

### Step 3: Set Up Email Forwarding

Forward emails from `mariavidaud@gmail.com` to the service account's Gmail account.

**Option A: Automatic Setup (Recommended)**
```powershell
# Follow the interactive prompts in GMAIL_FORWARDING_SETUP.md
# This will guide you through setting up forwarding rules
```

**Option B: Manual Setup**
1. Go to [Gmail Settings](https://mail.google.com/mail/u/0/#settings/fwdandpop)
2. In the "Forwarding" section, click "Add a forwarding address"
3. Enter: `medilink-gmail-orchestrator@genial-analyzer-476721-f8.iam.gserviceaccount.com`
4. Gmail will send a verification email to the service account
5. Go to the service account's Gmail inbox and click the verification link
6. Back in the original Gmail, select the forwarding address and choose "Forward a copy of incoming mail"

### Step 4: Register Gmail Watch

```powershell
cd ..\..\tools
.\setup_gmail_watch.ps1
```

**What this does:**
- ✅ Authenticates using Workload Identity Federation (no keys needed)
- ✅ Registers Gmail watch on the service account's inbox
- ✅ Creates Pub/Sub topic subscription for email notifications

### Step 5: Test the Pipeline

1. **Send a test email** to `mariavidaud@gmail.com`
2. **Monitor Cloud Run logs:**
   ```
   https://console.cloud.google.com/logs/viewer?project=genial-analyzer-476721-f8&resource=cloud_run_revision/service_name/medilink-gmail-orchestrator
   ```
3. **Check Pub/Sub metrics:**
   ```
   https://console.cloud.google.com/cloudpubsub/subscription/list?project=genial-analyzer-476721-f8
   ```

## 🔧 Troubleshooting

### Common Issues

**"Service account does not have Gmail API access"**
- Solution: Ensure the service account has `roles/gmail.modify` and `roles/gmail.readonly` IAM roles

**"Gmail watch registration failed"**
- Solution: Verify the service account has a Gmail account created
- Check that Workload Identity Federation is properly configured

**"No emails being processed"**
- Solution: Verify email forwarding is set up correctly
- Check that the service account's Gmail account exists and is accessible

### Verification Commands

**Check pipeline status:**
```powershell
.\tools\pipeline_status.ps1 -Action status
```

**Verify Gmail watch:**
```powershell
.\tools\setup_gmail_watch.ps1 -Force
```

**Test Pub/Sub connection:**
```powershell
.\tools\pipeline_status.ps1 -Action verify
```

## 📊 What Was Set Up

### GCP Resources Created
- **Service Account:** `medilink-gmail-orchestrator@genial-analyzer-476721-f8.iam.gserviceaccount.com`
- **Cloud Run Service:** `medilink-gmail-orchestrator`
- **Pub/Sub Topic:** `gmail-new-emails`
- **Pub/Sub Subscription:** `gmail-new-emails-sub`
- **Cloud Storage Bucket:** `medilink-gmail-staging`
- **Firestore Collections:** `queues`, `sync_state`

### IAM Roles Assigned
- `roles/storage.objectAdmin` - Cloud Storage access
- `roles/datastore.user` - Firestore access
- `roles/pubsub.publisher` - Pub/Sub publishing
- `roles/gmail.modify` - Gmail API access (you may need to add this)

### Authentication Method
- **Workload Identity Federation** - Direct ADC authentication
- **No service account keys** required
- **No domain-wide delegation** required

## 🎯 Key Differences from Old Setup

| Old Approach | New Approach |
|-------------|-------------|
| Domain-wide delegation | Direct service account access |
| Service account keys | Workload Identity Federation |
| User impersonation | Service account owns its Gmail |
| Complex OAuth flow | Simple ADC authentication |
| Security risk (keys) | Secure (no keys stored) |

## 📞 Support

If you encounter issues:

1. Check the troubleshooting section above
2. Run `./tools/pipeline_status.ps1 -Action status` for diagnostics
3. Review Cloud Run logs for error details
4. Verify all manual setup steps were completed

**The new direct service account access approach is simpler, more secure, and easier to maintain!** 🚀
















