Metadata-Version: 2.4
Name: pii-protector
Version: 2.2.1
Summary: Production-grade PII detection with multi-model ensemble
Author-email: tensoryug <jainsatyam26@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/sjain26/pii-guard
Keywords: pii,privacy,nlp,detection,presidio,data-protection
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Security
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Provides-Extra: presidio
Requires-Dist: presidio-analyzer>=2.2; extra == "presidio"
Requires-Dist: spacy>=3.7; extra == "presidio"
Provides-Extra: transformers
Requires-Dist: transformers>=4.40; extra == "transformers"
Requires-Dist: torch>=2.0; extra == "transformers"
Provides-Extra: full
Requires-Dist: presidio-analyzer>=2.2; extra == "full"
Requires-Dist: spacy>=3.7; extra == "full"
Requires-Dist: transformers>=4.40; extra == "full"
Requires-Dist: torch>=2.0; extra == "full"

# pii-protector

Production-grade PII (Personally Identifiable Information) detection library with a sequential multi-model ensemble. Designed for speed — 90%+ of texts are handled by the regex layer alone in under 3ms, with heavier NER models invoked only when needed.

## Architecture

```
Layer 1 — Regex                  always runs              ~0.3–3ms
    ↓  escalation score >= 3?
Layer 2 — Presidio + spaCy NER   names / orgs / loc       ~5ms
    ↓  confidence low or conflict?
Layer 3 — NER Transformer        high-accuracy names      ~20ms
    +
Layer 4 — PII Model              structured PII           ~15ms
```

Each layer decides whether the next one is needed. On clean structured text, only Layer 1 runs.

## Installation

```bash
# Regex layer only — zero dependencies
pip install pii-protector

# + Presidio / spaCy NER (Layer 2)
pip install "pii-protector[presidio]"
python -m spacy download en_core_web_lg

# + Transformer models (Layer 3 & 4)
pip install "pii-protector[transformers]"

# Full install (all layers)
pip install "pii-protector[full]"
python -m spacy download en_core_web_lg
```

## Quick Start

```python
from pii_detector import AdvancedPIIDetector

# Regex only (fastest, zero dependencies)
detector = AdvancedPIIDetector(use_presidio=False, use_transformers=False, use_pii_model=False)

results = detector.detect("My SSN is 123-45-6789 and email is john@example.com")
for entity in results:
    print(entity)
# {'entity_type': 'SSN',   'start': 10, 'end': 21, 'score': 0.95, 'text': '123-45-6789'}
# {'entity_type': 'EMAIL', 'start': 36, 'end': 52, 'score': 0.95, 'text': 'john@example.com'}
```

### Full ensemble (all layers)

```python
detector = AdvancedPIIDetector()

# Detailed analysis with risk score
analysis = detector.get_detailed_analysis("Dr. John Smith's Aadhaar is 1234 5678 9012")
print(analysis['risk_assessment'])
print(analysis['timing'])
```

### Batch detection

```python
texts = ["call me at +91-9876543210", "my PAN is ABCDE1234F", "nothing sensitive here"]
results = detector.detect_batch(texts)
```

### Risk assessment

```python
risk = detector.get_risk_assessment("SSN 123-45-6789, VISA 4111 1111 1111 1111")
# {'risk_level': 'CRITICAL', 'score': 0.95, 'high_value_pii': ['SSN', 'CREDIT_CARD']}
```

## CLI

```bash
# Basic scan
pii-detect "My credit card is 4111 1111 1111 1111"

# From file
pii-detect --file document.txt

# JSON output
pii-detect --json "Aadhaar: 1234 5678 9012"

# Detailed analysis
pii-detect --detailed "SSN 123-45-6789, John Smith, john@corp.com"

# Risk score only
pii-detect --risk "GSTIN 22ABCDE1234F1Z5, PAN ABCDE1234F"

# Disable heavy models for speed
pii-detect --no-presidio --no-transformer --no-pii-model "card: 4111111111111111"
```

## Supported PII Types

| Category | Types |
|---|---|
| Core | `EMAIL`, `PHONE`, `SSN`, `CREDIT_CARD`, `IP_ADDRESS`, `ADDRESS`, `PASSWORD`, `USERNAME` |
| India | `AADHAAR`, `PAN`, `GSTIN`, `UPI_ID`, `IFSC`, `VOTER_ID`, `MOBILE_IN`, `EPFO_UAN`, `DRIVING_LICENSE_IN`, `PASSPORT_IN`, `BANK_ACCOUNT_IN`, `PINCODE` |
| North America | `DRIVER_LICENSE`, `PASSPORT`, `BANK_ACCOUNT`, `TAX_ID`, `SIN`, `ZIP_CODE`, `POSTAL_CA` |
| NER | `NAME`, `ORGANIZATION`, `LOCATION`, `FACILITY` |
| Finance | `CREDIT_CARD`, `BANK_ACCOUNT`, `MONEY` |
| Dates & URLs | `DATE_OF_BIRTH`, `DATE`, `URL` |
| API Secrets | `AWS_ACCESS_KEY`, `OPENAI_API_KEY`, `GITHUB_TOKEN`, `GITLAB_TOKEN`, `STRIPE_KEY`, `JWT_TOKEN`, `BEARER_TOKEN`, `PRIVATE_KEY`, `DATABASE_URL`, `SLACK_TOKEN`, `DISCORD_TOKEN`, 20+ more |

## Configuration

```python
detector = AdvancedPIIDetector(
    use_presidio=True,           # Layer 2: spaCy NER via Presidio
    use_transformers=True,       # Layer 3: NER transformer
    use_pii_model=True,          # Layer 4: PII model
    confidence_threshold=0.5,   # Early-stop threshold
    spacy_model="en_core_web_lg",
    transformer_model="jainsatyam26/pii-ner-onnx",
    pii_model="jainsatyam26/pii-detector-onnx",
)
```

### Environment variable overrides

```bash
SPACY_MODEL_NAME=en_core_web_sm
TRANSFORMER_MODEL_NAME=jainsatyam26/pii-ner-onnx
PII_MODEL_NAME=jainsatyam26/pii-detector-onnx
SPACY_DATA_PATH=/path/to/models
```

## Models

| Layer | Model | What it detects |
|---|---|---|
| 2 | [`en_core_web_lg`](https://spacy.io/models/en) via Presidio | NAME, ORG, LOCATION |
| 3 | [`jainsatyam26/pii-ner-onnx`](https://huggingface.co/jainsatyam26/pii-ner-onnx) | High-accuracy person names |
| 4 | [`jainsatyam26/pii-detector-onnx`](https://huggingface.co/jainsatyam26/pii-detector-onnx) | PHONE, EMAIL, SSN, DOB, ADDRESS |

ONNX-optimized versions:
- [`jainsatyam26/pii-ner-onnx`](https://huggingface.co/jainsatyam26/pii-ner-onnx) — NER model
- [`jainsatyam26/pii-detector-onnx`](https://huggingface.co/jainsatyam26/pii-detector-onnx) — PII structured model

## License

MIT — tensoryug
