Metadata-Version: 2.4
Name: sentrix
Version: 0.2.0
Summary: Red-team, eval, and monitor your LLMs. Security-first, Python-native.
Project-URL: Homepage, https://github.com/pinexai/sentrix
Project-URL: Documentation, https://pinexai.github.io/sentrix
Project-URL: Repository, https://github.com/pinexai/sentrix
Project-URL: Issues, https://github.com/pinexai/sentrix/issues
License: MIT
Keywords: ai,eval,jailbreak,llm,observability,red-team,safety,security
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Security
Requires-Python: >=3.10
Provides-Extra: eval
Requires-Dist: jsonschema>=4.0; extra == 'eval'
Provides-Extra: full
Requires-Dist: fastapi>=0.100; extra == 'full'
Requires-Dist: jsonschema>=4.0; extra == 'full'
Requires-Dist: sentence-transformers>=2.0; extra == 'full'
Requires-Dist: uvicorn>=0.20; extra == 'full'
Requires-Dist: websockets>=11.0; extra == 'full'
Provides-Extra: server
Requires-Dist: fastapi>=0.100; extra == 'server'
Requires-Dist: uvicorn>=0.20; extra == 'server'
Requires-Dist: websockets>=11.0; extra == 'server'
Description-Content-Type: text/markdown

# sentrix — LLM Security Testing

<p align="center">
  <a href="https://pypi.org/project/sentrix/"><img src="https://img.shields.io/pypi/v/sentrix?color=blueviolet" alt="PyPI"></a>
  <a href="https://pypi.org/project/sentrix/"><img src="https://img.shields.io/pypi/pyversions/sentrix?color=blueviolet" alt="Python"></a>
  <a href="https://github.com/pinexai/sentrix/actions/workflows/tests.yml"><img src="https://img.shields.io/github/actions/workflow/status/pinexai/sentrix/tests.yml?label=tests" alt="Tests"></a>
  <a href="https://github.com/pinexai/sentrix/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blueviolet" alt="MIT license"></a>
  <img src="https://img.shields.io/badge/zero-dependencies-brightgreen" alt="zero deps">
</p>

<p align="center">
  <b>Red-team, fingerprint, and monitor your LLMs — pure Python, zero config.</b><br>
  Find vulnerabilities before your users do.
</p>

<p align="center">
  <a href="https://pinexai.github.io/sentrix/">Documentation</a> ·
  <a href="https://pinexai.github.io/sentrix/quickstart/">Quick Start</a> ·
  <a href="https://pinexai.github.io/sentrix/guard/">Red Teaming</a> ·
  <a href="https://pinexai.github.io/sentrix/fingerprint/">Attack Heatmap</a> ·
  <a href="https://github.com/pinexai/sentrix/issues">Issues</a>
</p>

---

## What is sentrix?

`sentrix` is a Python-native LLM security suite. In one `pip install`, you get automated red teaming, vulnerability fingerprinting across models, adversarial test generation, compliance reporting, and production monitoring — with a local SQLite store and a built-in dashboard. No YAML. No Node.js.

**Here's what the attack heatmap looks like:**

<p align="center">
  <img src="https://raw.githubusercontent.com/pinexai/sentrix/main/docs/images/heatmap.svg" alt="sentrix attack heatmap — vulnerability matrix across models and attack plugins" width="720">
</p>

**And the web dashboard:**

<p align="center">
  <img src="https://raw.githubusercontent.com/pinexai/sentrix/main/docs/images/dashboard.svg" alt="sentrix web dashboard — 7-tab real-time security monitoring" width="760">
</p>

**Red team report from the CLI:**

<p align="center">
  <img src="https://raw.githubusercontent.com/pinexai/sentrix/main/docs/images/red-team-report.svg" alt="sentrix red team report output" width="680">
</p>

---

## Quick Start

```bash
pip install sentrix
```

```python
import sentrix

sentrix.init()  # enable SQLite persistence + SDK cost tracking

def my_chatbot(prompt: str) -> str:
    return call_llm(prompt)

# Red team your chatbot
report = sentrix.red_team(my_chatbot, plugins=["jailbreak", "pii", "harmful"])
report.summary()
```

Or from the CLI:

```bash
sentrix scan myapp:chatbot --plugins jailbreak,pii,harmful --n 20
sentrix serve                  # open dashboard at localhost:7234
```

---

## Three killer features

### 1. Auto-generate adversarial test cases

No manual test writing. sentrix reads your function's signature and docstring, calls an LLM, and generates N test cases covering jailbreaks, PII extraction, injection attacks, and normal usage.

```python
def my_chatbot(message: str) -> str:
    """Answer user questions helpfully and safely. Refuse harmful requests."""
    ...

ds = sentrix.auto_dataset(my_chatbot, n=50, focus="adversarial")
# → 50 test cases generated for free
print(f"Generated {len(ds)} test cases")
```

### 2. Attack heatmap across models

Run the full attack suite against multiple models simultaneously. Get a vulnerability fingerprint showing exactly which attack categories break which models — so you can pick the cheapest safe option.

```python
fp = sentrix.guard.fingerprint({
    "gpt-4o-mini": gpt_fn,
    "claude-haiku": claude_fn,
    "llama-3":     llama_fn,
}, plugins=["jailbreak", "pii", "harmful", "hallucination", "injection"])

fp.heatmap()
print(f"Safest model: {fp.safest_model()}")
print(f"Most vulnerable: {fp.most_vulnerable_model()}")
```

### 3. Git-aware CI security gates

Every scan is tagged with the git commit SHA. Block PRs if the vulnerability rate regresses vs. `main`.

```bash
sentrix scan myapp:chatbot --git-compare main --fail-on-regression
# → exits 1 if vuln rate increased by >5% vs main branch
# → writes summary to $GITHUB_STEP_SUMMARY
```

```yaml
# .github/workflows/security.yml
- run: sentrix scan myapp:chatbot --git-compare origin/main --fail-on-regression
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
```

---

## Attack plugins

| Plugin | What it probes |
|---|---|
| `jailbreak` | Role-play overrides, DAN variants, persona jailbreaks |
| `pii` | PII extraction, system prompt leakage, training data fishing |
| `harmful` | Dangerous information, CBRN, illegal activity requests |
| `hallucination` | False premises, leading questions, factual traps |
| `injection` | Indirect prompt injection via user-controlled data |
| `competitor` | Brand manipulation, competitor endorsement attacks |

All plugins ship 15–20 templates each. Community plugins via `sentrix plugin install <name>`.

---

## Evaluation & monitoring

```python
# Evaluate quality with 9 built-in scorers
ds = sentrix.dataset("qa-suite")
ds.add(input="What is 2+2?", expected_output="4")

exp = sentrix.experiment(
    "math-eval",
    dataset=ds,
    fn=my_chatbot,
    scorers=[sentrix.scorers.exact_match, sentrix.scorers.no_pii],
)
results = exp.run(pass_threshold=0.8)
results.summary()

# Compare models — Pareto frontier included
comparison = sentrix.compare_models(
    models={"gpt-4o-mini": gpt_fn, "claude-haiku": claude_fn},
    dataset=ds,
    scorers=[sentrix.scorers.llm_judge(criteria="accuracy")],
)
comparison.summary()  # → shows Pareto frontier + best value model

# Production tracing
with sentrix.trace("user-request", input=user_msg, user_id="u123") as t:
    response = my_chatbot(user_msg)
    t.output = response
```

---

## Compliance reports

Generate audit-ready reports mapped to OWASP LLM Top 10, NIST AI RMF, EU AI Act, and SOC2 — automatically evidence-linked to your red team scan results.

```bash
sentrix compliance --framework owasp_llm_top10 --output report.html
sentrix compliance --framework eu_ai_act --output audit.html
```

---

## Supply chain & RAG security

Scan your RAG document corpus for poisoned inputs, PII leakage, and system prompt tampering — zero LLM calls required, pure regex pattern matching.

```python
from sentrix.guard.rag_scanner import scan_rag

report = scan_rag(
    documents=my_docs,
    system_prompt=my_system_prompt,
    baseline_hash="abc123...",   # tamper detection
)
report.summary()
```

---

## Why sentrix over promptfoo?

| | **sentrix** | promptfoo |
|---|---|---|
| Language | **Python** (pip install) | TypeScript (npm install) |
| Configuration | **Zero config** | YAML required |
| Attack heatmap across models | **✅** | ❌ |
| Auto test generation from fn signature | **✅** | ❌ |
| Git-aware regression tracking | **✅** | ❌ |
| Cost tracking per scan | **✅** | ❌ |
| Production monitoring + tracing | **✅** | ❌ |
| RAG supply chain security | **✅** | ❌ |
| Human review + annotation queue | **✅** | ❌ |
| Compliance reports (OWASP / NIST / EU AI Act) | **✅** | ❌ |
| Community plugin ecosystem | **✅** | Limited |
| Offline / privacy mode (Ollama) | **✅** | ❌ |
| Local SQLite — no external backend | **✅** | ❌ |
| Built-in web dashboard | **✅** | Limited |

---

## Install options

```bash
pip install sentrix              # core — zero required dependencies
pip install sentrix[server]      # + FastAPI dashboard (sentrix serve)
pip install sentrix[eval]        # + JSON schema validation scorer
pip install sentrix[full]        # everything
```

**LLM providers** — install only what you use:

```bash
pip install openai               # for OpenAI models
pip install anthropic            # for Claude models
pip install google-generativeai  # for Gemini models
# offline: ollama pull llama3    # no API key needed
```

---

## Full CLI reference

```bash
# Security scanning
sentrix scan myapp:chatbot                              # red team
sentrix scan myapp:chatbot --plugins all --n 50         # full scan
sentrix scan myapp:chatbot --git-compare main           # + regression gate
sentrix fingerprint myapp:gpt_fn myapp:claude_fn        # attack heatmap

# Test generation
sentrix auto-dataset myapp:chatbot --n 50 --focus adversarial

# Evaluation
sentrix eval run experiment.py --fail-below 0.8

# Security for agents & RAG
sentrix scan-agent myapp:my_agent
sentrix scan-rag --docs ./data/ --system-prompt prompt.txt

# Compliance
sentrix compliance --framework owasp_llm_top10 --output report.html

# Monitoring
sentrix monitor watch myapp:chatbot --interval 60 --webhook $SLACK_URL
sentrix monitor drift --baseline my-eval --window 24

# Plugin ecosystem
sentrix plugin list
sentrix plugin install advanced-jailbreak

# Dashboard & info
sentrix serve                                          # open at :7234
sentrix history                                        # past scans
sentrix costs --days 7                                 # cost breakdown
```

---

## Learn more

- [Quick Start](https://pinexai.github.io/sentrix/quickstart/)
- [Red Teaming Guide](https://pinexai.github.io/sentrix/guard/)
- [Attack Heatmap](https://pinexai.github.io/sentrix/fingerprint/)
- [Auto Test Generation](https://pinexai.github.io/sentrix/auto-dataset/)
- [Evaluation Framework](https://pinexai.github.io/sentrix/eval/)
- [Production Monitoring](https://pinexai.github.io/sentrix/monitor/)
- [CI/CD Integration](https://pinexai.github.io/sentrix/ci/)
- [Dashboard Guide](https://pinexai.github.io/sentrix/dashboard/)

---

## Contributing

Issues and PRs welcome. See [github.com/pinexai/sentrix](https://github.com/pinexai/sentrix).

---

<p align="center">MIT license · Built by <a href="https://github.com/pinexai">pinexai</a></p>
