Metadata-Version: 2.4
Name: llm-input-harderning
Version: 1.0.1
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Rust
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Text Processing :: Linguistic
Requires-Dist: ftfy>=6.2.0 ; extra == 'ftfy'
Requires-Dist: confusable-homoglyphs>=3.3.1 ; extra == 'security'
Provides-Extra: ftfy
Provides-Extra: security
License-File: LICENSE
Summary: Deterministic Unicode-aware input hardening for LLM applications.
Keywords: llm,security,unicode,sanitization,prompting
Home-Page: https://github.com/davidwebstar34/llm-input-harderning
Author: David Webster
Maintainer: David Webster
License: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/davidwebstar34/llm-input-harderning/releases
Project-URL: Homepage, https://github.com/davidwebstar34/llm-input-harderning
Project-URL: Issues, https://github.com/davidwebstar34/llm-input-harderning/issues
Project-URL: Source, https://github.com/davidwebstar34/llm-input-harderning

# llm-input-harderning

Deterministic Unicode-aware input hardening for LLM applications.

`llm-input-harderning` sits between untrusted text and your prompt or template layer. It normalizes Unicode, removes invisible or control characters that destabilize prompts, emits structured reports for logging and enforcement, and can add confusable mixed-script signals when you want stricter checks.

It is not prompt injection prevention. It addresses text-integrity problems, not model intent.

This repository keeps docs intentionally lightweight. The README is the main reference.

## Install

```bash
pip install llm-input-harderning
```

Optional extras:

- `pip install "llm-input-harderning[security]"` for the `confusable_homoglyphs` backend
- `pip install "llm-input-harderning[ftfy]"` for pre-sanitize text repair

## Quickstart

```python
from llm_input_hardener import sanitize

clean_text, report = sanitize(user_text, policy="balanced_chat")
```

Recursive JSON sanitization:

```python
from llm_input_hardener import sanitize_json

clean_payload, report = sanitize_json(payload, policy="balanced_chat")
```

Enforcement-oriented usage:

```python
from llm_input_hardener import sanitize_and_decide

clean_text, report, decision = sanitize_and_decide(
    user_text,
    sanitize_policy="strict_exec",
    confusables_backend="confusable_homoglyphs",
)
```

## Policies

| Policy | Normalization | Default-Ignorables | Whitespace tidy | Typical use |
| --- | --- | --- | --- | --- |
| `preserve` | NFC | preserve | no | low-friction logging and observability |
| `balanced_chat` | NFC | preserve | yes | general chat and prompt input |
| `strict_exec` | NFKC | strip | yes | tool arguments and execution-adjacent paths |
| `code_mode` | NFC | strip | no | code snippets and syntax-sensitive text |

Backward-compatible aliases are still accepted: `balanced`, `strict`, and `code`.

## Scope

Use this library for:

- user text before prompt interpolation
- retrieved text before it reaches the model
- tool and function argument strings
- JSON payloads that contain prompt-bearing string leaves

Do not use it as a substitute for:

- system prompt design
- tool sandboxing and allowlists
- output validation
- human review for high-risk actions

## CLI

```bash
llm-input-harderning sanitize --text "abc‮def"
llm-input-harderning inspect --text "abc‮def"
llm-input-harderning decide --text "abc‮def"
```

## Development

```bash
uv sync --all-extras
uv run maturin develop --release
uv run pytest -q
```

## Benchmarks And Bakeoff

The repo has two different evaluation paths.

- `bench/run_benchmarks.py` is a timing harness. It measures sanitizer latency and throughput on fixed synthetic inputs.
- `bench/run_bakeoff.py` is an evaluation harness. It runs labeled corpus cases, checks expected detections and postconditions, and computes precision, recall, benign-change rate, and stability signals.

Use the Make targets below:

```bash
make bench
make bench-plots

make bakeoff-smoke
make bakeoff-smoke-assert

make bakeoff-full
make bakeoff-full-assert
make bakeoff-diff
make bakeoff-diff-assert
```

What they mean:

- `bakeoff-smoke` is the fast local check. It evaluates only `llm-input-harderning` and writes results to `bench/results/bakeoff_smoke_latest.json`.
- `bakeoff-full` is the heavyweight comparison run. It installs `llm-guard` and `confusable-homoglyphs`, then writes comparable results to `bench/results/bakeoff_latest.json`.
- `bakeoff-diff` compares the current full run to `bench/results/bakeoff_baseline_v1.0.0.json`.

Generated reports stay under `bench/results/`.

## Repo Notes

- `CONTRIBUTING.md` covers contribution mechanics.
- `SECURITY.md` covers reporting and threat assumptions.
- `SCOPE.md` explains what the project is and is not trying to solve.

