Metadata-Version: 2.4
Name: toxpol-nlp
Version: 0.1.1
Summary: NLP toolkit for toxicity and polarization research: synthetic datasets and detection algorithms
Author-email: SwkratisCS <swkratisgiannoutsos@gmail.com>
License-Expression: MIT
Project-URL: Repository, https://github.com/SwkratisCS/polarizedtrees
Keywords: toxicity,polarization,annotation,synthetic data,nlp,disagreement,crowdsourcing,demographics
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: numpy
Requires-Dist: pandas
Provides-Extra: ndfu
Requires-Dist: ndfu; extra == "ndfu"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"

# toxpol-nlp

NLP toolkit for **toxicity and polarization research**. Provides tools for synthetic dataset generation and polarization detection in human annotation studies.

## Install

```bash
pip install toxpol-nlp

# with nDFU support
pip install "toxpol-nlp[ndfu]"
```

## Tools

| Module | Description | Status |
|---|---|---|
| `toxpol.datagen` | Synthetic annotator pool with injected, ground-truth polarization | Stable |
| `toxpol.trees` | Polarized Trees detection algorithm | Coming soon |

## `toxpol.datagen`

Builds a pool of annotators with explicit demographic identities and generates annotation datasets where each dimension is assigned a role:

- **Polarizing** — splits annotators into opposing rating poles (toxic vs. civil)
- **Unimodal** — converges all annotators toward one rating range

The bias configuration is returned alongside the dataset as ground truth, enabling direct validation of detection algorithms.

```python
from toxpol.datagen import AnnotatorPool, DEFAULT_DIMENSIONS

pool = AnnotatorPool(DEFAULT_DIMENSIONS)
pool.summary()
# Active dimensions : ['gender', 'politics', 'age', 'education', 'orientation']
# Pool size         : 1620  (162 identities × 10 annotators each)

dataset, bias_config = pool.generate_dataset(
    n_texts=100,
    n_annotators_per_text=100,
    noise=0.1,
    polarizing_prob=0.7,
)
# dataset columns: text_id, annotator_id, <dimensions>, rating
# bias_config: ground-truth role and poles for each dimension
```

Full API documentation available on [GitHub](https://github.com/SwkratisCS/polarized-trees).
