Metadata-Version: 2.4
Name: xrulefit
Version: 0.1.0
Summary: RuleFit: interpretable rule ensembles built on gradient-boosted trees.
Project-URL: Homepage, https://github.com/BastiaanvG/xrulefit
Project-URL: Repository, https://github.com/BastiaanvG/xrulefit
Project-URL: Documentation, https://bastiaanvg.github.io/xrulefit/
Project-URL: Issues, https://github.com/BastiaanvG/xrulefit/issues
Author-email: Bastiaan van Gaalen <bastiaan.vangaalen@gmail.com>
License-Expression: BSD-3-Clause
License-File: LICENSE
Keywords: gradient-boosting,interpretable-machine-learning,rule-ensemble,rulefit,scikit-learn
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: narwhals>=1.21
Requires-Dist: numpy>=1.26
Requires-Dist: scikit-learn>=1.9
Requires-Dist: scipy>=1.11
Provides-Extra: all
Requires-Dist: catboost>=1.2; extra == 'all'
Requires-Dist: lightgbm>=4.0; extra == 'all'
Requires-Dist: pandas>=2.0; extra == 'all'
Requires-Dist: polars>=1.0; extra == 'all'
Requires-Dist: statsmodels>=0.14; extra == 'all'
Requires-Dist: xgboost>=2.0; extra == 'all'
Provides-Extra: catboost
Requires-Dist: catboost>=1.2; extra == 'catboost'
Provides-Extra: lightgbm
Requires-Dist: lightgbm>=4.0; extra == 'lightgbm'
Provides-Extra: pandas
Requires-Dist: pandas>=2.0; extra == 'pandas'
Provides-Extra: polars
Requires-Dist: polars>=1.0; extra == 'polars'
Provides-Extra: stats
Requires-Dist: statsmodels>=0.14; extra == 'stats'
Provides-Extra: xgboost
Requires-Dist: xgboost>=2.0; extra == 'xgboost'
Description-Content-Type: text/markdown

# xrulefit

[![CI](https://github.com/BastiaanvG/xrulefit/actions/workflows/ci.yml/badge.svg)](https://github.com/BastiaanvG/xrulefit/actions/workflows/ci.yml)
[![Docs](https://github.com/BastiaanvG/xrulefit/actions/workflows/docs.yml/badge.svg)](https://bastiaanvg.github.io/xrulefit/)
[![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue.svg)](https://www.python.org/)
[![License](https://img.shields.io/badge/license-BSD--3--Clause-blue.svg)](LICENSE)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)

RuleFit for Python: rule ensembles built on gradient-boosted trees, following Friedman &
Popescu (2008).

A boosted forest is fitted and thrown away, keeping only the split conditions it found
worth making. Each of those becomes a binary feature, the original features are added
back as winsorised linear terms, and a penalised linear model over the lot decides what
survives. The result usually predicts close to the boosted ensemble it came from, but
you can read it.

This is a Python counterpart to the R package [`xrf`](https://cran.r-project.org/package=xrf).
Against the existing Python implementations it adds classification and multilabel
targets, pandas and Polars support, statistics on the fitted coefficients, a choice of
boosting backend, and export to SQL.

## Install

```bash
pip install xrulefit
```

`numpy`, `scipy`, `scikit-learn` and `narwhals` are the only requirements. The rest are
extras:

```bash
pip install "xrulefit[polars]"     # Polars frames
pip install "xrulefit[stats]"      # p-values and confidence intervals
pip install "xrulefit[xgboost]"    # XGBoost as the rule-mining backend
pip install "xrulefit[catboost]"   # CatBoost as the rule-mining backend
pip install "xrulefit[all]"        # everything
```

Python 3.11 or newer.

## Usage

```python
from sklearn.datasets import make_friedman1
from xrulefit import RuleFitRegressor

X, y = make_friedman1(n_samples=2000, noise=1.0, random_state=0)
model = RuleFitRegressor(n_estimators=150, max_depth=3, random_state=0).fit(X, y)

print(model.summary().top(5))
```

```
term                                                   coef   support  importance
---------------------------------------------------------------------------------
x0 > 0.29954 & x1 > 0.377976 & x3 > 0.534989         0.6844     0.196      0.2714
x3 <= 0.462704                                      -0.4910     0.463      0.2448
x2 > 0.250661 & x2 <= 0.940073 & x3 <= 0.210797     -0.5642     0.159      0.2060
x2 > 0.129908 & x2 <= 0.924132 & x4 <= 0.849423     -0.4365     0.676      0.2042
x0 > 0.856089 & x1 > 0.770448                       -1.0612     0.035      0.1964
```

Friedman #1 depends on `x0`–`x4` and nothing else, which is what comes out. Fit on a
pandas or Polars frame instead and the rules use your column names.

`support` is the fraction of rows a rule fires on; `importance` is the coefficient
weighted by the term's spread, so a large coefficient on a rule that rarely fires does
not dominate the list.

## Estimators

| | Target |
|---|---|
| `RuleFitRegressor` | continuous |
| `RuleFitClassifier` | binary and multiclass |
| `RuleFitMultiLabelClassifier` | several labels at once, over one shared rule pool |

All three pass scikit-learn's `check_estimator` and work inside `Pipeline`,
`GridSearchCV` and `cross_val_score`.

## Boosting backend

```python
RuleFitRegressor(tree_source="xgboost")  # or "lightgbm", "catboost", "sklearn"
```

scikit-learn is the default because it is already a dependency. Every backend is
registered whether or not its library is installed, so asking for one you do not have
tells you which extra to install.

Each lives in its own module under `xrulefit/tree_extraction/`, converting that
library's trees into a shared `ExtractedTree` shape. CatBoost is the awkward one: its
trees are oblivious, meaning every node at a given depth tests the same condition, so it
is expanded back into an ordinary binary tree on the way in.

## Statistics

An elastic net gives coefficients but no standard errors. `inference()` refits the
selected terms without a penalty and reports the usual table:

```python
from xrulefit import inference

print(inference(model, X, y))
```

```
term                                   coef    std err      stat         p    [95% interval]
--------------------------------------------------------------------------------------------
intercept                           12.1771     0.1902    64.024    0.0000   11.8040  12.5502
x0 > 0.231821 & x1 > 0.217089        2.7109     0.2178    12.445    0.0000    2.2836   3.1382
x2 > 0.072634 & x3 <= 0.575371      -1.7922     0.2364    -7.582    0.0000   -2.2560  -1.3285
```

These are post-selection p-values. The terms were chosen with the same data they are
being tested on, so the values are optimistic and the intervals too narrow. Use them to
rank terms, not to test hypotheses. Requires `xrulefit[stats]`.

## Export

A fitted model is a weighted sum of threshold comparisons, so it can be written out and
evaluated elsewhere. That is useful when scoring happens in a warehouse, or when there
is nowhere to keep a pickle:

```python
from xrulefit import to_sql

print(to_sql(model, table="features", dialect="sqlite"))
```

```sql
SELECT
    12.761347014194893
    + -0.11490190735236692 * CASE WHEN "x2" <= 0.3684648871421814 THEN 1.0 ELSE 0.0 END
    ...
    AS "prediction"
FROM features
```

Only the surviving terms appear. ANSI, SQLite and BigQuery dialects are available. The
test suite runs the generated SQL through SQLite and checks it against `predict()`.

The same model also compiles to a dataframe expression, which is an ordinary column
expression pandas and Polars both understand. That puts the model inside a query rather
than beside it:

```python
import narwhals as nw
from xrulefit import to_expression

score = to_expression(model)

nw.from_native(frame).filter(score > 3.0).to_native()
```

It evaluates on a Polars `LazyFrame` as well, so the model can sit in a query plan
without the rows being materialised. For ordinary scoring `model.predict(frame)` is
still the thing to reach for; the expression earns its place when you want to filter or
group by the prediction, or to keep the work inside the dataframe engine.

## Documentation

Guide, examples and API reference: <https://bastiaanvg.github.io/xrulefit/>.

The scripts in [`examples/`](examples/) are embedded verbatim into the documentation
and run by the test suite, so what is published is what works.

## Development

The project is managed with [uv](https://docs.astral.sh/uv/). One command builds the
environment, including every optional backend:

```bash
git clone https://github.com/BastiaanvG/xrulefit
cd xrulefit
uv sync --all-extras
```

`uv.lock` is committed, so that resolves to the same versions CI uses.

```bash
uv run pytest                  # the suite, including doctests and every example
uv run ruff check .            # lint
uv run ruff format .           # format
uv run mypy src                # types, in strict mode
uv run mkdocs serve            # docs at http://127.0.0.1:8000
```

`pytest` runs the examples end to end and executes the generated SQL against SQLite, so
a full run takes a few minutes. To iterate faster, narrow it:

```bash
uv run pytest tests/test_rule_generation.py
uv run pytest -k "not sklearn_compat and not examples"
```

Every check above runs in CI across Linux, macOS and Windows on Python 3.11 to 3.13,
plus one job that installs the package with no extras at all and fits a model, so the
bare install cannot quietly break.

### Releasing

Publishing runs from a tag, using PyPI Trusted Publishing, so there is no API token
stored anywhere:

```bash
git tag v0.1.0
git push origin v0.1.0
```

The workflow refuses to publish if the tag disagrees with the version in
`pyproject.toml`, and installs the built wheel into a clean environment and fits a model
with it before anything is uploaded.

## License

BSD 3-Clause. See [LICENSE](LICENSE).

## References

- Friedman, J. H. & Popescu, B. E. (2008). Predictive learning via rule ensembles.
  *The Annals of Applied Statistics*, 2(3), 916–954.
  [arXiv:0811.1679](https://arxiv.org/abs/0811.1679)
- Fokkema, M. (2020). Fitting prediction rule ensembles with R package `pre`.
  *Journal of Statistical Software*, 92(12).
  [arXiv:1707.07149](https://arxiv.org/abs/1707.07149)
- Holub, K. `xrf`: eXtreme RuleFit. CRAN. <https://cran.r-project.org/package=xrf>
