Metadata-Version: 2.4
Name: GRating
Version: 0.0.31
Summary: Algorithm ranking library
Author: Oscar A. Gonzalez Sanchez
Project-URL: Homepage, https://github.com/OscarAGonzalezSanchez/GRating
Keywords: optimization,ranking,metaheuristics
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: matplotlib
Requires-Dist: squarify
Requires-Dist: seaborn
Requires-Dist: pandas
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: openpyxl

# GRating

GRating is a Python library for ranking optimization algorithms using a pairwise-comparison framework based on the Bradley–Terry model.

The framework transforms benchmark results into win–loss statistics, estimates algorithm strengths through maximum-likelihood Bradley–Terry fitting, and provides statistical and visual tools for ranking analysis.

The methodology and theoretical foundations of GRating are described in the accompanying research paper:

> Óscar A. González-Sánchez, Erik Cuevas, Daniel Zaldivar, Héctor Escobar-Cuevas.*Is Newer Always Better? The G-Rating Metric: Quantifying
Good Performances and Exposing Bias Across Six Decades of
Metaheuristic Algorithms*. 2026.

---

## Citation

If you use GRating in academic research, please cite both the software and the associated paper.

```bibtex
@article{grating_paper,
  title={Is Newer Always Better? The G-Rating Metric: Quantifying Good Performances and Exposing Bias Across Six Decades of Metaheuristic Algorithms},
  author={Óscar A. González-Sánchez, Erik Cuevas, Daniel Zaldivar, Héctor Escobar-Cuevas},
  journal={...},
  year={2026}
}

@software{grating,
  title={GRating},
  author={OscarAGonzalezSanchez},
  year={2026},
  url={https://github.com/OscarAGonzalezSanchez/GRating}
}
```

---

## Features

- Bradley–Terry rating estimation from benchmark results.
- Automatic generation of win–loss matrices.
- Multiple comparison strategies:
  - Full permutation comparisons.
  - Random subsampling comparisons.
- Friedman Mean Rank computation.
- Statistical comparison between independent studies.
- Wilcoxon signed-rank significance testing.
- G-Rating system alignment and scaling.
- Treemap visualization.
- Scatter plot visualization.
- Publication year vs. G-Rating analysis.
- Excel export of rankings and comparison results.

---

## Installation

Install directly from PyPI:

```bash
pip install grating
```

---

## Benchmark Data Format

GRating expects **one CSV file per algorithm**.

Example directory structure:

```text
results/
├── PSO.csv
├── GA.csv
├── DE.csv
└── AEO.csv
```

Each CSV file must contain:

- Columns → Benchmark functions/problems.
- Rows → Independent runs.

Example:

| F1 | F2 | F3 |
|----|----|----|
| 0.15 | 1.24 | 3.12 |
| 0.11 | 1.10 | 2.94 |
| 0.14 | 1.30 | 3.05 |

Lower values are assumed to represent better performance.

---

## Quick Start

### Load Experimental Results

```python
from grating import GRating

G = GRating()

G.create_experimentation_table(
    path="results"
)
```

---

### Generate Win–Loss Matrix

#### Full Permutation Strategy

Compares every run against every run.

```python
G.create_win_lose_table(
    method="permutation"
)
```

#### Random Subsampling Strategy

Reduces computational cost.

```python
G.create_win_lose_table(
    method="subsample",
    samples=1000
)
```

---

### Train the Bradley–Terry Model

```python
algorithms, ratings = G.train_bt_model()
```

Example output:

```python
[
    ("PSO", 2.31),
    ("DE", 1.84),
    ("GA", 0.92),
    ("AEO", 0.61)
]
```

Higher values indicate stronger algorithms.

---

### Compute Friedman Mean Rank

```python
friedman_ranks = G.get_friedman_mean_rank()
```

---

### Export Results

```python
from grating import export_results_to_excel

export_results_to_excel(
    G,
    export_dir="exports"
)
```

Generated file:

```text
exports/
└── G-Rating.xlsx
```

---

## Visualizations

### Scatter Plot

```python
from grating import generate_scatter_plot

generate_scatter_plot(
    G,
    save_dir="figures"
)
```

Displays the ranking on a logarithmic G-Rating scale.

---

### Treemap

```python
from grating import generate_treemap

generate_treemap(
    G,
    save_dir="figures"
)
```

Rectangle area is proportional to algorithm strength.

---

### Comparing Independent Studies

GRating can compare two independent benchmark studies.

---

## Create Models

```python
control = GRating()
control.create_experimentation_table("baseline")
control.create_win_lose_table()
control.train_bt_model()

study = GRating()
study.create_experimentation_table("new_results")
study.create_win_lose_table()
study.train_bt_model()
```

---

## Compare Models

```python
from grating import compare_models

comparison = compare_models(
    control,
    study,
    significance_level=0.05,
    max_fails=11
)
```

The comparison process:

1. Performs Wilcoxon signed-rank tests.
2. Detects statistically significant differences.
3. Classifies biased algorithms.
4. Aligns both G-Rating systems.
5. Generates comparison visualizations.

---

## Comparison Plot

Algorithms are classified into:

- Statistically equivalent.
- Statistically different.

The plot shows:

- Reference G-Ratings.
- Scaled study G-Ratings.
- Difference regions between both studies.

---

## Custom Algorithm Labels

Long algorithm names can be replaced using aliases.

```python
aliases = {
    "OriginalPSO": "PSO",
    "OriginalGWO": "GWO",
    "AugmentedAEO": "AAEO"
}

generate_treemap(
    G,
    aliases=aliases
)
```

The same aliases can be used in:

- `generate_treemap()`
- `compare_models()`
- `export_results_to_excel()`

---

## Loading Preprocessed Data

Instead of CSV files, a preprocessed tensor can be loaded directly.

Shape:

```python
(
    algorithms,
    tests,
    runs
)
```

Example:

```python
import numpy as np

data = np.random.rand(
    5,   # algorithms
    30,  # benchmark functions
    25   # runs
)

G = GRating()
G.load_fit_data(data)
```

---

## Using Existing Win–Loss Matrices

```python
G.load_win_lose_table(win_loss_matrix)
```

This allows direct Bradley–Terry fitting without storing benchmark results.

---

## Mathematical Foundation

GRating is based on the Bradley–Terry model.

For two algorithms i and j:

```math
P(i \succ j) =
\frac{p_i}{p_i + p_j}
```

where:

- \(p_i\) = strength of algorithm i
- \(p_j\) = strength of algorithm j

The strengths are estimated using iterative maximum-likelihood optimization and normalized by their geometric mean.

---

## Requirements

Main dependencies:

- NumPy
- Pandas
- SciPy
- Matplotlib
- Seaborn
- Squarify

---

$# License

MIT License.
