Metadata-Version: 2.4
Name: spark-assets
Version: 1.0.3
Summary: SPARK - System for Processing & Asset Refinement Kit - Transform raw game assets into organized reference data
Author-email: KARAN SAXENA <saxenakaran1239@gmail.com>
Maintainer-email: KARAN SAXENA <saxenakaran1239@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/AristocratEnterprise/roxstar-spark
Project-URL: Documentation, https://github.com/AristocratEnterprise/roxstar-spark/blob/main/DEMO.md
Project-URL: Repository, https://github.com/AristocratEnterprise/roxstar-spark.git
Project-URL: Issues, https://github.com/AristocratEnterprise/roxstar-spark/issues
Project-URL: Changelog, https://github.com/AristocratEnterprise/roxstar-spark/releases
Keywords: asset-processing,game-assets,sprite-extraction,image-classification,deduplication,mpkg,clip,ocr
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Multimedia :: Graphics
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=1.9.0
Requires-Dist: torchvision>=0.10.0
Requires-Dist: transformers>=4.25.0
Requires-Dist: Pillow>=8.0.0
Requires-Dist: numpy>=1.19.0
Requires-Dist: opencv-python>=4.5.0
Requires-Dist: tqdm>=4.60.0
Provides-Extra: pipeline
Requires-Dist: easyocr>=1.6.0; extra == "pipeline"
Requires-Dist: imagehash>=4.3.0; extra == "pipeline"
Provides-Extra: aep-extract
Requires-Dist: py-aep>=0.10; extra == "aep-extract"
Requires-Dist: ImageHash>=4.3; extra == "aep-extract"
Requires-Dist: psd-tools>=1.10; extra == "aep-extract"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.20.0; extra == "dev"
Requires-Dist: black>=22.0.0; extra == "dev"
Requires-Dist: isort>=5.10.0; extra == "dev"
Requires-Dist: flake8>=5.0.0; extra == "dev"
Requires-Dist: mypy>=0.990; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=5.0.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "docs"
Requires-Dist: myst-parser>=0.18.0; extra == "docs"
Provides-Extra: all
Requires-Dist: spark-assets[aep-extract,dev,docs,pipeline]; extra == "all"
Dynamic: license-file

# ⚡ SPARK - System for Processing & Asset Refinement Kit

**SPARK** transforms raw game assets into organized, labeled reference data.

## 🚀 Quick Start

```bash
# 1. Install SPARK and dependencies
pip install -e .

# 2. Run on MPKG + PNG assets
spark pipeline --input game_resources/ --output processed/ --verbose
```

**CLIP model**: downloaded only when you run a command that uses CLIP (unless `--no-clip` is used).

### Mercury Support (Optional)

```bash
# One-time: Configure npm with GitHub token
npm config set @aristocrattechnologiesinc:registry https://npm.pkg.github.com
npm config set //npm.pkg.github.com/:_authToken YOUR_GITHUB_TOKEN

# Then: merc auto-installs on first use
spark merc_pipeline --input mercury_resources/ --output processed/ --verbose
```

**Note**: SPARK auto-installs `merc` CLI if npm is configured. See [DEMO.md](DEMO.md) for details.

## 🛠️ Requirements

### Core Requirements
- **Python 3.8+**
- **pip** (Python package manager)
- **Internet connection** (first run only - to download CLIP model)

### What Gets Installed Automatically

When you run `pip install -e .`, SPARK automatically:
1. ✅ Installs all Python dependencies (torch, transformers, opencv, etc.)
2. ✅ Installs CLI entrypoints (`spark`)
3. ✅ Keeps CLIP model download lazy (downloaded at first CLIP-enabled run)

**After first run**: 100% offline processing, no cloud access required.

### Optional (For Mercury Bundle Support)
- **Node.js 14+** with npm
- **merc CLI** - Install globally:
  ```bash
  npm install -g @aristocrattechnologiesinc/merc
  ```

For Mercury setup, see [DEMO.md](DEMO.md).

## ✨ Features

- **Single-command processing** - Extract, classify, deduplicate in one step
- **Composite image splitting** - Automatically detects atlas/sprite-sheet images and splits them into individual assets
- **Video / animation support** - Extracts distinct scene frames from `.ogv`, `.mp4`, and other video formats
- **Hybrid classification** - MPKG metadata + OCR + CLIP semantic
- **Smart deduplication** - Perceptual hash-based (~96% reduction)
- **Mercury support** - DXT texture extraction via lonestar-tools
- **Organized output** - Clean folder structure by label
- **One image = one asset** - Every output file contains a single visual unit

## 📋 CLI Commands

| Command | Purpose |
|---------|---------|
| `spark pipeline` | Process MPKG + PNG assets |
| `spark merc_pipeline` | Process Mercury bundles |
| `spark build-reference` | Build CLIP reference embeddings |
| `spark flatten` | Flatten hierarchical asset folders |
| `spark extract` | Parse `.aep` projects into position-verified, royal/game-symbol/button-categorized golden assets |

## ⚙️ Command combinations

### `spark pipeline`

```bash
spark pipeline -i <input> -o <output> [--no-ocr] [--no-clip] [--no-mpkg] [--no-cv] [--no-metadata] [--no-logs] [-v]
```

### `spark merc_pipeline`

```bash
spark merc_pipeline -i <input> -o <output> [--no-ocr] [--no-clip] [--no-cv] [--no-metadata] [-v]
```

### `spark flatten` (now quality-aware by default)

```bash
spark flatten -i <input> -o <output> [--no-dedup] [--similarity-threshold 0.95] [--dry-run] [-v]
```

By default, `flatten` removes duplicates per label and keeps the highest-quality image (larger resolution first).

### `spark extract`

```powershell
spark extract --test-input <dir> [--test-output <dir>] [--game-name <name>] [--no-copy] [--no-reuse-fingerprint-cache] [--full-output] [--complete] [--gamesymbols] [--audio] [-v]
```

By default, `golden_assets/` is categorized into `royal_symbols/` (A/K/Q/J/10/9/8/7),
`game_symbols/` (pic1, pic2, ... in paytable order), and `buttons/`, identified by
cross-referencing the game's own paytable help screen against the golden asset
library - see `spark/aep_extract/OUTPUT_STRUCTURE.md` for the full method.

| Flag | `golden_assets/` result |
| --- | --- |
| *(none)* | `royal_symbols/`, `game_symbols/`, `buttons/` only |
| `--complete` | `images/`, `animated/`, `review_required/` only - no categorization |
| `--gamesymbols` | Categorizes, then keeps only `game_symbols/` |
| `--audio` | Same as `--complete`, **plus** `audio/<label>.<ext>` for every audio asset |

| Command | `golden_assets/` contents |
| --- | --- |
| `spark extract --test-input <dir> --complete` | `images/`, `animated/`, `review_required/` only |
| `spark extract --test-input <dir>` | `royal_symbols/`, `game_symbols/`, `buttons/` only |
| `spark extract --test-input <dir> --gamesymbols` | `game_symbols/` only |
| `spark extract --test-input <dir> --audio` | `images/`, `animated/`, `review_required/`, `audio/` only |

Re-running `spark extract` against unchanged input skips AEP reparsing entirely
and only rebuilds symbol categorization (a few seconds instead of minutes).

### Run AEP extraction from `input\AE_Files`

```powershell
# From repo root; writes output to output\ by default
spark extract --test-input input\AE_Files --verbose
```

## 📁 Output Structure

```
processed/
├── final_labeled_assets/
│   ├── spin_button/
│   ├── wild_symbol/
│   └── ...
└── pipeline_metadata.json
```

## 🔧 Python API

```python
from spark import process_assets

results = process_assets(
    input_dir="game_resources/",
    output_dir="processed/"
)

print(f"Processed {results['unique_after_dedup']} unique assets")
```

## 📦 Project Structure

```
spark/
├── spark/                 # Main package
│   ├── cli.py             # CLI commands
│   ├── assets/            # Processing modules
│   │   ├── pipeline.py           # Core pipeline
│   │   ├── composite_splitter.py # Atlas/composite → single-asset splitting
│   │   └── merc_rip_bridge.py    # Mercury integration
│   └── detection/         # Detection utilities
├── examples/              # Usage examples
├── tests/                 # Unit tests
│   ├── test_core.py
│   ├── test_composite_splitter.py
│   └── test_pipeline.py
├── input/                 # Sample input data (for testing)
└── DEMO.md                # Getting started guide
```

**Note**: `merc` CLI tool is installed globally via npm, not bundled in the repository.

## 🛠️ Requirements

- Python 3.8+
- For Mercury: Node.js + lonestar-tools merc

## 📝 License

MIT License - see [LICENSE](LICENSE)

---

**SPARK** - Built for game asset processing ⚡
