Metadata-Version: 2.4
Name: strataregula-doe-runner
Version: 0.1.0
Summary: Batch experiment orchestrator for Strataregula ecosystem
Project-URL: Homepage, https://github.com/strataregula/strataregula-doe-runner
Project-URL: Documentation, https://strataregula-doe-runner.readthedocs.io/
Project-URL: Repository, https://github.com/strataregula/strataregula-doe-runner.git
Project-URL: Issues, https://github.com/strataregula/strataregula-doe-runner/issues
Author-email: Strataregula Team <team@strataregula.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: batch,csv,experiment,metrics,orchestrator,strataregula
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Systems Administration
Requires-Python: >=3.8
Requires-Dist: click>=8.0.0
Requires-Dist: pydantic>=1.8.0
Requires-Dist: python-dateutil>=2.8.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: black>=21.0.0; extra == 'dev'
Requires-Dist: flake8>=3.8.0; extra == 'dev'
Requires-Dist: isort>=5.0.0; extra == 'dev'
Requires-Dist: mypy>=0.910; extra == 'dev'
Requires-Dist: pytest-cov>=2.10.0; extra == 'dev'
Requires-Dist: pytest>=6.0.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: myst-parser>=0.15.0; extra == 'docs'
Requires-Dist: sphinx-rtd-theme>=0.5.0; extra == 'docs'
Requires-Dist: sphinx>=4.0.0; extra == 'docs'
Provides-Extra: simroute
Requires-Dist: numpy>=1.20.0; extra == 'simroute'
Requires-Dist: pandas>=1.3.0; extra == 'simroute'
Description-Content-Type: text/markdown

# Strataregula - DOE Runner

[![Tests](https://github.com/strataregula/strataregula-doe-runner/actions/workflows/test.yml/badge.svg)](https://github.com/strataregula/strataregula-doe-runner/actions/workflows/test.yml)
[![Coverage](https://codecov.io/gh/strataregula/strataregula-doe-runner/branch/main/graph/badge.svg)](https://codecov.io/gh/strataregula/strataregula-doe-runner)
[![PyPI](https://img.shields.io/pypi/v/strataregula-doe-runner.svg)](https://pypi.org/project/strataregula-doe-runner/)
[![Python](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![Strataregula Plugin](https://img.shields.io/badge/Strataregula-Plugin-green)](https://github.com/strataregula/strataregula)

Batch experiment orchestrator for Strataregula ecosystem.

## What is DOE Runner?

**DOE Runner is NOT a Design of Experiments tool.** It is a **batch experiment orchestrator** that:

- 📋 Executes predefined cases from `cases.csv`
- 🔀 Manages execution order and parallelism  
- 📊 Collects standardized metrics to `metrics.csv`
- 🔒 Provides deterministic, reproducible results
- 💾 Caches execution results with `case_hash`
- 📝 Generates comprehensive run logs (JST timezone)

**Definition (1 line)**: cases.csvで定義したケース集合を、決定論的に一括実行し、標準化されたmetrics.csv（＋Runログ）に集約する"バッチ実験オーケストレータ"

## Quick Start

```bash
# Install
pip install strataregula-doe-runner

# Execute cases
srd run --cases cases.csv --out metrics.csv

# With parallel execution  
srd run --cases cases.csv --out metrics.csv --max-workers 4

# Force re-execution (ignore cache)
srd run --cases cases.csv --out metrics.csv --force

# Dry run (validation only)
srd run --cases cases.csv --dry-run
```

## Exit Codes

| Code | Meaning |
|------|---------|
| 0 | All cases executed successfully, thresholds met |
| 2 | Execution completed but threshold violations detected |
| 3 | I/O error, invalid configuration, or execution failure |

## Cases CSV Contract

### Required Columns
- `case_id`: Unique case identifier
- `backend`: Execution backend (`shell`/`dummy`/`simroute`)
- `cmd_template`: Command template with `{placeholders}`
- `timeout_s`: Timeout in seconds

### Recommended Columns  
- `seed`: Random seed for reproducibility
- `retries`: Number of retry attempts
- `resource_group`: Resource grouping for execution

### Metrics Expectation Columns (Optional)
- `expected_p95`, `expected_p99`: Expected metric values
- `threshold_p95`, `threshold_p99`: Threshold limits

### Example cases.csv
```csv
case_id,backend,cmd_template,timeout_s,seed,retries,resource_group,expected_p95,threshold_p95
test-01,shell,"echo 'p95=0.05 p99=0.08 throughput_rps=1000'",10,42,2,default,0.05,0.06
test-02,simroute,"python -m cli_run --seed {seed}",120,123,3,compute,0.10,0.12
```

## Metrics CSV Output

### Required Columns (Fixed Order)
```csv
case_id,status,run_seconds,p95,p99,throughput_rps,errors,ts_start,ts_end
```

### Status Values
- `OK`: Successful execution
- `FAIL`: Execution failed
- `TIMEOUT`: Execution timed out

### Additional Columns
- Performance: `cpu_util`, `mem_peak_mb`, `queue_depth_p95`
- Parameters: `param_*` (input case parameters)
- Extensions: `ext_*`, `tag_*`, `note_*`

## Features

### 🔄 Deterministic Execution
- Same input always produces same output
- Fixed column ordering in CSV files
- Consistent number formatting
- LF line endings enforced

### 💾 Intelligent Caching
- `case_hash` calculated from deterministic case elements
- Automatic cache hit detection
- `--force` flag to override cache

### 📊 Threshold Validation
- Automatic metric threshold checking
- Exit code 2 for threshold violations
- Detailed violation reporting

### 🔧 Multiple Backends
- `shell`: Execute shell commands
- `dummy`: Testing/simulation backend
- `simroute`: Integration with world-simulation (extras dependency)

### 📝 Rich Logging
- JST timezone run logs
- Execution summaries with statistics
- Artifact tracking
- Error reporting

## Environment Variables

- `RUN_LOG_DIR`: Directory for run logs (default: `docs/run`)
- `RUN_LOG_WRITE_COMPAT`: Compatibility mode flag (default: `0`)

## Integration with Strataregula

This plugin integrates with Strataregula through entry points:

```toml
[project.entry-points."strataregula.plugins"]
"doe_runner" = "strataregula_doe_runner.plugin:DOERunnerPlugin"
```

When installed, it's automatically discoverable by the Strataregula framework.

### Plugin Commands
- `execute_cases`: Run cases from CSV
- `validate_cases`: Validate case format  
- `get_progress`: Check execution progress
- `export_results`: Export execution results

## Development

```bash
# Clone repository
git clone https://github.com/strataregula/strataregula-doe-runner
cd strataregula-doe-runner

# Install development dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Run with coverage
pytest --cov=strataregula_doe_runner

# Format code
black src tests
isort src tests

# Type checking
mypy src
```

## Examples

See the `examples/` directory for:
- `simple/`: Basic cases.csv example
- `complex/`: Advanced configuration with all features
- `simroute/`: Integration with world-simulation

## License

Apache License 2.0 - see [LICENSE](LICENSE) file.

## Contributing

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

## Support

- 📚 [Documentation](https://strataregula-doe-runner.readthedocs.io/)
- 🐛 [Issue Tracker](https://github.com/strataregula/strataregula-doe-runner/issues)
- 💬 [Discussions](https://github.com/strataregula/strataregula-doe-runner/discussions)