Metadata-Version: 2.4
Name: revenium-middleware-litellm
Version: 0.1.11
Summary: A Python library that meters LiteLLM usage to Revenium.
Author-email: Revenium <info@revenium.io>
License: Apache-2.0
Project-URL: Homepage, https://github.com/revenium/revenium-middleware-litellm-python
Project-URL: Bug Tracker, https://github.com/revenium/revenium-middleware-litellm-python/issues
Project-URL: Documentation, https://github.com/revenium/revenium-middleware-litellm-python/blob/main/README.md
Keywords: litellm,middleware,logging,token-usage,metering,revenium
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: wrapt
Requires-Dist: revenium_middleware
Requires-Dist: litellm
Provides-Extra: litellm-proxy
Requires-Dist: litellm[proxy]; extra == "litellm-proxy"
Requires-Dist: logging; extra == "litellm-proxy"
Provides-Extra: litellm-client
Requires-Dist: litellm; extra == "litellm-client"
Requires-Dist: logging; extra == "litellm-client"
Requires-Dist: wrapt; extra == "litellm-client"
Requires-Dist: revenium_middleware; extra == "litellm-client"
Provides-Extra: litellm-all
Requires-Dist: revenium-middleware[litellm_proxy]; extra == "litellm-all"
Requires-Dist: revenium-middleware[litellm_client]; extra == "litellm-all"

# 🤖 Revenium Middleware for LiteLLM

[![PyPI version](https://img.shields.io/pypi/v/revenium-middleware-litellm.svg)](https://pypi.org/project/revenium-middleware-litellm/)
[![Python Versions](https://img.shields.io/pypi/pyversions/revenium-middleware-litellm.svg)](https://pypi.org/project/revenium-middleware-litellm/)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://www.apache.org/licenses/LICENSE-2.0)

A middleware library for metering and monitoring LiteLLM usage in Python applications, supporting both direct client calls and the LiteLLM proxy. 🐍✨

## ✨ Features

- **📊 Precise Usage Tracking**: Monitor tokens, costs, and request counts across all LLM API endpoints via LiteLLM.
- **🔌 Seamless Integration**:
    - **Client:** Drop-in middleware that works with minimal code changes for `litellm.completion`.
    - **Proxy:** Custom callback middleware for easy integration with the LiteLLM proxy.
- **⚙️ Flexible Configuration**: Customize metering behavior to suit your application needs.
- **📈 Enhanced Metadata**: Track detailed usage context like trace IDs, task types, and user identifiers.

## 📥 Installation

Install the base package along with the specific middleware you need:

```bash
# For LiteLLM client-side metering ONLY
pip install revenium-middleware-litellm[litellm_client]

# For LiteLLM proxy metering ONLY
pip install revenium-middleware-litellm[litellm_proxy]

# To install BOTH client and proxy middleware support
pip install revenium-middleware-litellm[litellm_all]
```

## 🔧 Usage: LiteLLM Client Middleware

This middleware automatically intercepts `litellm.completion` calls made directly from your Python code.

### 🔄 Zero-Config Integration

Simply export your `REVENIUM_METERING_API_KEY` (and other relevant Revenium environment variables) and import the middleware module *before* your first call to `litellm.completion`. Your LiteLLM calls will be metered automatically:

```python
import litellm
import os

# Ensure Revenium environment variables (e.g., REVENIUM_METERING_API_KEY) are set
# os.environ["REVENIUM_METERING_API_KEY"] = "YOUR_REVENIUM_METERING_API_KEY"
# os.environ["REVENIUM_PRODUCT_ID"] = "YOUR_REVENIUM_PRODUCT_ID" # Optional, but recommended

# Import the client middleware module to activate patching
import revenium_middleware_litellm_client.middleware

# Now use litellm.completion as usual
response = litellm.completion(
    model="gpt-3.5-turbo",
    messages=[{ "content": "Why is the sky blue?","role": "user"}]
)
print(response.choices[0].message.content)
```

The middleware automatically intercepts LiteLLM API calls and sends metering data to Revenium without requiring changes to your existing `litellm.completion` calls. Make sure to configure the underlying `revenium_middleware` (e.g., set the `REVENIUM_METERING_API_KEY` environment variable) for authentication with the Revenium service.

### 📈 Enhanced Tracking with Metadata

For more granular usage tracking and detailed reporting, add the `usage_metadata` parameter directly to your `litellm.completion` call:

```python
import litellm
import revenium_middleware_litellm_client.middleware # Ensure middleware is imported

response = litellm.completion(
    model="gpt-3.5-turbo",
    messages=[{ "content": "Why is the sky blue?","role": "user"}],
    usage_metadata={
        "trace_id": "conv-28a7e9d4-1c3b-4e5f-8a9b-7d1e3f2c1b4a",
        "task_id": "chat-summary-af23c910",
        "task_type": "text-classification",
        "subscriber_identity": "customer-email@example.com",
        "organization_id": "acme-corporation-12345",
        "subscription_id": "startup-plan-quarterly-2025-Q1",
        "product_id": "intelligent-document-processor-v3",
        "source_id": "backend-service-v1.2",
        "ai_provider_key_name": "openai-production-key1", # If applicable
        "agent": "customer-support-assistant-v2",
    },
)
print(response.choices[0].message.content)
```

#### 🏷️ Client Metadata Fields (`usage_metadata`)

The `usage_metadata` parameter supports the following fields for client-side metering:

| Field                    | Description                                     | Use Case                                                          |
|--------------------------|-------------------------------------------------|-------------------------------------------------------------------|
| `trace_id`               | Unique identifier for a conversation or session | Track multi-turn conversations                                    |
| `task_id`                | Identifier for a specific AI task               | Group related API calls for a single task                         |
| `task_type`              | Classification of the AI operation              | Categorize usage by purpose (e.g., classification, summarization) |
| `subscriber_email`       | The email address of the subscriber             | Track usage by individual users                                   |
| `subscriber_credential`  | The credential associated with the subscriber   | Track usage by individual users                                   |
| `organization_id`        | Customer or department identifier               | Allocate costs to business units                                  |
| `subscription_id`        | Reference to a billing plan                     | Associate usage with specific subscriptions                       |
| `product_id`             | The product or feature using AI                 | Track usage across different products                             |
| `agent`                  | Identifier for the specific AI agent            | Compare performance across different AI agents                    |
| `response_quality_score` | The quality of the AI response (0..1)           | Track AI response quality                                         |
| `ai_provider_key_name`   | Identifier for the specific LLM provider API key used | Track usage per API key                                       |
| `source_id`              | Identifier for the system initiating the call   | Track usage origin (e.g., web app, mobile app, backend job)     |


All metadata fields are optional. Adding them enables more detailed reporting and analytics in Revenium.

---

## 🔧 Usage: LiteLLM Proxy Middleware

This middleware integrates with the LiteLLM proxy using its custom callback mechanism.

### 🔄 Integration with LiteLLM Proxy

To integrate the Revenium middleware with your LiteLLM proxy, you need to:

1. Install the middleware (`pip install revenium-middleware-litellm[litellm_proxy]`).
2. Ensure Revenium environment variables (e.g., `REVENIUM_METERING_API_KEY`) are set where the proxy runs.
3. Configure your LiteLLM proxy to use the Revenium middleware callback.

#### Configuration

Add the Revenium middleware callback path to your LiteLLM `config.yaml`:

```yaml
model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo # Example using OpenAI provider prefix
      api_key: "os.environ/OPENAI_API_KEY"

general_settings:
  master_key: "sk-1234" # Example master key

litellm_settings:
  # Add the absolute path to the installed middleware's proxy_handler_instance
  callbacks: ["revenium_middleware_litellm_proxy.middleware.proxy_handler_instance"]
  # Example database URL (optional, for LiteLLM's own logging)
  # database_url: "postgresql://user:password@localhost/litellm"
```

> **⚠️ Important Note on Callbacks**: LiteLLM proxy expects the `callbacks` list to contain importable Python paths to the callback handler instances. Ensure the `revenium-middleware-litellm` package is installed in the Python environment where the LiteLLM proxy runs. You do *not* need an absolute file path if the package is correctly installed.

### 📈 Enhanced Tracking with Custom Headers (Proxy)

For more granular usage tracking when using the proxy, add custom HTTP headers to your API requests made *to* the LiteLLM proxy endpoint:

```python
# Example using requests library to call the LiteLLM proxy
import requests
import json

proxy_url = "http://localhost:4000/chat/completions" # Your LiteLLM proxy URL
api_key = "sk-1234" # Your LiteLLM proxy master key or virtual key

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {api_key}",
    # Revenium Custom Headers
    "x-revenium-trace-id": "conv-28a7e9d4-1c3b-4e5f-8a9b-7d1e3f2c1b4a",
    "x-revenium-task-id": "chat-summary-af23c910",
    "x-revenium-task-type": "text-classification",
    "x-revenium-organization-id": "acme-corporation-12345",
    "x-revenium-subscription-id": "startup-plan-quarterly-2025-Q1",
    "x-revenium-product-id": "intelligent-document-processor-v3",
    "x-revenium-agent": "customer-support-assistant-v2",
    "x-revenium-subscriber-identity": "user@example.com", # Added for proxy
    "x-revenium-source-id": "web-frontend-v2.1", # Added for proxy
}

data = {
    "model": "gpt-3.5-turbo",
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the meaning of life?"}
    ]
}

response = requests.post(proxy_url, headers=headers, data=json.dumps(data))

print(response.status_code)
print(response.json())

```

#### 🏷️ Custom HTTP Headers (Proxy)

The following custom HTTP headers can be added to your API requests *to the proxy*:

| Header                       | Description                                     | Use Case                                                          |
|------------------------------|-------------------------------------------------|-------------------------------------------------------------------|
| `x-revenium-trace-id`        | Unique identifier for a conversation or session | Track multi-turn conversations                                    |
| `x-revenium-task-id`         | Identifier for a specific AI task               | Group related API calls for a single task                         |
| `x-revenium-task-type`       | Classification of the AI operation              | Categorize usage by purpose (e.g., classification, summarization) |
| `x-revenium-subscriber-identity` | Email or unique ID of the end-user/subscriber | Track usage by individual users via the proxy                     |
| `x-revenium-organization-id` | Customer or department identifier               | Allocate costs to business units                                  |
| `x-revenium-subscription-id` | Reference to a billing plan                     | Associate usage with specific subscriptions                       |
| `x-revenium-product-id`      | The product or feature using AI                 | Track usage across different products                             |
| `x-revenium-agent`           | Identifier for the specific AI agent            | Compare performance across different AI agents                    |
| `x-revenium-source-id`       | Identifier for the system calling the proxy     | Track usage origin (e.g., web app, mobile app)                    |

All custom headers are optional. Adding them enables more detailed reporting and analytics in Revenium. The proxy middleware extracts these headers from the incoming request.

---

## 🔄 Compatibility

- 🐍 Python 3.8+
- 🤖 LiteLLM (Client usage & Proxy v1.15.0+)
- 🌐 Works with all LLM models and providers supported by LiteLLM

## 🔍 Logging

This module uses Python's standard logging system, inheriting configuration from the core `revenium_middleware`. You can control the log level by setting the `REVENIUM_LOG_LEVEL` environment variable:

```bash
# Enable debug logging
export REVENIUM_LOG_LEVEL=DEBUG

# Or when running your script (for client middleware)
REVENIUM_LOG_LEVEL=DEBUG python your_script.py

# Or when starting the proxy (for proxy middleware)
REVENIUM_LOG_LEVEL=DEBUG litellm --config /path/to/config.yaml
```

Available log levels:
- `DEBUG`: Detailed debugging information
- `INFO`: General information (default)
- `WARNING`: Warning messages only
- `ERROR`: Error messages only
- `CRITICAL`: Critical error messages only

## 👥 Contributing

Contributions are welcome! Please check out our contributing guidelines for details.

1. 🍴 Fork the repository
2. 🌿 Create your feature branch (`git checkout -b feature/amazing-feature`)
3. 💾 Commit your changes (`git commit -m 'Add some amazing feature'`)
4. 🚀 Push to the branch (`git push origin feature/amazing-feature`)
5. 🔍 Open a Pull Request

## 📄 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

## 🙏 Acknowledgments

- 🔥 Thanks to the LiteLLM team for creating an excellent library and proxy
- 💖 Built with ❤️ by the Revenium team
