Metadata-Version: 2.5
Name: pocket-o-llama
Version: 0.1.0
Summary: Lightweight, hardware-independent local GGUF parser and Ollama substitute server with Hugging Face Hub integration.
Project-URL: Homepage, https://huggingface.co/spaces
Project-URL: Repository, https://github.com/Hrutu34/Pocket-O-Llama
Author-email: Hrutu Surve <survehrutu@gmail.com>
License: MIT
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.110.0
Requires-Dist: gguf>=0.19.0
Requires-Dist: huggingface-hub>=0.20.0
Requires-Dist: numpy>=1.22.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: uvicorn>=0.28.0
Description-Content-Type: text/markdown

# Pocket-O-Llama

<p align="center">
  <img src="Logo.png" alt="Pocket-O-Llama Logo" width="350" height="220">
</p>

<p align="center">
  <strong>Your local, high-precision GGUF model server.</strong><br>
  <em>Designed to run smoothly on low-spec hardware without throttling your system.</em>
</p>

<p align="center">
  <a href="https://www.python.org/"><img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="Python Version"></a>
  <a href="https://github.com/Hrutu34/Pocket-O-Llama/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License"></a>
</p>

---

## 🚀 Why Pocket-O-Llama?

Running local LLMs shouldn't require a supercomputer or a massive, bloated installation. Pocket-O-Llama is a lightweight, hardware-agnostic alternative to heavier engines. It is designed from the ground up to maximize the performance of highly quantized models (like those from **Unsloth AI**) on everyday laptops and desktops.

* **Native Hugging Face Hub Integration:** Pass any GGUF repository ID (e.g., `unsloth/Llama-3.2-1B-Instruct-GGUF`), and the server will auto-resolve, download, and cache the optimal quantization for you.
* **True Zero-Copy Streaming:** Real-time token streaming using `llama_cpp` for instant Time-To-First-Token.
* **Interactive Web Dashboard:** Comes with a beautiful, built-in dark-mode UI with Markdown rendering, model switching, and real-time conversation history.
* **OpenAI Drop-In Replacement:** Fully schema-compatible with `/v1/chat/completions`, meaning it instantly works with your existing LangChain, LlamaIndex, or AutoGen scripts.
* **Zero Port Conflicts:** Runs independently on custom ports (default `11435`) so it never clashes with other local services.

---

## 📦 Installation

Clone the repository and install it locally via pip:

```bash
cd Pocket-O-Llama
python -m pip install .

```

or

Install Pocket-O-Llama directly via `pip`. It is recommended to use a virtual environment.

```bash
pip install pocket-o-llama
```

---

## ⚙️ Usage

Launch your local GGUF model server directly from the command line by passing the path to your model file:

```powershell
# Example 1: Auto-download and run an Unsloth model from Hugging Face
pocket-chat --model unsloth/Llama-3.2-1B-Instruct-GGUF

# Example 2: Run a local .gguf file with custom thread allocation
pocket-chat --model "/path/to/your/model.gguf" --port 11435 --threads 4

```
Once running, open your browser and navigate to http://localhost:11435 to access the interactive chat dashboard!

### Command-Line Arguments

| Argument | Description | Default |
| --- | --- | --- |
| `--model` | Hugging Face Repo ID or absolute path to a local .gguf file | `unsloth/Llama-3.2-1B-Instruct-GGUF` |
| `--port` | Port to run the FastAPI server on | `11435` |
| `--threads` | Maximum CPU threads to allocate | `4` |

---

## 🧪 API Usage/Testing

Pocket-O-Llama acts as a drop-in replacement for OpenAI/Gemini/Anthropic API. You can hit the /v1/chat/completions endpoint exactly as you normally would.

Using Python requests:

```python
import requests

response = requests.post(
    "http://localhost:11435/v1/chat/completions",
    json={
        "messages": [
            {"role": "system", "content": "You are a helpful, brilliant coding assistant."},
            {"role": "user", "content": "Write a Python function to calculate the Fibonacci sequence."}
        ]
    }
)

print(response.json()["choices"][0]["message"]["content"])

```

Using cURL (Streaming):

```bash
curl -X POST http://localhost:11435/api/chat \
-H "Content-Type: application/json" \
-d '{
  "model": "unsloth/Llama-3.2-1B-Instruct-GGUF",
  "messages": [{"role": "user", "content": "Explain quantum computing in one sentence."}],
  "stream": true
}'
```



---

### An effort by **[Hrutu Surve](https://www.linkedin.com/in/hrutu-surve-767228253/)**.
Empowering developers to run AI locally, efficiently, and privately.