Metadata-Version: 2.4
Name: memosift
Version: 0.1.0
Summary: Context compression engine for AI agents — deterministic, framework-agnostic middleware
Project-URL: Homepage, https://github.com/memosift/memosift
Project-URL: Documentation, https://memosift.dev/docs
Project-URL: Repository, https://github.com/memosift/memosift
Project-URL: Issues, https://github.com/memosift/memosift/issues
Project-URL: Changelog, https://github.com/memosift/memosift/blob/main/CHANGELOG.md
Author-email: MemoSift <hello@memosift.dev>
License: MIT
Keywords: agents,anthropic,compression,context,context-window,google-adk,langchain,llm,openai,token-management
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.12
Provides-Extra: all
Requires-Dist: anthropic>=0.40; extra == 'all'
Requires-Dist: google-adk>=1.16; extra == 'all'
Requires-Dist: langchain-core>=0.3; extra == 'all'
Requires-Dist: openai>=1.50; extra == 'all'
Requires-Dist: scikit-learn>=1.5; extra == 'all'
Requires-Dist: tiktoken>=0.8; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: coverage>=7.0; extra == 'dev'
Requires-Dist: hypothesis>=6.100; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.8; extra == 'dev'
Provides-Extra: google-adk
Requires-Dist: google-adk>=1.16; extra == 'google-adk'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.3; extra == 'langchain'
Provides-Extra: openai
Requires-Dist: openai>=1.50; extra == 'openai'
Provides-Extra: scikit
Requires-Dist: scikit-learn>=1.5; extra == 'scikit'
Provides-Extra: tiktoken
Requires-Dist: tiktoken>=0.8; extra == 'tiktoken'
Description-Content-Type: text/markdown

# MemoSift

**Your AI agent dies at 128K tokens. MemoSift keeps it alive.**

Deterministic context compression engine that sits between your agent and the LLM. Zero LLM calls, sub-2s latency, 97% fact retention. Drop-in adapters for OpenAI, Anthropic, Google ADK, and LangChain.

## Install

```bash
# Python — core (zero dependencies)
pip install memosift

# With framework adapters
pip install memosift[openai]
pip install memosift[anthropic]
pip install memosift[langchain]
pip install memosift[google-adk]
pip install memosift[all]
```

```bash
# TypeScript
npm install memosift
```

## Quick Start

```python
from memosift import compress, MemoSiftConfig

messages = [...]  # Your conversation history
compressed, report = await compress(messages)

print(f"{report.compression_ratio:.1f}x compression")
print(f"{report.original_tokens:,} → {report.compressed_tokens:,} tokens")
```

### With OpenAI

```python
from memosift.adapters.openai_sdk import compress_openai_messages

compressed_msgs, report = await compress_openai_messages(messages)
# Feed compressed_msgs directly to client.chat.completions.create()
```

### With Anthropic

```python
from memosift.adapters.anthropic_sdk import compress_anthropic_messages

result, report = await compress_anthropic_messages(
    messages, system=system_prompt, client=client
)
# result.system and result.messages go directly to client.messages.create()
```

### With LangChain

```python
from memosift.adapters.langchain import compress_langchain_messages

compressed_msgs, report = await compress_langchain_messages(messages)
```

## How It Works

MemoSift compresses LLM conversation context through a 6-layer pipeline:

```
Messages In
  → L1: Classify content types (deterministic)
  → L2: Deduplicate via SHA-256 + TF-IDF (deterministic)
  → L3: Compress per-type — 7 engines (deterministic, Engine D optional LLM)
  → L4: Score relevance (deterministic or optional LLM)
  → L5: Optimize position for attention curve (deterministic, disabled by default)
  → L6: Enforce token budget (deterministic)
Messages Out + CompressionReport
```

### Key Properties

- **Zero external dependencies** in core — no ML models bundled
- **Lossless by default** — only dedup and verbatim deletion unless you opt in
- **Framework-agnostic** — operates on `MemoSiftMessage[]`, adapters handle conversion
- **Tool call integrity** — if a tool_call survives, its tool_result survives too
- **Deterministic layers < 100ms** for 100K tokens
- **Fault-tolerant** — any layer that throws is skipped, never crashes the pipeline

## Configuration

```python
from memosift import MemoSiftConfig

# Use a preset
config = MemoSiftConfig.preset("coding")  # or: research, support, data, general

# Or customize
config = MemoSiftConfig(
    recent_turns=3,
    token_budget=100_000,
    enable_summarization=False,  # True requires an LLM provider
)

compressed, report = await compress(messages, config=config)
```

## Anchor Ledger

MemoSift extracts and preserves critical facts (file paths, errors, decisions, identifiers) in an append-only ledger that survives compression:

```python
from memosift import compress, AnchorLedger

ledger = AnchorLedger()
compressed, report = await compress(messages, ledger=ledger)

# Ledger contains extracted facts
for fact in ledger.facts:
    print(f"[{fact.category}] {fact.content}")
```

## Cross-Window Compression

For long-running agents, share dedup state across compression cycles:

```python
from memosift import compress, CrossWindowState

state = CrossWindowState()

# Window 1
compressed_1, _ = await compress(window_1, cross_window=state)

# Window 2 — dedup catches repeats from window 1
compressed_2, _ = await compress(window_2, cross_window=state)
```

## Adapters

| Adapter | Input | Output | Function |
|---------|-------|--------|----------|
| OpenAI SDK | OpenAI dicts | OpenAI dicts | `compress_openai_messages()` |
| Anthropic SDK | Anthropic blocks + system | `AnthropicCompressedResult` | `compress_anthropic_messages()` |
| Claude Agent SDK | Agent SDK objects/dicts | Anthropic-compatible dicts | `compress_agent_sdk_messages()` |
| Google ADK | ADK events | ADK events | `compress_adk_events()` |
| LangChain | LangChain messages/dicts | LangChain-compatible dicts | `compress_langchain_messages()` |

All adapters support lossless round-trip conversion.

## Development

```bash
# Python
cd python && pip install -e ".[dev]"
cd .. && pytest tests/python/ -x -q

# TypeScript
cd typescript && npm install && npm test

# Lint
cd python && ruff format src/ && ruff check src/
cd typescript && npm run lint
```

## License

MIT
