Metadata-Version: 2.4
Name: pipecat-boson
Version: 0.1.0
Summary: Pipecat realtime LLM service for the Boson Realtime API
License-Expression: BSD-2-Clause
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: loguru<1,>=0.7.3
Requires-Dist: pipecat-ai<2,>=1.4.0
Requires-Dist: websockets>=13.1
Provides-Extra: webrtc
Requires-Dist: pipecat-ai[runner,webrtc]<2,>=1.4.0; extra == "webrtc"
Requires-Dist: python-dotenv<2,>=1.0.0; extra == "webrtc"
Provides-Extra: test
Requires-Dist: pytest==8.4.2; extra == "test"
Requires-Dist: pytest-asyncio==1.2.0; extra == "test"
Provides-Extra: dev
Requires-Dist: pipecat-ai[runner,webrtc]<2,>=1.4.0; extra == "dev"
Requires-Dist: pytest==8.4.2; extra == "dev"
Requires-Dist: pytest-asyncio==1.2.0; extra == "dev"
Requires-Dist: python-dotenv<2,>=1.0.0; extra == "dev"
Requires-Dist: ruff==0.3.4; extra == "dev"
Dynamic: license-file

# Boson Pipecat Realtime Service

This package lets Pipecat pipelines use the Boson Realtime API as a
speech-to-speech `LLMService`.

This integration is maintained by Boson AI for the Boson Realtime API.

It follows Boson's OpenAI-compatible realtime interface:

- `output_modalities` is exactly `["audio"]` or `["text"]`.
- Audio defaults to 24 kHz PCM input/output.
- Server VAD is enabled by default.
- User transcript events are emitted only when `input_audio_transcription.model`
  is non-empty.
- Input noise reduction uses OpenAI's object shape:
  `{"type": "near_field"}` or `{"type": "far_field"}`.
- `voice` is a string preset or voice id.
- `truncation` is `"auto"` or `"disabled"`.
- Per-response output modality overrides are not supported. Configure
  text-only output at the session level with `modalities=["text"]`; Pipecat
  `LLMConfigureOutputFrame(skip_tts=True)` is ignored by this service.

## Installation

Install the package from PyPI:

```bash
uv add pipecat-boson
```

The core service does not require WebRTC. Install the WebRTC extra only when
you want to run the browser example or use Pipecat's WebRTC transport:

```bash
uv add "pipecat-boson[webrtc]"
```

For local development from this repository, use:

```bash
uv sync --extra dev
```

Or run the example directly with `uv run --extra webrtc`, as shown below.

## Prerequisites

- A Boson API key.
- A Boson Realtime API WebSocket endpoint.
- Python 3.11 or newer.

## Local development

This package is intentionally isolated from Boson's backend Python environment.
It talks to Boson only through the realtime WebSocket API.

Run commands from the repository root:

```bash
cp .env.example .env
```

## Run the Browser Example

The example enables user transcripts, typed chat display handling, and a local
`get_weather` function tool. It supports Pipecat's WebRTC and WebSocket
transports. WebRTC needs the `webrtc` extra, which installs `cv2` through
`opencv-python`.

Set the connection values in `.env`:

```bash
BOSON_REALTIME_URL="ws://localhost:12344/v1/realtime/"
BOSON_API_KEY="..."
BOSON_PIPECAT_LOG_LEVEL=INFO
```

Then run the example from the repository root:

```bash
uv run --extra webrtc --python 3.12 \
  python examples/pipecat_boson_realtime_agent.py \
    -t webrtc \
    --host 127.0.0.1 \
    --port 7860
```

For SSH tunneling or other environments where WebRTC ICE cannot reach the
server, run the same example with the WebSocket transport:

```bash
uv run --extra webrtc --python 3.12 \
  python examples/pipecat_boson_realtime_agent.py \
    -t websocket \
    --host localhost \
    --port 7860
```

Open `http://localhost:7860` in the browser. Select the matching transport in
the page header before connecting. Use `localhost` for local plain WebSocket
runs; Pipecat's development runner advertises `wss://` for other host values.
The example enables user transcripts by default with
`BOSON_ASR_MODEL=higgs-audio-understanding-v3-asr`. Set `BOSON_ASR_MODEL` to a
different ASR model if your backend uses another name, or set it to `none` to
suppress client-facing user transcript events.

The example also registers a local `get_weather` function tool. Ask something
like "what is the weather in Shanghai?" to exercise realtime function calling;
the example logs the tool call and returns a mock weather result.

## Runtime Compatibility

This package targets `pipecat-ai>=1.4.0,<2` and has been tested with Pipecat
`1.5.0`. The only runtime contract is the realtime WebSocket protocol exposed
at `BOSON_REALTIME_URL`.

## Usage

```python
from pipecat_boson.realtime import BosonRealtimeLLMService

llm = BosonRealtimeLLMService(
    url="ws://localhost:12344/v1/realtime/",
    api_key="...",
    model="Qwen2.5-72B-Instruct",
    voice="en_woman",
    instructions="You are a concise voice assistant.",
)
```

Pipeline usage sketch:

The snippet below assumes `transport` is a configured Pipecat transport. For a
complete runnable version, see `examples/pipecat_boson_realtime_agent.py`.

```python
from pipecat.frames.frames import LLMRunFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import LLMContextAggregatorPair

from pipecat_boson.realtime import BosonRealtimeLLMService


llm = BosonRealtimeLLMService(
    url="ws://localhost:12344/v1/realtime/",
    api_key="...",
    model="Qwen2.5-72B-Instruct",
    voice="default",
)

context = LLMContext()
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
    context,
    realtime_service_mode=True,
)

pipeline = Pipeline(
    [
        transport.input(),
        user_aggregator,
        llm,
        transport.output(),
        assistant_aggregator,
    ]
)

worker = PipelineWorker(
    pipeline,
    params=PipelineParams(enable_metrics=True, enable_usage_metrics=True),
)

await worker.queue_frames([LLMRunFrame()])
```

With supported Pipecat versions, create context aggregators with realtime
service mode:

```python
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
    context,
    realtime_service_mode=True,
)
```

For realtime services, this writes user messages to context when the assistant
response starts, which matches Pipecat's expected context semantics for
server-driven turn detection.

To receive user transcripts:

```python
llm = BosonRealtimeLLMService(
    url="ws://localhost:12344/v1/realtime/",
    api_key="...",
    model="Qwen2.5-72B-Instruct",
    input_audio_transcription={"model": "whisper-1", "language": "zh"},
)
```

Omitting transcription, passing `None`, or passing a dict without `model` still
allows Boson to run ASR internally, but no client-facing transcript events
are emitted.

Advanced: use Pipecat/local VAD instead of server VAD:

```python
llm = BosonRealtimeLLMService(
    url="ws://localhost:12344/v1/realtime/",
    api_key="...",
    model="Qwen2.5-72B-Instruct",
    turn_detection=False,
)
```

When server VAD is disabled, the surrounding Pipecat pipeline must include
local turn-detection processors that emit user turn boundary frames. The
included WebRTC examples use Boson's server VAD by default; changing only this
service option to `False` is not enough for voice turns to commit.

For text-only output:

```python
llm = BosonRealtimeLLMService(
    url="ws://localhost:12344/v1/realtime/",
    api_key="...",
    model="Qwen2.5-72B-Instruct",
    modalities=["text"],
)
```

Text-only sessions stream `response.output_text.delta` and do not emit audio
frames.

Boson session lifecycle extensions are exposed as service events:
`on_session_created` receives the full `session.created` event, including the
provider session ID in `event.session.id`;
`on_session_terminated` receives `session.idle_timeout` and
`session.max_duration_reached`; `on_should_end_call` receives the
`should_end_call` signal. The service does not automatically close Pipecat
transports for these events.

## License

BSD-2-Clause.
