Metadata-Version: 2.4
Name: mobile-agent-mcp
Version: 0.1.0
Summary: MCP server for Android device automation via ADB and uiautomator2
Author-email: Kamal Zala <kamalzala07@gmail.com>
License: MIT
Keywords: adb,android,automation,mcp,uiautomator2
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Requires-Dist: lxml>=4.9.0
Requires-Dist: mcp>=1.0.0
Requires-Dist: openai>=1.0.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: pydantic-settings>=2.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: uiautomator2>=3.0.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Description-Content-Type: text/markdown

# mobile-agent-mcp

MCP server for Android device automation. Exposes 23 tools over stdio so an LLM orchestrator can control an Android device via ADB and uiautomator2.

## Prerequisites

- Python 3.10+
- ADB (`android-platform-tools`) in `PATH`
- Android device or emulator with USB debugging enabled
- An OpenAI-compatible LLM endpoint for text reasoning (e.g. DeepSeek)
- Optionally a separate vision-capable LLM endpoint for SoM fallback (e.g. GPT-4o)

## Installation

```bash
pip install -e .
```

## Configuration

Create a `.env` file in the project root (or set environment variables directly):

| Variable | Required | Default | Description |
|---|---|---|---|
| `TEXT_LLM_API_KEY` | Yes | — | API key for text/reasoning LLM |
| `TEXT_LLM_BASE_URL` | No | `https://api.deepseek.com` | Base URL for text LLM |
| `TEXT_LLM_MODEL` | No | `deepseek-chat` | Model name for text LLM |
| `VISION_LLM_API_KEY` | No | — | API key for vision LLM (falls back to text LLM) |
| `VISION_LLM_BASE_URL` | No | — | Base URL for vision LLM |
| `VISION_LLM_MODEL` | No | `gpt-4o` | Model name for vision LLM |
| `ANDROID_SERIAL` | No | — | Device serial (auto-detected if only one device) |
| `SCREENSHOT_DIR` | No | `~/.mobile_agent_mcp/screenshots` | Screenshot storage path |
| `DB_PATH` | No | `~/.mobile_agent_mcp/mobile_agent_mcp.db` | SQLite skill cache path |

## Setup

### 1. Connect your Android device

Enable **USB debugging** on the device (`Settings → Developer Options → USB Debugging`), plug it in via USB, then verify the connection:

```bash
adb devices
python scripts/check_device.py
```

`check_device.py` runs a full diagnostic: ADB connection, shell access, uiautomator2, and accessibility tree dump.

### 2. Configure environment

```bash
# Copy the example and fill in your LLM API key
copy .env.example .env   # Windows
cp .env.example .env     # macOS / Linux
```

Minimum required setting in `.env`:
```
TEXT_LLM_API_KEY=your_api_key_here
```

## Running

```bash
# Via installed script
mobile-agent-mcp

# Or directly
python -m mobile_agent_mcp.server
```

The server communicates over **stdio** (JSON-RPC). Logs are written to stderr; MCP messages flow on stdin/stdout.

## Claude Desktop Integration

### Step 1 — Find your config file

| Platform | Path |
|---|---|
| Windows | `%APPDATA%\Claude\claude_desktop_config.json` |
| macOS | `~/Library/Application Support/Claude/claude_desktop_config.json` |

### Step 2 — Add the server

```json
{
  "mcpServers": {
    "mobile-agent": {
      "command": "mobile-agent-mcp"
    }
  }
}
```

If `mobile-agent-mcp` is not on your system PATH, use the full path to the script:

```json
{
  "mcpServers": {
    "mobile-agent": {
      "command": "C:/path/to/venv/Scripts/mobile-agent-mcp.exe",
      "env": {
        "TEXT_LLM_API_KEY": "your_api_key_here"
      }
    }
  }
}
```

> **Tip:** The `env` block is an alternative to a `.env` file — useful when running via Claude Desktop so you do not need to configure environment variables separately.

### Step 3 — Restart Claude Desktop

After saving the config, restart Claude Desktop. It will launch the MCP server automatically on startup. You should see 23 tools available in the tool panel.

## Available Tools

| Tool | Description |
|---|---|
| `list_installed_apps` | List installed package names, with optional filter query |
| `open_app` | Launch an app by display name or package name |
| `open_url` | Open a URL in the default browser |
| `read_screen` | Capture full screen state as an element list |
| `get_screen_layout` | Compact human-readable screen summary |
| `find_element` | Locate a UI element by natural-language description |
| `tap_element` | Tap a UI element by `element_id` |
| `type_into` | Focus an input field and type text |
| `swipe` | Swipe in a direction: `up`, `down`, `left`, `right` |
| `press_key` | Press `back`, `home`, `recent`, or `enter` |
| `wait_for` | Wait until a condition string appears on screen |
| `send_intent` | Fire a raw Android intent via `am start` |
| `recall_skill` | Look up a previously saved skill by goal description |
| `save_skill` | Persist a successful action sequence for future reuse |
| `list_skills` | List all stored skills ordered by most recently used |
| `delete_skill` | Delete a stored skill by its numeric id |
| `long_press` | Long-press an element to open context menus or start drag |
| `get_device_info` | Return model, Android version, screen size, battery level |
| `scroll_to_element` | Scroll until a described element comes into view |
| `take_screenshot` | Capture the screen and return the local PNG path |
| `pinch_zoom` | Two-finger pinch gesture to zoom in or out |
| `clear_field` | Clear all text from an input field |
| `device_status` | Check if the device is connected and responsive |

## How It Works

```
MCP client (Claude / orchestrator)
        │  stdio (JSON-RPC)
        ▼
  mobile-agent-mcp server
        │
        ├── Perception
        │     ├── accessibility.py  — fast path: uiautomator2 XML tree
        │     └── som.py            — fallback: screenshot + vision LLM
        │
        ├── Control
        │     ├── adb.py            — ADB shell commands
        │     ├── device.py         — uiautomator2 device connection
        │     └── intents.py        — Android intent helpers
        │
        └── Skills
              └── store.py          — SQLite cache for reusable action sequences
```

**Perception strategy:** Each `read_screen` / `find_element` call first tries the accessibility tree (fast, no LLM needed). If fewer than `SOM_TRIGGER_MIN_ELEMENTS` (default: 3) clickable elements are found — e.g. on a game or WebView screen — it falls back to Set-of-Marks (SoM): a screenshot is annotated with numbered boxes and a vision LLM picks the matching element.

**Skill cache:** After a goal is successfully achieved the orchestrator can call `save_skill` to store the action sequence. On future runs `recall_skill` retrieves it, avoiding re-planning. Credentials in action arguments are automatically redacted before storage.

## Development

```bash
pip install -e ".[dev]"
pytest
```
