Metadata-Version: 2.4
Name: logging-mixin
Version: 0.4.0
Summary: Class-bound structured logging with auto-injected correlation IDs for Python services.
Project-URL: Homepage, https://github.com/jekhator/logging-mixin
Project-URL: Repository, https://github.com/jekhator/logging-mixin.git
Project-URL: Issues, https://github.com/jekhator/logging-mixin/issues
Project-URL: Changelog, https://github.com/jekhator/logging-mixin/releases
Author: C. James Ekhator
License: Apache-2.0
License-File: LICENSE
Keywords: correlation-id,distributed-tracing,logging,observability,structured-logging
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Logging
Classifier: Typing :: Typed
Requires-Python: >=3.11
Provides-Extra: aiohttp
Requires-Dist: aiohttp; extra == 'aiohttp'
Provides-Extra: all
Requires-Dist: aiohttp; extra == 'all'
Requires-Dist: botocore; extra == 'all'
Requires-Dist: celery; extra == 'all'
Requires-Dist: grpcio; extra == 'all'
Requires-Dist: httpx; extra == 'all'
Requires-Dist: requests; extra == 'all'
Requires-Dist: urllib3; extra == 'all'
Provides-Extra: botocore
Requires-Dist: botocore; extra == 'botocore'
Provides-Extra: celery
Requires-Dist: celery; extra == 'celery'
Provides-Extra: grpc
Requires-Dist: grpcio; extra == 'grpc'
Provides-Extra: httpx
Requires-Dist: httpx; extra == 'httpx'
Provides-Extra: requests
Requires-Dist: requests; extra == 'requests'
Provides-Extra: urllib3
Requires-Dist: urllib3; extra == 'urllib3'
Description-Content-Type: text/markdown

# logging-mixin

[![PyPI version](https://img.shields.io/pypi/v/logging-mixin.svg)](https://pypi.org/project/logging-mixin/)
[![CI](https://github.com/jekhator/logging-mixin/actions/workflows/ci.yml/badge.svg)](https://github.com/jekhator/logging-mixin/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
[![Python Versions](https://img.shields.io/pypi/pyversions/logging-mixin.svg)](https://pypi.org/project/logging-mixin/)

**End-to-end correlation-ID propagation for Python services.** Automatic correlation-ID injection across logs, HTTP clients, task queues, and AWS services. Built for distributed systems.

- **Correlation-ID context** via `contextvars.ContextVar` - survives async/await, thread pools, and background tasks
- **13 adapters** for inbound/outbound/task/logging/cloud/edge-protocol scenarios
- **LoggingMixin** class and `logged` decorator for zero-boilerplate logging
- **Python 3.11+** with `uv` package management

## What It Does

Tracking a single request through a distributed system requires correlation IDs on every log line, HTTP call, database query, and background task. Traditional approaches require threading the ID through every function.

logging-mixin propagates correlation IDs automatically:

```python
from logging_mixin import LoggingMixin, set_correlation_id

# Request handler: set once
def handle_request(request):
    set_correlation_id(request.headers.get("X-Correlation-ID", "req-123"))
    service = OrderService()
    service.create_order(123)  # Correlation ID is now in the context

# Service: logs include correlation ID automatically
class OrderService(LoggingMixin):
    def create_order(self, user_id: int):
        self.log_info("order.create", user_id=user_id)
        # Logs with: {"correlation_id": "req-123", "user_id": 123, ...}
        
        # Outbound HTTP call: correlation ID injected automatically
        self.send_notification(user_id)

# Background task: inherits correlation ID from request context
@celery.shared_task
def send_notification(user_id: int):
    self = NotificationService()
    self.log_info("notification.send", user_id=user_id)
    # Same correlation ID propagates here
```

## Install

```bash
uv add logging-mixin
```

Or with pip:

```bash
pip install logging-mixin
```

With optional dependencies for specific frameworks/clients:

```bash
# Individual adapters
uv add "logging-mixin[aiohttp]"     # aiohttp client instrumentation
uv add "logging-mixin[urllib3]"     # urllib3 client instrumentation
uv add "logging-mixin[httpx]"       # HTTPX client instrumentation
uv add "logging-mixin[requests]"    # Requests client instrumentation
uv add "logging-mixin[celery]"      # Celery task propagation
uv add "logging-mixin[botocore]"    # AWS SDK instrumentation
uv add "logging-mixin[grpc]"        # gRPC server instrumentation

# Install all adapters at once
uv add "logging-mixin[all]"
```

Or with pip:

```bash
pip install "logging-mixin[all]"
```

Requires **Python 3.11+** (3.11 and 3.12 tested).

## Quick Start

### 1. Add stdlib adapter to your logging config

Stamps `correlation_id` on every log record:

```python
import logging
from logging_mixin.adapters.stdlib.stdlib_client import CorrelationLogFilter

# Add the filter to your logger
logging.basicConfig()
logging.getLogger().addFilter(CorrelationLogFilter())
```

### 2. Set correlation ID at request boundary

```python
from logging_mixin import set_correlation_id

# FastAPI
from fastapi import FastAPI, Request
app = FastAPI()

@app.middleware("http")
async def correlation_middleware(request: Request, call_next):
    set_correlation_id(request.headers.get("X-Correlation-ID", str(uuid.uuid4())))
    return await call_next(request)
```

Or use the built-in ASGI adapter:

```python
from logging_mixin.adapters.asgi.asgi_client import CorrelationIdMiddleware

app.add_middleware(CorrelationIdMiddleware)
```

### 3. Use LoggingMixin in your classes

```python
from logging_mixin import LoggingMixin

class UserService(LoggingMixin):
    def create_user(self, user_name: str):
        self.log_info("user.create", user_name=user_name)
        # Logs include correlation_id automatically
        self.save(user_name)
```

### 4. Instrument outbound clients

HTTP clients automatically inject `X-Correlation-ID` header:

```python
from logging_mixin.adapters.httpx.httpx_client import CorrelationIdInjector
from logging_mixin.adapters.requests.requests_client import CorrelationHTTPAdapter

# For httpx
client = httpx.Client(event_hooks=CorrelationIdInjector.event_hooks())
client.get("https://api.example.com")  # Sends X-Correlation-ID header

# For requests
session = requests.Session()
CorrelationHTTPAdapter.register_on_session(session)
session.get("https://api.example.com")  # Sends X-Correlation-ID header
```

AWS SDK (botocore):

```python
import boto3
from logging_mixin.adapters.botocore.botocore_client import CorrelationIdInjector

s3 = boto3.client("s3")
CorrelationIdInjector.register_on_client(s3)
s3.get_object(Bucket="my-bucket", Key="file.txt")  # Includes correlation ID in AWS service calls
```

### 5. Propagate to background tasks (Celery)

Install the optional `[celery]` dependency, then:

```python
from celery import Celery
from logging_mixin.adapters.celery.celery_client import CorrelationSignals

app = Celery()
CorrelationSignals.connect()

@app.task
def process_order(order_id: int):
    service = OrderService()
    service.log_info("processing", order_id=order_id)
    # Correlation ID from the original request is automatically here
```

## Core API

### LoggingMixin (instance methods)

```python
from logging_mixin import LoggingMixin

class MyService(LoggingMixin):
    def do_something(self):
        self.log_debug("debug message", key="value")        # DEBUG level
        self.log_info("info message", key="value")          # INFO level
        self.log_warning("warning message", key="value")    # WARNING level
        self.log_error("error message", key="value")        # ERROR level
        self.log_exception("error with traceback")          # ERROR level + traceback
```

All methods:
- Accept an event name (string) + optional keyword arguments
- Automatically inject `correlation_id` into log `extra` dict
- Read from the per-class logger (`module.ClassName`)
- Support composition with masking mixins (call `mask_for_logging()` if it exists)

### Correlation context

```python
from logging_mixin import get_correlation_id, set_correlation_id, clear_correlation_id

cid = get_correlation_id()            # Get current correlation ID (None if not set)
set_correlation_id("my-request-id")   # Set manually (tests, background tasks)
clear_correlation_id()                # Clear (test isolation, request boundaries)
```

### Logged decorator

Decorate `LoggingMixin` methods to auto-log entry/exit and errors:

```python
from logging_mixin import LoggingMixin, logged

class StripeClient(LoggingMixin):
    @logged("stripe.create_intent")
    def create_intent(self, customer_id: str) -> dict:
        return {"status": "ok"}

# Logs "stripe.create_intent.start" on entry
# Logs "stripe.create_intent.error" on exception (with error_type and code)
```

See `docs/apps/decorators/logged.md` for detailed usage and composability with `@phi_aware` and `@translate` decorators.

## The 13 Adapters

All adapters live in `logging_mixin/adapters/`. Each adapter is optional except `stdlib` (included with the core package). Install specific adapters via extras:

| Adapter | Purpose | Category | Install | Best For |
|---------|---------|----------|---------|----------|
| **ASGI** | Extract/generate correlation ID from ASGI requests; inject into responses | Inbound HTTP | (core) | FastAPI, Starlette, Quart |
| **WSGI** | Extract/generate correlation ID from WSGI requests; inject into responses | Inbound HTTP | (core) | Django, Flask, Pyramid |
| **WebSocket** | Extract/generate correlation ID from WebSocket handshake headers | Inbound Edge | (core) | Starlette, Channels |
| **gRPC** | Extract/generate correlation ID from gRPC invocation metadata via server interceptor | Inbound Edge | `[grpc]` | gRPC servers |
| **GraphQL** | Inject correlation ID into resolver context for downstream resolvers | Inbound Edge | (core) | Strawberry, Ariadne |
| **Stdlib** | Stamp correlation ID on all `logging.LogRecord` objects | Output Sink | (core) | All Python logging |
| **HTTPX** | Inject correlation ID into outbound HTTPX requests | Outbound HTTP | `[httpx]` | HTTPX clients (sync/async) |
| **Requests** | Inject correlation ID into outbound Requests HTTP requests | Outbound HTTP | `[requests]` | Requests sessions |
| **aiohttp** | Inject correlation ID into outbound aiohttp requests via TraceConfig | Outbound HTTP | `[aiohttp]` | aiohttp clients (async) |
| **urllib3** | Inject correlation ID into outbound urllib3 requests | Outbound HTTP | `[urllib3]` | urllib3 PoolManager |
| **Botocore** | Inject correlation ID into AWS SDK (boto3) service calls | AWS Cloud | `[botocore]` | Boto3 clients (S3, DynamoDB, etc.) |
| **Celery** | Propagate correlation ID across Celery task boundaries (enqueue → prerun → postrun) | Cross-Boundary Task | `[celery]` | Celery tasks and signals |
| **Cloud** | Extract correlation ID from Lambda event (API Gateway v1/v2, ALB, SQS, SNS, EventBridge); auto-generate fallback | Serverless | (core) | AWS Lambda handlers |

### Adapter Details

#### Inbound (Request Entry Points)

**ASGI** (`docs/apps/adapters/asgi.md`)
- Extract `X-Correlation-ID` header from ASGI scope, or generate UUID4 hex[:12]
- Validate against unsafe chars (CRLF, null bytes, oversized >128 bytes)
- Inject safe correlation ID into response headers
- Works alongside WSGI if needed

**WSGI** (`docs/apps/adapters/wsgi.md`)
- Extract `X-Correlation-ID` header from WSGI environ, or generate UUID4 hex[:12]
- Validate against unsafe chars (CRLF, null bytes, oversized >128 bytes)
- Inject safe correlation ID into response headers via `start_response`
- Supports legacy/synchronous frameworks

**Cloud** (`docs/apps/adapters/cloud.md`)
- Extract correlation ID from Lambda event: API Gateway (v1/v2), ALB, SQS, SNS, EventBridge, or direct invoke
- Generate fallback UUID4 hex[:12] if not present
- Return event-specific extraction strategy
- No response injection (serverless context)

#### Inbound (Edge Protocols)

**WebSocket** (`docs/apps/adapters/websocket.md`)
- Extract `x-correlation-id` from WebSocket handshake headers (ASGI scope)
- Generate UUID4 hex[:12] if not present or unsafe
- Set context for WebSocket connection lifecycle
- ASGI middleware for Starlette, Channels, or any ASGI framework

**gRPC** (`docs/apps/adapters/grpc.md`)
- Extract `x-correlation-id` from gRPC invocation metadata
- Generate UUID4 hex[:12] if not present or unsafe
- Set context via server interceptor for handler execution
- Works with all gRPC service definitions

**GraphQL** (`docs/apps/adapters/graphql.md`)
- Read correlation ID from upstream context (set by ASGI/WSGI)
- Inject into resolver context dict for downstream resolvers
- Supports Strawberry, Ariadne, and other frameworks
- No external dependencies

#### Outbound (Propagate Downstream)

**HTTPX** (`docs/apps/adapters/httpx.md`)
- Inject `X-Correlation-ID` header via event hooks
- Works with `httpx.Client` and `httpx.AsyncClient`
- Silent passthrough if correlation ID unset or unsafe

**Requests** (`docs/apps/adapters/requests.md`)
- Inject `X-Correlation-ID` header via HTTP adapter registration on `requests.Session`
- Silent passthrough if correlation ID unset or unsafe

**aiohttp** (`docs/apps/adapters/aiohttp.md`)
- Inject `X-Correlation-ID` header via TraceConfig on_request_start hook
- Works with `aiohttp.ClientSession` (async)
- Silent passthrough if correlation ID unset or unsafe

**urllib3** (`docs/apps/adapters/urllib3.md`)
- Inject `X-Correlation-ID` header via PoolManager subclass `urlopen()` override
- Works with any urllib3 request
- Silent passthrough if correlation ID unset or unsafe

**Botocore** (`docs/apps/adapters/botocore.md`)
- Hook into botocore event system to inject correlation ID into service call metadata
- Works with all boto3 service clients (S3, DynamoDB, SQS, etc.)
- Includes as custom header in HTTP-based services

#### Task/Async Boundaries

**Celery** (`docs/apps/adapters/celery.md`)
- Connect signal handlers to Celery's task lifecycle
- Propagate correlation ID across `task-prerun`, `task-postrun`, and async results
- Preserve context for delayed/scheduled tasks

#### Output Sink

**Stdlib** (`docs/apps/adapters/stdlib.md`)
- Add as a `logging.Filter` to any logger
- Stamp `correlation_id` on every `LogRecord` (from context-var)
- Zero dependencies; works with all Python logging handlers

See `docs/apps/adapters/` for detailed per-adapter documentation, security considerations, and integration patterns.

## Correlation-ID Semantics

### Format and Validation

**Correlation IDs** are strings used to track a request through a distributed system. logging-mixin enforces:

- **Length:** 1–128 bytes (UTF-8 encoded)
- **Safe characters:** No CRLF (`\r\n`), null bytes (`\0`), or control characters
- **Generation:** UUID4 hex format, shortened to first 12 chars (`hex[:12]`) for readability

**Validation occurs at:**
- **Inbound (ASGI, WSGI, Cloud):** Extract from headers/events; validate; regenerate UUID4 if unsafe
- **Outbound (HTTPX, Requests, Botocore):** Extract from context; skip injection (silent passthrough) if unset or unsafe
- **Stdlib filter:** Direct passthrough (no validation at output sink)

### Unsafe-Character Rejection

Inbound adapters use a reject-and-regenerate pattern:

```python
def _is_safe(value: str) -> bool:
    if not value or len(value) > 128:
        return False
    if any(c in value for c in {'\r', '\n', '\0'}):
        return False
    return True

# Usage in ASGI/WSGI
if _is_safe(correlation_id):
    # Use the provided ID
else:
    # Generate fresh UUID4 hex[:12]
    correlation_id = uuid4().hex[:12]
```

This defense-in-depth approach prevents:
- **Log injection attacks:** CRLF in correlation ID cannot break into separate log lines
- **HTTP header injection:** Control characters cannot be injected into response headers
- **DoS via oversized values:** 128-byte cap prevents memory exhaustion

### Fallback Generation

When a request lacks a correlation ID header, inbound adapters generate one:

```python
import uuid

correlation_id = uuid4().hex[:12]  # 12-char hex string, e.g. "a1b2c3d4e5f6"
```

This ensures every request is traceable, even if upstream services don't provide one.

### Context Propagation Semantics

**ContextVar behavior:**
- Set once per request entry point (ASGI, WSGI, Cloud middleware)
- Automatically inherited by child tasks, async calls, and thread-pool workers
- Must be cleared on request exit (middleware `finally` block)

**Async safety:**
- Survives `asyncio` context switches
- Works across `async def`, `await`, task spawning
- Context-local per request (not global)

**Task queue propagation (Celery):**
- On task enqueue: Celery signal extracts current correlation ID from context, attaches to task
- On task execution: Celery signal sets correlation ID into new worker's context
- On task completion: Signal clears context

## Troubleshooting and FAQ

### No correlation ID appearing in logs

**Symptom:** Logs include no `correlation_id` field.

**Causes and fixes:**

1. **Stdlib filter not added:** Ensure `CorrelationLogFilter()` is registered with your logger.
   ```python
   import logging
   from logging_mixin.adapters.stdlib.stdlib_client import CorrelationLogFilter
   
   logging.getLogger().addFilter(CorrelationLogFilter())
   ```

2. **Correlation ID never set:** No inbound adapter running, or `set_correlation_id()` was never called.
   ```python
   from logging_mixin import set_correlation_id
   
   set_correlation_id("my-request-id")  # Call at request entry
   ```

3. **Correlation ID cleared before use:** If `clear_correlation_id()` was called before logging, context is unset.
   ```python
   from logging_mixin import clear_correlation_id
   
   # Only call in cleanup/finally, not in the middle of request handling
   ```

### Async context not propagating

**Symptom:** Correlation ID works in sync code but disappears in `async def` or task queue.

**Causes and fixes:**

1. **Using `asyncio.create_task()` without context copy:** `asyncio.create_task()` in Python 3.11+ inherits context automatically; ensure you're not manually clearing it.
   ```python
   # This works (context inherited)
   asyncio.create_task(async_handler())
   
   # This does NOT work (new context)
   threading.Thread(target=handler).start()  # Use asyncio.run_in_executor instead
   ```

2. **Thread-pool without context propagation:** If using `ThreadPoolExecutor`, correlation ID does not automatically propagate to worker threads.
   ```python
   from logging_mixin import get_correlation_id, set_correlation_id
   
   def worker():
       cid = get_correlation_id()  # Will be None in thread
       # Manually set if needed
       set_correlation_id(cid)
   
   # Better: use asyncio.run_in_executor
   loop.run_in_executor(None, worker)  # Still won't have context
   ```

3. **Celery tasks not receiving correlation ID:** Ensure `CorrelationSignals.connect()` was called at app startup.
   ```python
   from logging_mixin.adapters.celery.celery_client import CorrelationSignals
   
   celery_app = Celery()
   CorrelationSignals.connect()  # Call at startup
   ```

### Correlation ID header not injected into outbound requests

**Symptom:** Outbound HTTPX/Requests calls don't include `X-Correlation-ID` header.

**Causes and fixes:**

1. **Adapter not registered:** HTTPX/Requests adapters are optional; ensure you installed the extra and registered the adapter.
   ```bash
   uv add "logging-mixin[httpx]"
   ```
   ```python
   from logging_mixin.adapters.httpx.httpx_client import CorrelationIdInjector
   
   client = httpx.Client(event_hooks=CorrelationIdInjector.event_hooks())
   ```

2. **Correlation ID not set in context:** No inbound adapter set the ID, or context was cleared.
   ```python
   from logging_mixin import set_correlation_id
   
   set_correlation_id("test-id")  # Set before making outbound call
   ```

3. **Unsafe correlation ID (skipped):** If the context has an unsafe correlation ID (oversized, contains CRLF), outbound adapters silently skip injection.
   ```python
   from logging_mixin import set_correlation_id
   
   # This will be skipped (>128 bytes)
   set_correlation_id("x" * 200)
   
   # Use a safe value
   set_correlation_id("safe-id-123")
   ```

### AWS Lambda events not extracting correlation ID

**Symptom:** Cloud adapter not finding correlation ID in Lambda event.

**Causes and fixes:**

1. **Wrong event source:** Cloud adapter only recognizes API Gateway v1/v2, ALB, SQS, SNS, EventBridge. Other event types fall back to UUID generation.
   ```python
   from logging_mixin.adapters.cloud.cloud_client import CloudSetup
   
   # Supported event structures
   # - API Gateway v1/v2 (request/requestContext)
   # - ALB (headers)
   # - SQS (messageAttributes)
   # - SNS (MessageAttributes)
   # - EventBridge (detail)
   ```

2. **Header name mismatch:** Ensure upstream API Gateway is configured to pass `X-Correlation-ID` header.
   ```python
   # In API Gateway settings, allow X-Correlation-ID to pass through
   # (default is often restricted)
   ```

3. **Manual extraction:** If your event source is custom, manually extract and set the ID.
   ```python
   from logging_mixin import set_correlation_id
   
   def lambda_handler(event, context):
       cid = event.get("correlation_id") or str(uuid.uuid4())
       set_correlation_id(cid)
       ...
   ```

## Public API

The top-level `logging_mixin` package exports:

```python
from logging_mixin import (
    # Core
    LoggingMixin,                    # Base class for logging
    logged,                          # Decorator for auto-logging entry/exit
    LoggedContainer,                 # DTO for @logged internals
    
    # Correlation context
    CorrelationContext,              # DTO for context value
    ContextVarClient,                # Context manager (rarely used directly)
    get_correlation_id,              # Read current correlation ID
    set_correlation_id,              # Set correlation ID
    clear_correlation_id,            # Clear correlation ID
    
    # API contract
    PUBLIC_API,                      # frozenset of public names (for introspection)
)
```

Adapter-specific exports are available via subpackages:

```python
from logging_mixin.adapters.asgi import CorrelationIdMiddleware
from logging_mixin.adapters.wsgi import CorrelationIdMiddleware
from logging_mixin.adapters.httpx import CorrelationIdInjector
from logging_mixin.adapters.requests import CorrelationHTTPAdapter
from logging_mixin.adapters.botocore import CorrelationIdInjector
from logging_mixin.adapters.celery import CorrelationSignals
from logging_mixin.adapters.cloud import CloudCorrelationExtractor
from logging_mixin.adapters.stdlib import CorrelationLogFilter
```

## Design Principles

- **ContextVar-based** - Survives async/await, thread pools, and background tasks
- **Instance-only** - LoggingMixin methods read `self._logger` (cannot be used in `@classmethod`/`@staticmethod`)
- **Framework-agnostic** - Core library has zero dependencies
- **Adapter ecosystem** - Install only what you use (celery, requests, etc. are optional)
- **Security-hardened** - ASGI/WSGI adapters validate all input (CRLF injection, control characters, length limits)
- **Composable** - Works with masking mixins and other decorators

## Testing

```python
import logging
from logging_mixin import LoggingMixin, set_correlation_id

def test_logs_with_correlation_id(caplog):
    set_correlation_id("test-123")
    
    service = MyService()
    with caplog.at_level(logging.INFO):
        service.do_something()
    
    assert caplog.records[0].correlation_id == "test-123"
```

## Design Trade-offs

- **@classmethod/@staticmethod** - LoggingMixin cannot be used there (use module logger + manual injection)
- **Implicit behavior** - Correlation ID is silently injected (can be surprising if not documented)
- **Setup required** - Must call `set_correlation_id()` at request entry or use a framework adapter

## License

Apache 2.0 - see LICENSE file.
