Add a lightweight chat simulator demonstrating dirty worker streaming:
- Token-by-token SSE streaming via async generators
- FastAPI endpoint with browser UI
- Multiple canned responses based on keywords
- Docker deployment with docker-compose
- Integration tests for SSE protocol
Update docs/content/dirty.md to link to both examples.
- Enhance architecture diagram with Unix socket paths, signal flow, and
heartbeat indicators
- Add process relationships table
- Expand signal handling section with flow diagram, comprehensive signal
reference table, and async handling explanation
- Add new "Liveness and Health Monitoring" section covering heartbeat
mechanism, timeout detection, parent death detection, orphan cleanup,
and respawn behavior
- Add link to embedding service example
Add comprehensive Docker integration tests verifying dirty arbiter
lifecycle under realistic conditions:
- Parent death detection via ppid monitoring
- Orphan cleanup on restart
- Dirty arbiter respawning after crash
- Graceful shutdown with SIGTERM
Also fix race condition in manage_workers() by checking self.alive
before spawning new workers during shutdown.
Add ppid monitoring to dirty arbiter's worker monitor loop. If the
main arbiter dies unexpectedly (SIGKILL, crash, OOM), the dirty
arbiter detects the parent change and shuts itself down gracefully.
This complements the existing orphan cleanup on startup.
- Close coroutines in mocked asyncio.run to prevent "never awaited" warning
- Fix flaky integration tests with proper async cleanup and try/finally
- Add uvloop to testing dependencies so uvloop test runs
- Add pytest warning filter for eventlet/asyncio incompatibility
When the main arbiter crashes and restarts, orphaned dirty arbiters
may continue running. This adds detection and cleanup:
- Add well-known PID file location based on proc_name
- Dirty arbiter writes PID on startup, removes on exit
- Main arbiter checks for orphans on fresh start (not USR2)
- Uses self.proc_name for USR2 compatibility (myapp vs myapp.2)
During USR2 upgrade, old and new dirty arbiters coexist with
separate PID files, preventing the old from removing the new's file.
Add support for streaming responses when dirty app actions return
generators (sync or async). This enables real-time delivery of
incremental results for use cases like LLM token generation.
Features:
- Streaming protocol with chunk/end/error message types
- Worker support for sync and async generators
- Arbiter forwarding of streaming messages
- Deadline-based timeout handling
- Async client streaming API
Protocol:
- Chunk messages (type: "chunk") contain partial data
- End messages (type: "end") signal stream completion
- Error messages can occur mid-stream
New files:
- benchmarks/dirty_streaming.py: Streaming benchmark suite
- tests/dirty/test_*_streaming*.py: Streaming test coverage
- docs/content/dirty.md: Streaming documentation with examples
Add tests to verify that when multiple dirty apps are configured,
messages are correctly routed to the appropriate app based on app_path.
New files:
- tests/support_dirty_apps.py: CounterApp and EchoApp test apps
- tests/dirty/test_multi_app_routing.py: 13 routing tests covering
app loading, routing, state separation, error handling, and
concurrent requests
Add a complete example demonstrating dirty workers with sentence-transformers
for text embeddings via FastAPI:
- EmbeddingApp DirtyApp that loads and manages the ML model
- FastAPI endpoints for /embed and /health
- Docker and docker-compose configuration
- Integration tests with numpy similarity checks
- GitHub Actions CI workflow
Replace lock-based request serialization with queue-based approach:
- Each worker now has a dedicated asyncio.Queue and consumer task
- route_request() submits (request, future) to queue and awaits future
- Consumer task processes requests sequentially per worker
- No lock contention - pure async queue operations
Benefits:
- Clearer separation of concerns
- Better visibility into request backlog (queue.qsize())
- Eliminates lock contention under high concurrency
Changes:
- worker_locks dict replaced with worker_queues and worker_consumers
- Added _start_worker_consumer() to create queue and consumer per worker
- Added _execute_on_worker() for actual worker communication
- Updated _cleanup_worker() to cancel consumer tasks
- Updated stop() to cancel all consumers before shutdown
Benchmark results (4 workers, isolated):
- throughput_10ms: 333 req/s, 0 failures
- overload_10ms (200 clients): 334 req/s, 0 failures
- All tests pass with perfect round-robin distribution
1. Split respawning logic from reap_dirty_arbiter() into manage_dirty_arbiter()
to avoid respawning during shutdown/re-exec (follows reap_workers pattern)
2. Reduce public API surface in __all__:
- Keep errors, DirtyApp, client functions as public
- Internal protocol helpers remain importable from submodules
- DirtyArbiter and set_dirty_socket_path kept for gunicorn core
- Use dirty_threads config for thread pool size (default: 1)
- Enforce dirty_timeout at worker level via asyncio.wait_for
- Heartbeat runs independently, not blocked by task execution
- Document thread safety and state persistence in docstrings
Introduce Dirty Arbiters - a separate process pool for executing
long-running, blocking operations (AI model loading, heavy computation)
without blocking HTTP workers. Inspired by Erlang's dirty schedulers.
Key features:
- Completely separate from HTTP workers - can be killed/restarted independently
- Stateful - loaded resources persist in dirty worker memory
- Message-passing IPC via Unix sockets with JSON serialization
- Explicit execute() API from HTTP workers
- Asyncio-based for clean concurrent handling
Architecture:
- DirtyArbiter: manages the dirty worker pool, routes requests
- DirtyWorker: executes functions, maintains state, handles requests
- DirtyClient: sync/async API for HTTP workers to call dirty apps
- DirtyProtocol: length-prefixed JSON messages over Unix sockets
- DirtyApp: base class for dirty applications
Configuration options:
- dirty_apps: list of import paths for dirty applications
- dirty_workers: number of dirty workers (default: 0)
- dirty_timeout: task timeout in seconds (default: 300)
- dirty_graceful_timeout: shutdown timeout (default: 30)
Lifecycle hooks:
- on_dirty_starting(arbiter)
- dirty_post_fork(arbiter, worker)
- dirty_worker_init(worker)
- dirty_worker_exit(arbiter, worker)
Includes comprehensive test suite with 164 tests covering:
- Protocol encoding/decoding
- Worker and arbiter lifecycle
- Client sync/async APIs
- Signal handling
- Error handling and timeouts
- Integration tests
setproctitle causes segfaults on macOS due to fork() safety issues
introduced in newer macOS versions. The mere import of setproctitle
can trigger crashes in forked worker processes.
Fixes#3021
* Python3: refactor returned traceback
Exceptions provide __traceback__ reference since Python 3.0
(and creating cyclic references has not been big deal since Python 2.2)
* --reload: publish entire exception, not just traceback
This is dangerous insofar as the exception text is more
likely to contain secrets than the quoted lines from traceback are.
However, the difference between the two is minor compared to the
primary danger of enabling this on a production machine, so focus
on that instead!
The CIDR network support added in 24.1.0 changed forwarded_allow_ips
and proxy_allow_ips from string lists to ipaddress.ip_network objects.
This broke external tools like uvicorn that expect strings.
This fix validates IP/CIDR format during config parsing but keeps the
string representation. Network objects are cached in Config methods
(forwarded_allow_networks() and proxy_allow_networks()) for efficient
IP checking without repeated conversions.
Also uses strict mode for ip_network validation to detect mistakes like
192.168.1.1/24 where host bits are set (should be 192.168.1.0/24).
Fixes#3458
On Linux, SIGCLD and SIGCHLD are aliases for the same signal number (17).
The SIG_NAMES dict iteration order can map to either name, causing
"Unhandled signal: cld" errors when workers fail during boot.
Fixes#3453
- Add docker/Dockerfile with non-root user and configurable environment
- Add GitHub Actions workflow to build multi-platform images (amd64/arm64)
- Publish to ghcr.io/benoitc/gunicorn on version tags
- Update documentation with official image usage examples
- Bump version to 24.1.0
- Add PROXY protocol v2 documentation to deploy guide
- Add 24.1.0 changelog with new features and bug fixes
- Update all docs.gunicorn.org URLs to gunicorn.org
Extend --proxy-protocol to accept version values (off, v1, v2, auto) instead
of being boolean-only. This allows explicit control over which PROXY protocol
versions are accepted.
Changes:
- Add InvalidProxyHeader exception for v2 binary header errors
- Add validate_proxy_protocol() validator with backwards compatibility
- Update ProxyProtocol setting with nargs="?" and const="auto"
- Add PROXY v2 constants (PP_V2_SIGNATURE, PPCommand, PPFamily, PPProtocol)
- Add _parse_proxy_protocol_v1() and _parse_proxy_protocol_v2() methods
- Update both sync (message.py) and async (asgi/message.py) parsers
- Add hex escape handling in treq.py for v2 binary test data
- Add test cases for v2 TCPv4 and TCPv6
Backwards compatible: --proxy-protocol alone (or True) maps to "auto".
Closes#2912
On keepalive connections, finish_request() sets the socket to non-blocking
for selector registration. When the connection is reused, handle() calls
conn.init() which returns early (already initialized) without restoring
blocking mode. This caused SSLWantReadError when WSGI apps read the
request body on SSL connections.
Fix by explicitly setting blocking mode at the start of handle().
Fixes#3448
